With the merger of DM-17029 and DM-10156, the LSST Science Pipelines now provides calibrated fluxes in units of nanojansky (nJy), internally uses reference catalogs in nJy, and uses afw.image.PhotoCalib to manage the photometric calibration of images, replacing afw.image.Calib. This post summarizes the changes to reference catalogs as well as the API changes involved, including where we did and did not provide backwards compatibility, and provides some suggestions how how to transition code to use the new objects.
Reference Catalogs
New refcats will now be in nJy
IngestIndexedReferenceTask now generates reference catalogs with flux units of nJy; similarly the schema returned from LoadReferenceObjectsTask.makeMinimalSchema has nJy flux units. Indexed reference catalogs now have format_version=1, and unversioned catalogs (with jansky flux units) are assumed to be format_version=0. In addition, new reference catalogs will have their units explicitly specified as units=nJy, so that you can check the units on the catalog fluxes directly.
Custom code that previously assumed the fluxes in reference catalogs were Jy will now return values that are off by 1e9. Everything tested by lsst_distrib, lsst_dm_stack_demo, and ci_hsc has been converted to treat fluxes correctly.
As we are phasing out Astrometry.net reference catalogs, we do not provide a way to generate new a.net-style catalogs with nJy fluxes.
Converting old reference catalogs
We provide backwards compatibility with old reference catalogs (both HTM Indexed and Astrometry.net) by converting them in memory when they are loaded from disk. If you use version 0 reference catalogs, you will see warning messages in the logs about this conversion, including a description of how to convert your catalogs on disk to have nJy fluxes. You can convert your on-disk HTM Indexed catalogs via the command-line tool meas_algorithms/bin/convert_refcat_to_nJy.py: see its help description (via running with -h) for more details.
In the next few weeks, we will provide converted versions of the existing gaia, ps1, and sdss catalogs on lsst-dev. Watch for an announcement.
PhotoCalib replaces Calib
Overview of PhotoCalib
The PhotoCalib object can contain a position-aware afw.math.BoundedField, which allows for spatially-varying photometric calibrations (as produced by Jointcal), in addition to non-spatially-varying “mean” calibrations (as produced by PhotoCalTask). PhotoCalib is mathematically defined as:
instFlux [counts] * calibration(x,y) = flux [nJy]
PhotoCalib provides methods for:
- converting instrumental fluxes and errors (
instFlux,instFluxErr) to AB magnitude (instFluxToMagnitude) and flux (instFluxToNanojansky). These methods take inputs as either scalars orSourceRecordsandSourceCatalogswith a flux field name. If you pass bothinstFluxandinstFluxErrto the conversion functions, you will get back aMeasurementobject containing thevalueanderror(think of it as anamedtuple). - efficiently converting an entire image to calibrated fluxes (
calibrateImage). - converting an AB magnitude to instrumental flux (
magnitudeToInstFlux). - other methods for extracting various components of the calibration (
getCalibrationMean,getCalibrationErr,getInstFluxAtZeroMagnitude,computeScaledCalibration).
In general it is strongly preferred to use the provided conversion functions rather than extracting numerical parameters and doing conversions by hand.
PhotoCalib is a C++ object, so you can find the detailed documentation on the LSST doxygen page.
Deprecated interfaces and behavior changes
To ease the transition from Calib to PhotoCalib, I’ve added deprecated interfaces that behave like the old Calib methods, including:
-
getMagnitudeto convert instFlux to AB magnitude, andgetFluxAB magnitude to instFlux. Instead useinstFluxToMagnitudeandmagnitudeToInstFluxfor these conversions. These new methods can take a wider range of arguments, and have clearer names. -
getFluxMag0now always returnsNaNfor the second element (previouslyfluxMag0Err). Instead usegetInstFluxAtZeroMagnitudeand/orgetCalibrationMean/getCalibrationErr, depending on your exact needs. -
PhotoCalibis immutable, which means that thesetFluxMag0method that was previously used to create a newCalibobject is no longer valid: calling it will raise aRuntimeErrorwith a description of how to create aPhotoCalib. -
PhotoCalibwill never raise an exception when computing magnitudes from instrumental fluxes: zero or negative instFlux values transform to NaN magnitudes. I have added deprecatedsetThrowOnNegativeFluxandgetThrowOnNegativeFluxmethods to PhotoCalib as no-ops. In addition, because PhotoCalib does not special-case negative instrumental fluxes, calling e.g.photoCalib.instFluxToMagnitude(instFlux=-5.0, instFluxErr=1.0)will result inNaNfor magnitude, but a real, positive value for magnitude error, instead of returningNaNfor both. -
getCalibonExposureandExposureInfois deprecated in favour ofgetPhotoCalib; both will return thePhotoCalibassociated with that exposure. - Use
butler.get("calexp_photoCalib", dataId)to get thePhotoCalibof an exposure if you do not need to load the entire exposure from disk.
Due to the timing of merging DM-10156 and adding the deprecated package to our conda environment, the deprecated interfaces described above do not yet emit warnings at the python level (they should emit C++ compiler warnings). These warnings should start appearing next week, once @ktl and I sort out exactly how to implement deprecation warnings for pybind11 objects.
Using astropy for unit conversion
To convert between nJy and AB magnitudes (or other flux units), we recommend using astropy.units. For example, you can see that 575 nJy is approximately the 5-sigma 30s imaging depth of LSST, and the AB magnitude zero is nearly, but not exactly, 3631 Jy as it is often hard coded to be:
>>> import astropy.units as u
>>> (575*u.nJy).to(u.ABmag)
<Magnitude 24.50083039 mag(AB)>
>>> (0*u.ABmag).to_value(u.nJy)
3630780547701.003
The existing afw.image functions abMagFromFlux and fluxFromABMag will be deprecated in favour of the above at some point in the future. Be aware that they assume fluxes in janskys, not nanojanskys, and use the imprecise 3631 Jy reference flux value. The error conversion methods abMagErrFromFluxErr and fluxErrFromABMagErr do not yet have obvious replacements from astropy. See DM-16903 for details.