Shipping bandpass information with LSST Data

Data Management
#1

Recently I touched off a great discussion on Twitter about how observatories can help astronomers link observations to bandpasses. This is critically important for panchromatic SED work that involves data from several observatories and instruments.

And it turns out there’s a VO standard for it

https://twitter.com/augustmuench/status/729728518163202048

I’m not sure of the status of this VO project; it seems to be called an ‘experiment’ at this stage.

If this is, or can become, a standard, then that’s awesome. In a world where all photometry/imaging includes a universal identifier/link for its bandpass, then it becomes so much easier to build standardized SED analysis and modelling tools.

The missing link is that observatories need to include these universal filter identifiers in their FITS headers and catalogs. This is critical because filters might get swapped for a single instrument. For example. CFHT/MegaCam changed their i-band filter. CFHT uses an ad-hoc identifier for their two i-band filters, but they have nothing to do with the VO identifier at http://svo2.cab.inta-csic.es/svo/theory/fps3/index.php?mode=browse&gname=CFHT&gname2=MegaCam

Given that we have lots of VO experts in DM, my questions are:

  1. Do we know if http://svo2.cab.inta-csic.es/svo/theory/fps3/svo.cab.inta-csic.es is a true VO standard?
  2. Can we include a VO-standard filter ID in the metadata of all our images and catalogs?
#2

There is also this one: http://voservices.net/filter/

But that one doesn’t mention a VO standard.

1 Like
#3

The answer to 2 is yes, provided such a standard exists (it might be in addition to our own internal name, of course). The bandpass information is already public; providing it to an external service doesn’t seem hard. We would already be expecting to provide it in some format as part of our metadata – additional formats are of course possible.

1 Like
#4

There are two levels:

  1. Standard bandpass.
  • On order 10-20 per instrument+telescope combination (optical).
  • Standardized to some nominal transmission function at a certain airmass with a certain amount of H20, ozone, and aerosols.
  1. Actual bandpass for this particular observation.
  • LSST will have 1 million bandpasses that will each be a function of position on focal plane.

We, as a community, should totally do a better job at #1. Doing so is relatively possible and conceivable and would be a tremendous benefit. The quality of these bandpasses varies significantly, but people do use something and so these should be published and referenced clearly.

We, as LSST, need to figure out how to do #2 well and to have provided to and educated the community about the tools to easily deal with #2.

2 Likes
#5

There are bandpasses and bandpasses, and a decision needs to be made as to how much detail is necessary. Some want quite a lot! For example, some are happy to accept that i-band on CFHT MegaCam is SDSS-like, others want to know whether it’s i1 or i2, and some want to fold in the efficiency curve for the CCD and the transmission of the atmosphere at the airmass of interest.

#6

I think it’s been recognized (@ivezic @RHL, right?) that we need to provide phi(lambda, t) + ZP (equivalent to transmission(lambda, t)) for each source for the users who really care, either as part of table in db or as a function that can construct this based on (time, x_camera, y_camera, alt_atm, az_atm).
It seems that that requirement has been difficult to capture - the information doesn’t show up in the DPDD (at least to a quick search) for example.
I don’t think there is a plan for how to implement this yet, as @mwv mentioned.

In broad terms, things that affect phi(lambda, t) + ZP are:

  • clouds (time (fast), alt_atm, az_atm)
  • atmospheric absorption - various coefficients, with various timescales (time(fast-slow), alt_atm, az_atm)
  • filter transmission (time (slow), x_camera, y_camera)
  • camera response - dust, stuff on dewar, QE response (time(med? fast?), x_camera, y_camera)

… so we could provide a function that constructs phi(lambda, time) + ZP based on other information that would be smaller to store than a full phi(lambda, time) + ZP for each individual source.

There’s more documentation about the variation of phi and ZP’s in LSE-180 and if you want to play with some of this in python code:

2 Likes
#7

I think the relevant documentation is in the IVOA Notes here and here.

For the uninitiated, a Note indicates that this was work which was performed in the context of a IVOA Working Group and which the authors thought was worth reporting on, but which hasn’t undergone any sort of review or standardization by the IVOA itself. If you like, it’s a recommendation by the authors of that particular note, rather than a recommendation by the standards body.

1 Like
#8

Thanks for the clarification @jdswinbank. I’m really encouraged by the discussion here. Thank you everyone for indulging me.

I think it would be fantastic if we could use our influence, in conjunction with other observatories like STScI and tool-makers like Astropy, to complete this effort to standardize bandpass metadata. As @mwv says, this would give us a first-order solution. Still, I think this first-order solution would go a long ways towards helping the current generation of SED analysis and modelling tools. Perhaps we can also expose the space-time dependent bandpass metadata through the same interface?

#9

LSE-180 is the controlling document here and describes in gory detail what will be provided (I think it’s safe to say we’ll probably know and provide more information about our bandpasses than any comparable survey before us :slight_smile: ). We should probably reference it from the DPDD as well, so it’s not forgotten (@ivezic take note).

PS: @ljones, can you make lse-180 publicly accessible? Docushare won’t let me view it w/o logging in.

1 Like
#10

This has been considered calibration data and so should be documented along with the other components of the Calibration Database (for which the time is now ripe).

1 Like
#11

We need to handle effective filter shapes that vary with position in the focal plane and with time (due to spatially inhomogeneous filters [cf. HSC!], spatially varying CCD QE [again cf. HSC in the g, but other vendors can be worse], and the atmosphere).

As described in e.g. https://github.com/lsst-dm/calibration/blob/master/calibrationPipelines.pdf we will be providing enough information for users to carry out photometry of an object of any specified SED (e.g. @mwv says it’s an SNe Ia and wants to redo the photometry); my intention is to choose a SED for each object based on its colours, and use this SED to construct the standard LSST fluxes.

#12

Hi

I have just discovered this conversation, so I’m sorry to revive the post so late.

I am responsible of the SVO Filter Profile Service and author of the two IVOA notes that were mentioned. So I just wanted to clarify some details.

Right now, there is only one standard in the VO about Photometry:
http://ivoa.net/documents/PHOTDM/index.html

The identifiers that we use for the filters at the SVO Filter Profile Service are not VO-standard. There is not (at least yet) a VO standard for this. We just label the filters in a way that we find adequate and those names are also used in other services and applications as identifiers for the filters.

What is standardized in the VO Photometry Data Model is just:

  • the possibility of building Filter Profile Services.
  • the existence of unique filter IDs within a Filter Profile Service.

By the way, are the LSST transmission curves already publicly available? If yes, can you help me to find them so that we can include them in the FPS?

1 Like
#13

The filter curves are available (@ljones would be a good reference). The nominal CCD QEs from the two possible vendors are probably also public – you need both, of course.

A quick glance through the proposal suggests that it’s a possible starting point. I see that you plan to include the atmosphere as part of the filter bandpass, so the use of a single date range will lead to a very large number of filters even if the atmosphere is statistically stationary. You also don’t seem to have thought about spatial structure in the filter transmissions/QE variation over the CCDs.

Where should we go? When LSST has defined its internal representation for effective filter transmissions we can tell you what would be needed in an IVOA standard to make it useful to us. I don’t have a timeline for this.