Do we need a high-level "CameraExposure" object?

Data Management
, , ,
#1

After discussing some of the metadata questions in DM-5502, @rowen and I began to wonder if we don’t need a “CameraExposure” object, that contains all of the Exposures and some extra high level metadata (e.g. telescope boresight, date/time of exposure, optical distortions and seeing, etc.). At present we have a CameraGeom object which provides nominal information about some things, but we don’t have an object that holds all of the exposures from a single visit.

For the sake of memory management, CameraExposure would have to have some “lazy loading” of exposures or some smart pointers, so that it could depersist the top-level metadata and, say, the individual Exposure metadata independent of the pixels.

Conceptually, this would be a useful thing for jointcal/meas_mosaic, as we have things we want to fit on the per-Detector level (tip, tilt, twist, shift, scale), and things to fit on the whole-camera level (optical distortion and possibly seeing distortion). Being able to store and track the latter outside each individual ExposureInfo seems like a good thing.

I don’t have a real design for this new thing, and was not happy with the first couple things I sketched out, but I’m wondering if such a beast was rejected at some time in the past, or if we never had a need for it? Certainly, there’s plenty of metadata that more appropriately lives above the ExposureInfo or CameraGeom level.

1 Like
#2

I think I’m correct in assuming that when you talk about “Exposure” here, you’re talking about a CCD-level data product. Right? In general, we use the Exposure class to contain effectively any masked image together with metadata, which can include warps, coadds, etc. My understanding is that you are effectively asking for a single object through which associates Exposures describing all of the CCDs from a full field of view.

Given that, there’s scope for a container at this level: during DRP, we expect to associate certain data with the field of view (PSF, WCS, background, …) and others with the individual detectors (image, mask, variance, …) [tip of the hat to Jim’s text in the current LDM-151 draft]. This seems consistent with that vision. We’ve not yet finished thinking through the detailed design implications, though.

#3

I think we’ve just never had a need for it; I can’t recall any past failed attempt. I share both your sense that this is something we want and that it may be something hard to design.

#4

@swinbank Yes, you’re correct: a single object that contains all the CCDExposures (currently Exposures) for a current visit, or snap, or whatever.

I’m now coming up with even more use cases for this high level object: jointcal and its tests need to associate stars between visits, but all of the source catalogs are per-CCD which makes building those associations more complicated than necessary.

Do we have any high-level “per visit” catalogs at present, or am I stuck bundling all the source catalogs per visit manually?

#5

I suspect you’re stuck; I don’t know of any other contexts where we bundle up the source catalogs for a visit.

#6

I don’t understand why this seems so difficult. The Camera object provides an array of detector objects, so why not just bundle that, an array of calexp_md and an array of src catalogs into a Struct and be done with it?