It looks like one of the remaining issues on Testing DM-4692: the new ProcessCcdTask is astrometry problems on CFHT, due to some combination of the following:
-
We don’t have a good distortion model for this camera, making it likely we’ll get bad matches unless the reference catalog and our own measurements are sparse.
-
We’re matching a deeper set of detections to the reference catalog on DM-4692 and deblending them, making our own measurements much less sparse.
-
We don’t have enough flexibility in filtering our own measurements before we feed them to the matcher.
-
Our fitter doesn’t do any outlier rejection, making it very susceptible to bad matches.
While we really need to fix at least one of (1) and (4) (@price, @rowen, and @boutigny are working on this now), and (2) should be an improvement in the absence of these other problems, I’d like to focus on (3), which may be the easiest way to get things working for now, but which also raises some code duplication and interface issues.
We have multiple operations that need to select a sample of reasonably bright, unsaturated, possibly isolated stars in single-frame processing: PSF estimation, aperture correction estimation, astrometry, photometric calibration. Eventually, external catalogs may play a role (that’s a separate issue I don’t want to deal with now), but we’ll always need to do some filtering based on our own measurements, rejecting objects due to some flags or other cuts, or possibly selecting objects due to some more complex criteria (“the brightest N objects in each spatial cell”).
I think we want some sort of unified system for specifying these sort of selections, which we could view as a generalization of the StarSelector interface. The problem is code reuse: do we want to have a selector that checks for condition A, another that checks for condition B, and two more to check for “A and B” as well as “A or B”? Some sort of expression parser would perhaps be the most natural approach for simple filtering, but even if we did want to bite off writing one, some important operations can’t be written out easily in a single line. Passing callable objects (sometimes lambdas, sometimes more) into configuration would be a a really slick approach, but I strongly suspect that will make config persistence a nightmare, and probably raise some security concerns.
Right now, I think just having a plugin system with a few multi-purpose general implementations (e.g. “AND together a lis of flags”), along with letting plugins compose themselves via actual class composition seems to be the approach with the lowest activation energy. Any other ideas?