Verification Datasets telecon
Attendees: Simon, David, MWV
Regrets: Colin
Simon:
- trying to make a repository that is agnostic to camera
- put files in repository and run processCcd on it (must have previously ISRed)
- just need camera info for ISR
- need to know variance pretty well otherwise CR rejection falls apart
if background subtracted then you need to give the noise level (scalar) - just assumes a single 2D grid of flux values, no variance or mask plane, but could add that in
- giving people the ability to use processCcd like SExtractor
- it internally bulids the variance (using gain, config parameter) and mask plane
can also set lower threshold for “good” values, also needs to know the noise level - uses repository, can use Butler to get the source catalog, etc.
- could actually hide the butler by adding a little bit of code that checks if the
file exists in the repository and if not then it adds it - currently a branch under SimonKrughoff obs_file
tickets/DM-6924 branch of both obs_file and pipe_tasks
c.l.o post about this - probably useful for the collaborations
David:
-
had meeting with PanSTARRS people at MPIA and discussed QA, mostly done after the fact to see what issues there are (forensic mode)
-
Simon, discussion of prompt processing, send out alerts on 99.9% of images that aren’t corrupted by any of the processing
-
image also needs to be scientifically useable, who makes this choice
-
maybe use OCS flags?
-
need red light/green light on Level 1 products
-
NCSA is putting up a verification cluster and they wanted to know what workflow people want,
slurm and condor a possibility -
SQuaRE doesn’t want to much effort on developing a new workflow system
-
David, want to run PSFEx on COSMOS to see if the PSF model and photometry is better
-
CR, problems on DECam COSMOS images, but SK ran some it found ~80% of the CRs
-
is CRblaster work better, CP uses it to identify and mask them
should compare LSST to CRblaster -
MWV suggested having a CI test to find a certain number of CRs in an image
-
issues with variance plane can cause problems with the CR software, was an issue with
CP-processed DECam data in the past, wanted to double-check with LSST ISR but never quite got there
MWV:
- planning improvement for LDM-151 on SDQA
- a lot of pieces in SDQA still need to be designed