1 April meeting

Data Management Camera-DM Visualization
#1

Agenda:

  • Status of code in main branch (does it work at SLAC?)
  • Report from BNL (@tony_johnson).
  • Generic utility functions, useful for many use cases (eg, ROI specification).

Notes:

  • CCS system is installed at BNL. CCS has access to images. They will want
    visualization soon. Tony will have more time to help with this.

  • Code in the Main Github branch has not been tested since update.

  • The image location is hard-coded in the back-end code. We need to make image access more flexible.
    Action: SLAC or BNL needs to write a server that passes the location of the latest image to the back-end code.
    The user needs to be able to pause updating of the image display.
    Gregory says: The FF people could write a custom iimage search utility. This is not the highest priority now. It would require a F2F meeting to architect it.

Actions: for @jjt: Talk to @tony_johnson Tony about:
  ° Image access at BNL.
  ° Also, we need to have one of the test stand files for code debugging.
  ° What other amplifier metadata (besides the WCS) we need to persist in the combined ( single extension) FITS file.
  ° There is not yet a reference file that conforms to the standard Camera FITS format.

  • Status of the single-extension FITS file:
    Conversion of the test-stand 16-extension FITS file (per CCD) works, but is not done very efficiently (all in memory). It’s good enough for single CCDs, but needs to be improved (perhaps) when we do a whole raft.
    Action: @weiren2: Improve the documentation to make the code more easily usable.

  • DS9 does not (AFAIK) do gain and offset correction by amplifier. We may need to do this in the back end.
    We’ll also want to subtract the bias (zero-time exposure) image. The overscans may have some of this information (not pixel-by pixel).
    Action: @JJT: Describe the various image display options that the TS8 folks will want.

Implementation priority:
1: a. Get access to the latest TS8 image
    b. Display it with and w/o overscan.
    c. Pause button.
    This is the MVP (“minimum viable product”).
2: Apply amplifier gain and offset corrections.
3: Other stuff:

  • ROI utility, for use by multiple user commands. We need to specify the UI, to accommodate both command line and FF toolbar ROI selection. We want to display the selected ROI on the FF image display.
  • Live readout of cursor position and amplifier region in image coordinates. WCS is not currently selectable. FF picks one.

Action: @gpdf Make sure that Tatiana and Xiu Qin (and other team members) have access to SLAC Confluence.

#2

Hi Jon, unfortunately I will be driving to Newark this morning to return to SLAC from BNL, so will be unlikely to make the meeting. We did have a successful install of DAQ and CCS during the week, Paul OConnor mentioned many times his desire to see image displayed as soon as possible, so now that the basic test stand is working I hope I will be able to devote more serious time to visualization issues.

#3

@xiuqin @tatianag Documentation of the FITS “alternate axes” feature:

http://www.aanda.org/articles/aa/full/2002/45/aah3859/aah3859.right.html#SECTION00025000000000000000

This is what lies under the approach Stuart has recommended be used to preserve a hierarchy of coordinate systems at the amplifier, CCD, and raft level. It facilitates laying out a mosaic in generic FITS-viewing tools.

@marshall, can you reply with a pointer back to your recommendation?

#4

Here is the current page on fits headers, it is a work in progress so will change.
https://confluence.slac.stanford.edu/display/LSSTCAM/Specification+of+Raft+and+Focal+Plane+FITS+Images

#5

Thanks, @marshall, that’s helpful.

#6

I saw the comment,

DS9 does not (AFAIK) do gain and offset correction by amplifier

I would not expect us to rely on any display code to do things like this. There’s DM code to do all of this stuff (trim, overclock, bias image, dark, linearity, bad columns, …) for you, and I think it’d be a good idea to use it before pushing the images off to the display.

If you have problems with our interfaces (or anything else!) please complain and we’ll try to fi them. I am concerned that there are many implementations in the camera team of this sort of thing, and DM is going to have to try to extract the expertise at some point. It’d be much better to share code, and let the camera fix artefacts the way they want – but in a way that is automatically captured.

#7

Robert,

It would be very helpful if we could use existing DM code for these functions. Will this require that our test stand SW incorporate (pieces of) the DM environment?

Jon

#8

Yes, of course. But I think you should expect to do this anyway!

Also, it’s now relatively easy to do an install (including binary installs, although I haven’t done so)

#9

I put a set of 9 16-extension fits files written by the CCS image-utilities at slac.stanford.edu/~marshall. They are to be considered as a good example of what to expect from test-stand 8 but are a work in progress. They have a simulated pattern. I can update these with new ones as changes are made. In particular it may be good to have a different pattern and there will be some wcs keyword updates as well. Eventually we may be able to add gain, bias and noise.