2016_12_02 visualization meeting

Data Management Camera-DM Visualization
#1

Bluejeans: 107541143
UIUC people meet in Loomis

Agenda:

  • Single extension headers
    ° How segment geometry (e.g., pre and post scan regions) is specified.

  • Short term (mid-January) goals. See the list in @jjt’s reply below.

Notes:

  • The Firefly people want to decommission (remove from Github) the old version.

  • The UIUC students have no problems with the new version.

  • @marshall will install the new version at SLAC soon.

  • Unless unforeseen problems arise, decommissioning will happen by the end of January. See @tatianag’s reply below.

  • Homer Neal is making 9-CCD (raft scale) images (noise only) at BNL. They are supposed to be mirrored at SLAC.

  • Action: @tony_johnson will make some images available for testing today.

  • At this time, his 9-to-1 extension conversion ignores the headers. We need to discuss what is a good header format.

  • Leftovers from the September SLAC-IPAC workshop:

  • Action: @jjt will complete a draft of the visualization Concept of Operations document.

  • Action: @marshall will post the “yellow box” architectural diagram on the workshop web page.

#2

Near-term (mid-January, or so) visualization goals:

We plan to have a visualization tool that incorporates, in one application, all the parts that we have prototyped. It will run on both the UIUC and SLAC servers.

  • Image display (all image display is done by Firefly):

  • Display latest image, with controls as described in the documentation.

  • Display an image by specifying its URI.

  • Multiple image display (in separate viewers).

  • Region specification and display:

  • By coordinates on the command line

  • Using the Firefly tool

  • By name (e.g., amplifier, CCD, raft). Also pre- post-, and over-scan

  • Enable/disable overlay of amplifier/CCD boundaries.

  • Enable/disable display of mouse coordinates and HW region.

  • Command entry:

  • All commands can be entered on the command line.

  • Commands that take region parameters can also be entered from the Firefly tool bar.

  • Commands can have default parameters, which are specified in a configuration file.

  • Commands implemented (details in the documentation)
    Except as noted, all command output appears in a box.

  • Create, clear, and delete box.

  • Show and hide (minimize) box

  • Show and hide amplifier boundaries

  • Create and delete viewer.

  • Pixel statistics in a region:

    • average_pixel
    • second_moment
    • hot_pixel (highlights hot pixels in the image viewer)

  • Histograms:

    • Pixel distribution in a region
    • second moment in each of a CCD’s amplifiers (perhaps)
    • Row projection of a pre- or post-scan region (perhaps)
#3

Histogram use cases:
Many will need region specification.

  • Pixel distribution.

  • In an area or line.

  • Spatial distributions of pixel values

    • Radially from a point.
    • y (or x) projection in a 2D region.
      Perhaps with baseline (eg, overscan or sky) subtraction.

  • Distributions of analysis results:

    • Noise by amplifier in a CCD.

    • hot/dead pixels by amp in a CCD.

    • History plots (e.g., noise vs time in each amp)

  • Comparisons with templates or other images:

    • Ratio of present CCD amplifier noise to previous values.
    • Average pixel in an amplifier vs image illumination (for gain measurement)
    • Pixel dependence on other metadata quantities

  • Features that may apply to many (or all) use cases:

  • x and/or y scaling (to zoom in; to ignore large bins; etc.)

  • Overlay of multiple histograms

  • Overlay of function on a historam (eg, results of fit to the distribution)

  • Axis labels

  • Priority:
    1: Pixel distribution in a region.
    2: Analysis results (eg, noise vs amp).
    3: x or y projection (eg, y distribution in an overscan region)
    4: Comparison with a template (subtraction, ratio, or overlay).

#4

Some of the items, related to IPAC, discussed at this meeting:

Can we decommission the old GWT based Firefly?

  • The students are working with new JS-based Firefly, and don’t expect any issues if we decommission the old GWT-based one.
  • Stuart built the new Firefly based software successfully on a laptop, the next step is to deploy it on the dedicated visualization server.
  • Stuart and Tony would like to prove that JS-based Firefly is fully functional on their visualization server before agreeing to decommission of the old Firefly. Hopefully, it can be done by the end of January.

TODO items from SLAC-IPAC meeting

@tony_johnson has reminded that we have a number of TODO items from the meeting, and asked if we have tickets for each of this items. He would like to refer people to Camera Image Visualization meeting with IPAC page to demonstrate what Camera needs from various teams. Unfortunately, with none of the action items completed the page is not very useful.

In particular, Tony mentioned the first item on the list: "Make gliffy diagram of “yellow box” architecture including I&T diagnostic cluster”, assigned to @gpdf and @marshall . Without this diagram it’s very hard to talk about anything else.

We need to make sure the TODO items are being addressed or at the very least we have tickets/stories for them.