DM Highlights Week of 2015-11-11 to 2015-11-17

Data Management DM Notifications
#1

Accomplishments:

  • Added fixes throughout the stack for Mac OS X El Capitan compatibility including workarounds for security constraints and upgraded ActiveMQ messaging; documented what had to be done and why
  • Made the DECam-supporting camera-specific package (obs_decam) an official LSST product (though it is still under development)
  • Released infrastructure for publishing technical notes
  • Delivered initial prototype of new pipeline construction component (“SuperTask”) and associated command-line wrapper (“Activator”)

Other work:

  • Ported and fixed various changes from HSC fork: skymap, non-vignetted polygon bounds for exposures
  • Analyzed power requirements and rack layouts for FY16 equipment purchase
  • Converted some parts of SUI/T client code to React/Flux JavaScript framework
  • Fixed problems when fringe processing is only needed for certain filters
  • Tested memcached for Qserv objectId index; looks like it’s probably inadequate
  • Improved build infrastructure by dumping results of failed tests
  • Designed improved repository-local spatial image search mechanism

DM Leadership Team meeting

  • Agreed on refinements to DM Roadmap (LDM-240) to make it self-consistent and a reasonable projection of achievable milestones in line with external milestones; simulation requirements will be derived from this ASAP
  • Clarified roles and functions of people and documents; to be written up as a document policy
  • Identified needs for more cross-team interactions, architectural-level documents
  • Identified need for usability “czar” to identify issues, propose technical solutions, draft RFCs, do some coding
  • Exposed additional resource needs
  • Discussed possible disruptive technical risks deriving from hardware or software environment evolution (like many-core computing and the rise of AstroPy)
  • Came up with a number of other actions to improve teamwork and communication
  • Adopted proposal to create a Science Architecture Forum using the “RFD” timeslot (Tuesday 12:30 Pacific); to kick off in December 2015
  • Adopted (and implemented) “Implemented” state for RFC workflow
  • Developed ideas for sessions at DM All Hands meetings:
  • System architecture review including what is certain and what is just a workable initial baseline
  • Presentations from each team about plans for the next cycle
  • At least one hour per team for them to organize as they see fit, possibly with parallel tracks (e.g. apps and middleware/infrastructure)