The DM monthly status report covering May has been posted in Docushare, Collection-4640. For your convenience I am pasting high-level summary and project status below. Direct link to the full report (pdf): http://ls.st/rhx.
High-level Summary
The DM Team closed the X16 cycle, and produced the rolling wave plan for the F16 cycle. The team continued refining design documents and construction plan. Prototype of QA service has been released, major progress with comparison of HSC and LSST software stacks have been made, noise correction algorithm for difference images have been implemented, LCR-606 has been recommended for approval, major improvements to Qserv shared scan scheduler have been implemented, and the Firefly code conversion goal has been achieved.
We continue refining the plan, and started preparing for the upcoming reviews and the LSST2016 Project meeting.
Project Status
The DMLT Team met in Tucson to discuss short- and long-term plans and priorities. Following that meeting, detailed plan for the Fall 16 (F16) release cycle has been produced.
The X16 cycle has been completed, detailed statistics will be compiled in June.
The team prepared for the internal Verification Review, and started preparing for the Directors’ Review and the joint NSF/DOE Status Review, as well as for the LSST2016 Project meeting.
The team continued an effort to refine and improve the level of details present in design documents and the construction plan. We also continued working on previously scheduled activities as planned, no major deviations from the plan has been identified.
Selected technical accomplishments:
- A subset of the DM Team has been working on a thorough revision of LDM-151 (DM Science Pipeline Design Doc).
- SQuaRE has released a minimal version of SQuaSH, the prototype of its QA service. It provides a dashboard of three Key Performance Metrics from thrice-daily builds of the LSST stack’s master branch.
- Data Release Production team made major progress on comparison of the Hyper Suprime-Cam and LSST software stacks, with a view to future HSC development taking place on HSC. Kron photometry algorithm as well as high-level distributed measurement drivers has been distributed with the LSST stack.
- Alert Production team implemented the noise correction algorithm for difference images. They can now run simulated images with realistic DCR signal through the image differencing pipelines. The team has also made a recommendation for generalized coordinate transform framework available.
- In the area of International Communications and Base Site, (1) LCR-606 was recommended for approval by the CCB. This reconciles the LSE-239 Base Center requirements document and the cost estimates for increasing the power and cooling feeds to allow for future spare racks and equipment, the increased unit cost of power and cooling, and the incorporation of seismic isolation. (2) The 10G equipment in order to provision the La Serena - Santiago 4G link was received and installed. The link was configured to rate limit the bandwidth to 4Gb (LSST’s allocation). Tests were run between La Serena and Sao Paolo whereby 3.2Gb was achieved which was free bandwidth at the time of test. And (3) by May 25th, for the first time, the 100G link connecting Santiago and Miami was running error-free. All tests were performed using the IXIA 100G packet generator. Using the IXIA 100G packet generator, a constant packet flow rate of 8.600.000 packets per second, using standard MTU (1522 Bytes) was created. This packet rate forced all interfaces to run at 100% of their capacities or 98.9 Gbps.
- Science Data Archive and Application Services Team implemented improvements to Qserv related to handling of large (multi-gigabyte) result sets. New Qserv shared-scan scheduling with page-lock assist is now under test on the IN2P3 test cluster. Preliminary results show substantial performance gains (~50% improvement over performance with no page-lock assist) though some technical details of the implementation remain to be worked out.
- Science User Interface and Tools Team reached their goal of finishing 90% Firefly client side code conversion from Java/GWT based to JavaScript/React/Redux based.
- The Science User Interface and Tools, Data Access, and Science Data Archive and Application Services teams proposed a plan for a coordinated Prototype Data Access Center (PDAC) deployment in the F16 and subsequent cycles. This deployment will exercise DM subsystem interfaces and will expose the Data Access query capabilities and elements of the Science User Interface and Tools to scientific users. The proposed plan to use the 2013 reprocessing of the SDSS Stripe82 data for the initial PDAC dataset in the F16 cycle has been approved by DMLT. The WISE infrared space-based survey’s public data will be added to the PDAC in the S17 cycle to test this at a substantially larger scale.