Test/Validation data now available on the shared filesystem

Happy Monday!

This note is to let everyone know that the test data sets and validation data sets are now being made available on the shared filesystem. They will be updated nightly at around 02:00 local. For details see conversation in the ticket. There is also information showing up in the dev guide imminently.

The validation data sets will be maintained in /project/shared/data/validation_data and include:

validation_data_hsc
validation_data_decam
validation_data_cfht

The test data sets will be maintained in /project/shared/data/test_data and include:

testdata_jointcal
testdata_cfht
testdata_subaru
testdata_decam
ap_verify_testdata
ap_pipe_testdata
ci_hsc
afwdata

These repositories are intentionally read only for normal users. They are maintained by the lsstsw user. If you want to do branch work on any of these repositories, you will need your own copy, though I believe you can copy these to your personal space rather than cloning your own from github via git-lfs.

Please put any questions you have in the comments. I welcome suggestions for wording for the developer guide. I hope to merge that early this week.

2 Likes

Isn’t there a new obs_lsst test dataset that’s of order 20GB? Can you please add that one?

Very good point. It will show up tomorrow morning.

1 Like