Recreating the LSST Science pipeline tutorial (gen 2) only using Generation 3 command line tasks and the pipetasks

Data Management
#141

Simon,
I found another anomaly that you may already be aware of.
In tutorial 7 the parm tract is spelled as trct in the “Note” section.
Cheers KSK

Also, Simon, I’m working through Step 7 of the Tutorial. I’m not clear if meas_coadd is accurate. I see that my execution has created a directory actually named coad_meas . Can you advise if this is what we want here?
See cut/paste below:

import os
from lsst.daf.butler import Butler
collections = [f"u/{os.environ[‘USER’]}/objects", f"u/{os.environ[‘USER’]}/meas_coadd"]
butler = Butler(‘SMALL_HSC’, collections=collections)

#142

OK. I’ll take care of those typos

#143

Thank you Simon…

#144

I just closely watched the Delegate assembly recordings from 09/24.
Is it convenient to locate and download the notebook that Yusra A. used , entitled 05_Intro_to_Source_Detection…
Thanks

#145

This notebook is at tutorial-notebooks/05_Intro_to_Source_Detection.ipynb at main · rubin-dp0/tutorial-notebooks · GitHub ; the other tutorial notebooks are in the same repo.

#146

thanks KTL. I’ve got it functioning locally using Jupyter notebook.
We know it uses test data at:
repo = ‘s3://butler-us-central1-dp01’
collection = ‘2.2i/runs/DP0.1’
Is there a convenient zip file or tarball with this data? or is that notion unrealistic (due to size)

#147

…even though I probably can use some of the gen3 tutorial raw fits files. Do you agree?

#148

You will not be able to access the DESC data but you can use the notebooks as a guide to help you experiment with the test repositories you do have access to.

#149

okay Tim, thanks.
And,
I presume there is no convenient zip/tarballs/etc of the S3 files…

#150

Correct. It’s a huge dataset.

#151

So far I have been successful in readadapting the Jupyter notebooks from the 09/24 delegate assembly to the smaller tutorial gen3 datasets. I keep working through the notebooks some each day…
I’m examining the output datasets from the coadd processing and do not know what a .hsp filetype is…While I get some hits on open web search, can someone explain what this is for LSST …?
Happy Turkey day.

#152

It’s a FITS file – you can tell this by using the file command on a file. hsp stands for healsparse. Running fitsinfo on a file I have here reports:

No.    Name      Ver    Type      Cards   Dimensions   Format
  0  COV           1 PrimaryHDU       8   (12288,)   int64   
  1  SPARSE        1 CompImageHDU     11   (3145728,)   float64