Qs about Import error running ingestImagesDecam.py and location of raw and calibration fits files

Support
#1

Dear LSST Support Forum,

I am a postdoctoral researcher at KASI, and trying to run LSST pipeline using Docker (lsstsqre/centos:7-stack-lsst_distrib-v19_0_0) to process DECam data (https://github.com/lsst/obs_decam). I’m having an error saying

Traceback (most recent call last):
File "/home/lsst/mnt/obs_decam/bin/ingestImagesDecam.py", line 2, in <module>
from lsst.obs.decam.ingest import DecamIngestTask
File "/home/lsst/mnt/obs_decam/python/lsst/obs/decam/__init__.py", line 5, in <module>
from .instrument import *
File "/home/lsst/mnt/obs_decam/python/lsst/obs/decam/instrument.py", line 31, in <module>
from lsst.obs.base.gen2to3 import AbstractToPhysicalFilterKeyHandler, TranslatorFactory
ImportError: cannot import name 'AbstractToPhysicalFilterKeyHandler' from 'lsst.obs.base.gen2to3' (/opt/lsst/software/stack/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_base/19.0.0/python/lsst/obs/base/gen2to3/__init__.py)

when I run ‘ingestImagesDecam.py -h’ on the step 3 and 4 on README.md on GitHub page.

Also, I could not locate any raw or calibration fits files except an array of DECam mosaic under decam/camGeom/ in the GitHub repository.

Am I missing something?

Thank you for reading,

Duho

#2

You appear to be using a master version of obs_decam (or at least one since April 7) along with a v19.0.0 version of obs_base. This is not expected to work and is causing your error. You likely should be using the v19.0.0 version of obs_decam. You can get this by simply saying setup obs_decam in your container. This replaces all of step 2 in the README.

If you need to modify the code, you can clone it as in the README, but you will then need master versions of many other stack packages (or you will need to do git checkout 19.0.0). I should emphasize that modifying this code can be a challenge even for experts.

obs_decam is the software and configuration that tells the Rubin/LSST Science Pipelines how to deal with DECam data. It does not provide the data itself. You will need to obtain that elsewhere, although a limited amount is available in https://github.com/lsst/validation_data_decam, and https://github.com/lsst/testdata_decam. (The defects files in obs_decam_data should automatically be made available when you setup obs_decam.)

I have created an issue (DM-24913) to ask someone to improve the README file.

#3

Indeed, the error message was gone when I ran ingestImagesDecam.py -h after I run setup obs_decam. But I still don’t know why.

At first, I tried setup -t obs_decam -r . following the README.md instruction, which gave me an error Ignoring unsupported tags in VRO: obs_decam setup: Unsupported tag(s): obs_decam,

So I tried setup -j -r obs_decam next following the HSC tutorial (https://pipelines.lsst.io/getting-started/data-setup.html), which gave me the error that I shared above.

For the data, I’m downloading from the first link that you shared, and I’ll update the progress on this page. Thank you very much!

#4

setup obs_decam sets up the “current” version, which in the container is 19.0.0, consistent with the rest of the stack.

setup -t obs_decam -r . tries to setup the local version you checked out (which is incompatible), along with dependencies that are tagged with the label “obs_decam” (which doesn’t exist).

setup -j -r obs_decam sets up the local version you checked out (which is still incompatible).

For a little bit more information on eups and setup, you might look at https://pipelines.lsst.io/install/setup.html
If you execute setup lsst_distrib as suggested there, you should not need to separately setup obs_decam.

1 Like
#5

If you’re using version 19.0.0 of the LSST stack, you should use version 19.0.0 of obs_decam. In the obs_decam git repository:

git checkout 19.0.0

will give you version 19.0.0.