TypeError: runDetection() missing 1 required positional argument: 'expId'

Support
,
#1

Hi,

I’m trying to stack some HSC data using the LSST stack, and while running coaddDriver get lots of repeated errors like this:

142540 INFO  2018-03-05T18:29:41.167-0600 coaddDriver: trantor:142540: Start detection on DataId(initialdata={'tract': 0, 'filter': 'HSC-Z', 'patch': '2,3'}, tag=set())
142540 WARN  2018-03-05T18:29:41.182-0600 coaddDriver: trantor:142540: Caught TypeError while detection on DataId(initialdata={'tract': 0, 'filter': 'HSC-Z', 'patch': '2,3'}, tag=set()): runDetection() missing 1 required positional argument: 'expId'
142540 INFO  2018-03-05T18:29:41.187-0600 coaddDriver: trantor:142540: Traceback:
Traceback (most recent call last):
  File "/opt/lsst/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/14.0-1-ga2912ff+25/python/lsst/ctrl/pool/parallel.py", line 496, in logOperation
    yield
  File "/opt/lsst/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_drivers/6.0b0-hsc-13-g219e95d+3/python/lsst/pipe/drivers/coaddDriver.py", line 315, in coadd
    detResults = self.detectCoaddSources.runDetection(coadd, idFactory)
TypeError: runDetection() missing 1 required positional argument: 'expId'

For background information, I’m running LSST version w_2018_04, installed via the new_install.sh method on a SuSELinux Xeon Phi system. Data have been reduced using the singleFrameDriver method, followed by makeDiscreteSkyMap, and the full command for coaddDriver is listed below.

coaddDriver.py $(HOME) --rerun run1 \
--id tract=0 filter='HSC-Z' \
--selectId expTime=240 filter='HSC-Z' \
--job=stack_z --cores=230 \
-c makeCoaddTempExp.doApplySkyCorr=True \
--config assembleCoadd.doApplyUberCal=False makeCoaddTempExp.doApplyUberCal=False \
--clobber-config

(Note: I’ve added the --clobber-config to avoid problems with the missing meas_mosaic package).

Checking the already created output, all warp’ed and psfMatchedWarp’ed files seem to exist (at least there’s a whole lot of them), just not a single calExp frame yet.

Any help on how to fix or work-around this would be much appreciated. Let me know if you need more program output, system vars, etc.

Thanks
Ralf

#2

Is there any chance your versions of pipe_drivers and pipe_tasks are inconsistent? I believe there was an API change in pipe_tasks to add that expId argument, and pipe_drivers was modified accordingly to provide it.

#3

I’m not sure how to get the version info for installed packages, but so far all I have installed LSST-wise was done in a single run, so I wouldn’t expect any major version conflicts.

From what I gather from online-docs, I can use eups list to get versions, correct? If so, here’s what I get for the packages you mentioned:

:~> eups list pipe_drivers
   6.0b0-hsc-13-g219e95d+3      current d_2018_01_29 w_2018_04 w_latest
:~> eups list pipe_tasks  
   14.0-39-g03bf09b5+1  current d_2018_01_29 w_2018_04 w_latest

Using setup -v xxx I get pretty much the same info:

:~> setup -v pipe_drivers
Setting up: pipe_drivers                    Flavor: Linux64    Version: 6.0b0-hsc-13-g219e95d+3
...
:~> setup -v pipe_tasks  
Setting up: pipe_tasks                      Flavor: Linux64    Version: 14.0-39-g03bf09b5+1

Against my expectation they seem to be different, just as you expected. Any recommendations on how to update them to make them match - e.g. just run eups distrib install -t latest (I assume data is largely compatible between versions and I don’t have to start data processing all over)?

#4

Actually, I think that information does indicate that they’re probably consistent; it’s actually those tags (e.g. w_2018_04) that indicate the version numbers that we expect to be the same for different packages.

Just to make sure, once you have set up all of the packages you’re going to use, could you run:

eups list -s

to list which packages have been setup? You’ll hopefully see the same w_2018_* number next to all packages.

#5

Done that, and at least the simplified version

eups list -s | grep -v w_2018_04

gives no output, so all loaded packages contain the w_2018_04 tag.

For now I’m running the stack with the coaddDriver .... --config doDetection=False additional option that I could tease out of the source code (Are all these options documented somewhere? At least I couldn’t find them) to disable the problematic part of the code that throws the exception, but in the long run having a source list would be nice (if only to compare to and verify what I find otherwise).

Thanks
Ralf

#6

Hmm…I just looked at what’s actually in the w_2018_04 tag and it looks like those packages are actually inconsistent there. I have no idea how that happened (and moreover why I don’t recall this being a more general problem before, as we would certainly have tried to run coaddDrivers.py on that release).

In any case, I’m afraid it seems like that particular tag may just be broken. I’d recommend jumping straight to the latest, w_2018_09; we’ve been making a lot of changes to the detection and coaddition code over the past few weeks, and I think they’ve just now settled down.

#7

I think w_2018_04 is the release that got messed up.

#8

Ha, lucky me I guess. I always seemed had the “good” fortune of picking problematic cases for testing…

Anyway, I’ll update to the release you mentioned once the current job is done and report back if it’s still an issue.

Thanks again for the quick help.
Ralf