Thanks a lot Simon, it looks awesome!
We included some sources below the detection level on purpose in order to see how they affect so, maybe it’s a combination of your choice of noise and these sources.
@fjaviersanchez I should note that I accidentally did this work on a private fork. You’ll need to clone from here for the time being to use it.
I don’t think it’s crazy, but it might be a little tricky to do. The background should be fairly easy to attach, but your PSF will have to be represented using our classes. Maybe @jbosch has more to say about how to mock the exposure so it can be fed directly to the deblender.
I think it’ll take a lot of trial-and-error work to convince our PSF modeling code to work well on fairly artificial inputs like this; it’s really designed to expect a full CCD of star and galaxy images. Over in the other thread on this topic, I’ve posted some instructions for trying to run the relevant pieces of the pipeline on data like this, which doesn’t use processCcd.py at all.
Another option would be to follow those instructions for how to attach a PSF to one of our Exposure objects, save that to FITS, and then run processCcd.py on that with all of the PSF modeling steps disabled, but I still think this is likely to be harder than just starting from the snippet I’ve linked to.
Hi Simon. I was trying to follow your steps to run processFile on a simulation image, and I may have some problem when I tried to build pipe_tasks-tickets-DM-6924:
cd pipe_tasks-tickets-DM-6924
setup -k -r .
scons opt=3
…
The following tests failed: ~/miniconda2/opt/lsst/pipe_tasks-tickets-DM-6924/tests/.tests/testProcessCcd.py.failed ~/miniconda2/opt/lsst/pipe_tasks-tickets-DM-6924/tests/.tests/testTransform.py.failed ~/miniconda2/opt/lsst/pipe_tasks-tickets-DM-6924/tests/.tests/testPhotoCal.py.failed ~/miniconda2/opt/lsst/pipe_tasks-tickets-DM-6924/tests/.tests/testCoadds.py.failed 4 tests failed scons: *** [checkTestStatus] Error 1 scons: building terminated because of errors.
I am not sure where the problem is (obs_file-tickets-DM-6924 could be built without error). FYI, I installed Stack v12.0 with miniconda on a CentOS6 machine. Please let me know if you need any other information. Any hints would be appreciated!
It is possible that v12.0 is too old. Things on master move along pretty quickly and I made no attempt to be compatible with the most recent release. You could install master of everything and try again. Is that an option for you?
If you look in the .failed files, it will give you the traceback. That would also be useful information.
Chiming in here, it looks like following master would make sense for me as well, but the last time I tried the lsstsw installation I got this:
You are attempting to run "setup" which requires administrative
privileges, but more information is needed in order to do so.
Authenticating as "root"
Password:
Since I don’t have root access to the compute servers, I couldn’t use it. Is there a switch to do per-user installs?
Unfortunately, the eups setup command shadows a shell command. This results in the above error if you do not have the eups bin directory in your path. @josePhoenix what steps did you follow the ended up in that error. If you first run $> bin/deploy in your lsstsw install, it should put all the right things in your path.
Thanks for your reply!
I just tried to reinstall the master with miniconda, but the same test failures came out again. I looked into the .failed files:
testProcessCcd and testTransform both have the same error:
AttributeError: 'module' object has no attribute 'starSelectorRegistry'
testPhotoCal has the error:
TypeError: __init__() got an unexpected keyword argument 'refObjLoader'
and testCoadds has:
ERROR: testAlgMetadataOutput (__main__.CoaddsTestCase) Test to see if algMetadata is persisted correctly from MeasureMergedCoaddSourcesTask. ... ERROR: testForcedPhotCcdTask (__main__.CoaddsTestCase) ... ERROR: testForcedPhotCoaddTask (__main__.CoaddsTestCase) ... FAIL: testLeaks (lsst.utils.tests.MemoryTestCase) !Check for memory leaks in the preceding tests AssertionError: Leaked 74 blocks
Would you suggest any clue about these errors? I was wondering if it is possible to install the version of Stack that is compatible with tickets-DM-6924.
Hmmm. It looks like your stack is somehow inconsistent. It’s possible the ticket has fallen behind master. I will take a look and see if I can reproduce. Sorry for the issues you are having.
@rbliu Sorry to take so long to get back to you. I think there may be a couple of things here. One is that I think you need a newer stack. The second is that I found that there are some commits I never pushed.
If you get a new master stack and pull the ticket branches for obs_file and pipe_tasks, you should be able to essentially follow the instructions in this post with the small change that processFile.py should be called processCcd.py. You can find the file to run this example linked from this post.
@KSK Thank you so much for the updates!
I have been trying different methods to install DM stack: I still got the same failed tests with the conda installation, so I switched to lsstsw, which seems to work properly on my machine!
I followed your instructions and completed the process. The output looks promising. Just one quick question, could you give any hints about how to generate and display the source image as in your instruction post?