Git lfs not working for me with `testdata_jointcal`

Support
#1

I’ve set git config lfs.batch true globally (although it was not previously), but am today getting the following error (I was not yesterday) using lsstsw rebuild:

rebuild 
  File "/Users/wmwv/lsstsw/lsst_build/bin/lsst-build", line 41, in <module>
    args.func(args)
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 746, in run
    manifest = p.construct(args.products)
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 709, in construct
    self._add_product_tree(products, name)
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 697, in _add_product_tree
    dependencies.append( self._add_product_tree(products, dprod.name) )
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 697, in _add_product_tree
    dependencies.append( self._add_product_tree(products, dprod.name) )
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 697, in _add_product_tree
    dependencies.append( self._add_product_tree(products, dprod.name) )
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 680, in _add_product_tree
    ref, sha1 = self.product_fetcher.fetch(productName)
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/prepare.py", line 318, in fetch
    git.checkout("--force", ref)
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/git.py", line 42, in checkout
    return self('checkout', *args, **kwargs)
  File "/Users/wmwv/lsstsw/lsst_build/python/lsst/ci/git.py", line 37, in __call__
    raise GitError(retcode, cmd, stdout, stderr)
lsst.ci.git.GitError: Command '['git', 'checkout', '--force', 'master']' returned non-zero exit status 128.
stdout:

stderr:
Downloading cfht_minimal/src/06AL01/D3/2006-05-20/r/SRC-849375-12.fits (75.94 KB)
WARNING: Git LFS is using a deprecated API, which will be removed in v2.0.
         Consider enabling the latest API by running: `git config lfs.batch true`.
Error downloading object: cfht_minimal/src/06AL01/D3/2006-05-20/r/SRC-849375-12.fits (5071e4548d33b407c0438b3b7fcbf4fab390674f30ccfd5f7492e21b689e6653)

Errors logged to /Volumes/PS1/lsstsw/build/testdata_jointcal/.git/lfs/objects/logs/20170801T152549.098055978.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs smudge %f' failed 2
error: external filter 'git-lfs smudge %f' failed
fatal: cfht_minimal/src/06AL01/D3/2006-05-20/r/SRC-849375-12.fits: smudge filter lfs failed
#2

What version of git-lfs are you using?

#3

cfht_minimal was the thing actually added to testdata_jointcal and is thus triggering the need to get new files.

#4 
[serenity testdata_jointcal] git-lfs version
git-lfs/1.5.6 (GitHub; darwin amd64; go 1.7.5; git bbd328b1)
#5 
Smudge error: Error downloading 5071e4548d33b407c0438b3b7fcbf4fab390674f30ccfd5f7492e21b689e6653: Fatal error: Server error: https://git-lfs.lsst.codes/objects/5071e4548d33b407c0438b3b7fcbf4fab390674f30ccfd5f7492e21b689e6653: Fatal error: Server error: https://git-lfs.lsst.codes/objects/5071e4548d33b407c0438b3b7fcbf4fab390674f30ccfd5f7492e21b689e6653
#6

That’s ancient. Try a newer one. I’m using 2.2.1 from homebrew.

#7

I will grant you that lsstsw seems to default to v1.5.5. Seems a bit old @josh.

1 Like
#8

It’s the one lsstsw installed. I personally use 2.2.1.

#9

I confirm that using git-lfs 2.2.1 works and solves the problem:

[serenity ~] git lfs version
git-lfs/2.2.1 (GitHub; darwin amd64; go 1.8.3)
[serenity ~] cd lsstsw/build/testdata_jointcal/
[serenity testdata_jointcal] git checkout --force master
Downloading cfht_minimal/src/06AL01/D3/2006-05-20/r/SRC-849375-12.fits (78 KB)
Downloading cfht_minimal/src/06AL01/D3/2006-06-02/r/SRC-850587-12.fits (78 KB)
Already on 'master'
Your branch is up-to-date with 'origin/master'.

Good catch, @timj

1 Like
#10

What ref is your lsstsw checkout at?

#11

And what ref is the <lsstsw root>/lsst_build checkout at?

#12 
*   a260926      Merge branch 'tickets/DM-9961'

*   282677a      Merge pull request #8 from jhoblitt/tickets/DM-4113-lfs-support
#13

The ref for lsst_build is from 2015 – which is why it is blowing up with the current git-lfs server implementation. Sadly, deploy does not update the lsst_build checkout. I suspect if you do a manual pull it will fix the lfs errors.

#14

I’ve opened a ticket on adding a lsst_build update mechanism to lsstsw: https://jira.lsstcorp.org/browse/DM-11464

#15

Manually updating the lsst_build directory did fix things.