What makes this issue even weirder is that it is not completely reproducible. Sometimes after a fresh
git clone
3 files show up as modified, sometimes its only 2 of them. What is consistent is that
git status
only seems to be happy if only the pointer file of these 3 files is present in the working tree.
Sorry to hear you're having trouble. I'm not sure exactly what's going on, but I have some questions to help figure it out.
$ git clone ssh://git@xxx/xxx/sw.git
Cloning into 'sw'...
remote: Counting objects: 40405, done.
remote: Compressing objects: 100% (4142/4142), done.
remote: Total 40405 (delta 3488), reused 974 (delta 745)
Receiving objects: 100% (40405/40405), 28.49 MiB | 8.25 MiB/s, done.
Resolving deltas: 100% (28911/28911), done.
Checking out files: 100% (1314/1314), done.
Filtering content: 100% (56/56), 141.70 MiB | 11.56 MiB/s, done.
I cannot find anything interesting in the console output when running git clone
with GIT_TRACE=1
; the files in question show up in the list of LFS tracked files; I also see HTTP GET requests for all of the files that show up as modified later; please let me know if you are looking anything specific here; I would attach the log but there are some customer names in some of our files and I'd have to clean up the log first
Furthermore I have to correct my observation that the files that sometimes show up as modified only show up as modified if their binary version is checked out. After running git clone
a couple of times again I have also seen the situation where only one of the files shows up as modified (its binary file is checked out) and the other two files do not show up as modified (their binary files are checked out as well).
Something I forgot to mention in the initial post is that some of the files might have been initially pushed from a machine where git lfs install
had not been run, i.e. the binary file was actually pushed as binary file instead of to LFS storage. This was later rectified by deleting the binary file from git and committing/pushing again with git lfs install
properly setup.
Usually this problem occurs for one of a couple of reasons. One is the situation you describe in your last paragraph: the file is modified because it's tracked as a Git file but the file is listed in .gitattributes
as an LFS file. In such a case, trying git cat-file blob "HEAD:test1.bin" | git lfs pointer --check --stdin
(where you replace test1.bin
with an affected file) should exit nonzero, and you can then run git lfs migrate import
and commit the result to make sure that the file isn't modified.
The other reason this can occur is that some process on the system is causing the file to be modified or stat-dirty. Sometimes this can be an editor or other tool that handles your binary files, especially on Windows, where timestamps are more important since we lack an inode number. I don't know how you'd go about testing this, but it is a possibility.
I ran the command you suggested on one of the files that shows up as modified but it returned 0.
What still strikes me as extremely weird is that the issue is non-deterministic. I just had about 5 subsequent git clone
runs where none of the files in question showed up as modified and they were all checked out properly as binary files in the file system. Only after the 6th attempt I could get some of the files to show up as modified again.
There is no obvious time-stamp difference of the files that are shown as modified and those who do not.
Running git checkout
on the "modified" files has no effect. git
says Updated X paths from the index
but they still show up as modified afterwards.
Do you have any additional ideas/suggestions?
Do you have any additional ideas/suggestions?
The process that writes these files to disk is actually Git; Git LFS just filters them in this case. I don't know of a reason that Git would think of them being modified, especially non-deterministically, after it's just written them.
You could check the files by running sha256sum
on the modified files and seeing if the hashes and sizes match the values in the pointer files (which you can see by running the git cat-file blob
invocation I provided above, but without the pipe to git lfs pointer
). That would tell you at least if there's a legitimate reason for them being modified.
If they are indeed modified, you could look to see if you have some sort of antivirus or other tool that's causing a problem; that's a common source of problems with Git, so it wouldn't surprise me if it hit Git LFS as well. Beyond that, I don't have any other suggestions on tracking this down.
If I run git diff
on one of the modifies files I get the following output.
-version https://git-lfs.github.com/spec/v1
-oid sha256:92f443377fb52dd71221d54cb8522119293159d5c0a812f0652356072ad579f6
-size 648906
+version https://git-lfs.github.com/spec/v1
+oid sha256:92f443377fb52dd71221d54cb8522119293159d5c0a812f0652356072ad579f6
+size 648906
This matches the output when running sha256sum
over the file manually.
Interestingly if I run git diff -w
(ignore whitespace) on the files, the output is empty. Could this somehow be whitespace related?
Does your .gitattributes
file contain a -text
attribute for your binary files? It sounds like what might have happened is that these files are getting rewritten with CRLF line endings which they should not have.
Which version of the file is the one with CRLF endings and which one has LF endings? Are there any other attributes in the .gitattributes
file which set the text
attribute? Also, do you have any settings that modify core.eol
or core.autocrlf
?
Yes, there is a -text
attribute for all the binary files (this was automatically generated when using git lfs track
).
If I run git diff --ws-error-highlight=old
on a file that shows up as modified I get the following output.
-version https://git-lfs.github.com/spec/v1^M
-oid sha256:ccb05f59bf727a908ae5d7a76b871d974c29cadc077d4e091df2e1947e2fed88^M
-size 211456^M
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccb05f59bf727a908ae5d7a76b871d974c29cadc077d4e091df2e1947e2fed88
+size 211456
This means that the "unmodified" file contains CRLF line endings which were replaced by LF line endings?
All the .gitattributes
entries look like this
[file name] filter=lfs diff=lfs merge=lfs -text
My local .gitconfig
contains the following [core]
section
[core]
repositoryformatversion = 0
filemode = false
bare = false
logallrefupdates = true
symlinks = false
ignorecase = true
My global .gitconfig
contains the following [core]
section
[core]
editor = emacs -nw
autocrlf = true
[filter "lfs"]
process = git-lfs filter-process
required = true
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
If I run git diff --ws-error-highlight=old
on a file that shows up as modified I get the following output.
-version https://git-lfs.github.com/spec/v1^M
-oid sha256:ccb05f59bf727a908ae5d7a76b871d974c29cadc077d4e091df2e1947e2fed88^M
-size 211456^M
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccb05f59bf727a908ae5d7a76b871d974c29cadc077d4e091df2e1947e2fed88
+size 211456
This means that the "unmodified" file contains CRLF line endings which were replaced by LF line endings?
Yup. I have no idea why this would occur: we never write CRLF endings in a pointer file, only LF endings. This does definitely explain the problem, though, since if Git cleans the files, they'll then show up as modified. It's possible some other tool (or someone writing pointer files by hand) caused this issue.
I think we probably need to reject pointer files with carriage returns, which I'll try to get to soon. As for your situation, I think you'll need to remove the carriage returns from any files that have them and commit, which should prevent Git from thinking that the files are modified.
This seems to have solved the issue (just did 10 clones as the problem would not always show up) - thank you!
For anyone with the same problem, I used
GIT_LFS_SKIP_SMUDGE=1 git clone <repo>.git
cd <repo>
git lfs ls-files | cut -d" " -f3 | xargs dos2unix {}
git add -u .
git commit -m 'fix git lfs pointer file line endings'
git push
I still have no idea how this happened (maybe a machine without git lfs
installed cloned the repo, accidentally modified the pointer files and pushed them back?) or why the files showing up as modified is non-deterministic.
I have a very similar problem. My git diff shows the similar results containing version, oid, size, and no actual changes. This has been occurring after installing LFS and migrating info on a project that was about 6 months old without LFS.
I tried
git lfs ls-files | cut -d" " -f3 | xargs dos2unix {}
but it did not solve the issue. I was hoping that I didn't have to clone the project again. Cloning 10 times to fix an issue sounds unpleasant - so I'm going to avoid it until it is the only thing that actually fixes it.
I don't think that cloning multiple times fixes the issue; it's just that it appears that the files don't always appear modified for the original reporter. In order to fix the repository with that command, you'd need to clone the repo with GIT_LFS_SKIP_SMUDGE=1
; did you do that? Also, does your .gitattributes
file contain -text
for your LFS files? If not, it should.
I don't think that cloning multiple times fixes the issue; it's just that it appears that the files don't always appear modified for the original reporter. In order to fix the repository with that command, you'd need to clone the repo with GIT_LFS_SKIP_SMUDGE=1
; did you do that? Also, does your .gitattributes
file contain -text
for your LFS files? If not, it should.
I had recloned with SMUDGE=1. Looking at the .gitattributes now, they do have -text on all lines. LFS generated this .gitattributes when I manually tracked for the two very specific file paths listed here:
*.mp3 filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text
.obj filter=lfs diff=lfs merge=lfs -text
Sama/Assets/GeneratedMeshes/DotDiagram_3_0_0 filter=lfs diff=lfs merge=lfs -text
Sama/Assets/Scenes/Sama[:space:].unity filter=lfs diff=lfs merge=lfs -text
.obj,.mp3,.wav filter=lfs diff=lfs merge=lfs -text
*.gitignore filter=lfs diff=lfs merge=lfs -text
*.docx filter=lfs diff=lfs merge=lfs -text
*.pdf filter=lfs diff=lfs merge=lfs -text
*.meta filter=lfs diff=lfs merge=lfs -text
*.cs filter=lfs diff=lfs merge=lfs -text
*.unity filter=lfs diff=lfs merge=lfs -text
*.mat filter=lfs diff=lfs merge=lfs -text
*.prefab filter=lfs diff=lfs merge=lfs -text
*.shader filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.controller filter=lfs diff=lfs merge=lfs -text
*.fbx filter=lfs diff=lfs merge=lfs -text
*.physicMaterial filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
*.mixer filter=lfs diff=lfs merge=lfs -text
*.tga filter=lfs diff=lfs merge=lfs -text
*.psd filter=lfs diff=lfs merge=lfs -text
*.txt filter=lfs diff=lfs merge=lfs -text
*.anim filter=lfs diff=lfs merge=lfs -text
*.renderTexture filter=lfs diff=lfs merge=lfs -text
*.asset filter=lfs diff=lfs merge=lfs -text
*.csproj filter=lfs diff=lfs merge=lfs -text
*.sln filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.dropbox filter=lfs diff=lfs merge=lfs -text
*.ini filter=lfs diff=lfs merge=lfs -text
*.ttf filter=lfs diff=lfs merge=lfs -text
*.mtl filter=lfs diff=lfs merge=lfs -text
*.pptx filter=lfs diff=lfs merge=lfs -text
*.xlsx filter=lfs diff=lfs merge=lfs -text
*.cginc filter=lfs diff=lfs merge=lfs -text
*.xcf filter=lfs diff=lfs merge=lfs -text
*.dll filter=lfs diff=lfs merge=lfs -text
*.unitypackage filter=lfs diff=lfs merge=lfs -text
*.xml filter=lfs diff=lfs merge=lfs -text
*.jar filter=lfs diff=lfs merge=lfs -text
*.aar filter=lfs diff=lfs merge=lfs -text
*.so filter=lfs diff=lfs merge=lfs -text
*.bundle filter=lfs diff=lfs merge=lfs -text
*.c filter=lfs diff=lfs merge=lfs -text
*.h filter=lfs diff=lfs merge=lfs -text
*.a filter=lfs diff=lfs merge=lfs -text
*.jspre filter=lfs diff=lfs merge=lfs -text
*.jslib filter=lfs diff=lfs merge=lfs -text
*.m filter=lfs diff=lfs merge=lfs -text
*.json filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
*.mm filter=lfs diff=lfs merge=lfs -text
*.dylib filter=lfs diff=lfs merge=lfs -text
*.apk filter=lfs diff=lfs merge=lfs -text
*.tif filter=lfs diff=lfs merge=lfs -text
*.dat filter=lfs diff=lfs merge=lfs -text
*.otf filter=lfs diff=lfs merge=lfs -text
*.csv filter=lfs diff=lfs merge=lfs -text
*.net filter=lfs diff=lfs merge=lfs -text
*.old filter=lfs diff=lfs merge=lfs -text
*.log filter=lfs diff=lfs merge=lfs -text
*.TTF filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.FBX filter=lfs diff=lfs merge=lfs -text
*.disabled filter=lfs diff=lfs merge=lfs -text
*.plist filter=lfs diff=lfs merge=lfs -text
*.tsv filter=lfs diff=lfs merge=lfs -text
*.md filter=lfs diff=lfs merge=lfs -text
*.PNG filter=lfs diff=lfs merge=lfs -text
*.TGA filter=lfs diff=lfs merge=lfs -text
*.exr filter=lfs diff=lfs merge=lfs -text
*.removeme filter=lfs diff=lfs merge=lfs -text
*.js filter=lfs diff=lfs merge=lfs -text
*.ogg filter=lfs diff=lfs merge=lfs -text
*.asmdef filter=lfs diff=lfs merge=lfs -text
*.iml filter=lfs diff=lfs merge=lfs -text
*.user filter=lfs diff=lfs merge=lfs -text
*.mdb filter=lfs diff=lfs merge=lfs -text
*.pdb filter=lfs diff=lfs merge=lfs -text
*.spriteatlas filter=lfs diff=lfs merge=lfs -text
Problem still persists:
My git diff shows the similar results containing version, oid, size, and no actual changes.
If git diff
says that file is modified, it is modified. As @wrightsg shows here, you just sometimes need to take special actions to clearly see what exactly is modified in a file.
How to properly fix this:
Make sure that .gitattributes
has -text
for LFS files
Make sure you don't have any local changes that you don't want to lose
From working copy root: git fetch && git reset --hard origin/master && git rm -r --cached * && git reset && git commit -am "Renormalize files" && git push
The actual magic is git rm -r --cached * && git reset
. But doing it only is often not enough because people then find out they're not at the HEAD of remote branch, try to rebase (that will fail due to the problem we're fixing in the first place), then try to abort rebase (it will fail for exactly the same reason), etc, etc.
You might think that this sequence of commands deletes everything (there's a scary git rm
). No, it doesn't. You may examine produced commit before pushing it.
Why does magic fix work: it forces Git to reevaluate all files in index and convert them to 'canonical' state they're supposed to have based on current .gitattributes
As to how files get in such state - there are multiple ways to achieve that, even without LFS. It usually involves modification of .gitattributes
without proper modification of files that are affected by .gitattributes
, so their actual state in repo gets out of sync with what their state is supposed to be.
Exercise to the reader: think why putting .gitattributes
itself under LFS is a really bad idea.
Side note: I personally don't see any point storing .cs
and other text files in LFS.
Side note: I personally don't see any point storing .cs and other text files in LFS.
I don't either, and I guess Git didn't too. This is why this is happening. As I mentioned earlier, .gitattributes was auto generated with that huge list of track filters. *.cs was added by it, and not by me. So seeing that was actually a shocker. I have used "git rm -r --cached" to fix LFS issues regarding missing files before, I didnt think that would solve a 0 byte error. I'll try your fix suggestion.
git rm --cached '<file-type>'
git add '<file-type>'
git commit -m "restore '<file-type>' to git from lfs"
git push <remote> <branch>
I untracked all my CS files.
This fixed the problem for me. Surely, "git diff" does not work with tracked LFS files.
What you did is also fine. Just remember that each time you modify .gitattributes
you want to actually apply those modifications to actual files. And the easiest way to force that is to remove files from index (git rm --cached
), then readd them back to index (git add
or git reset
) and finally git commit
.
Hope this question is relevant enough for this issue (Google pointed me here).
I just created a repository and copied some data file to the data/
folder, so I ran the commands:
git lfs install
git lfs track data/*
Before adding anything to git, I was just browsing through the files, and found that 1 file, which should be a binary .npy
, was listed with -text
: data/dat.npy filter=lfs diff=lfs merge=lfs -text
.
This data file was created through another project, so I'm not sure how it is made (I guess a ndarray.dump()
, but not sure if by a Linux or a Windows machine).
Thinking that this shouldn't be treated as a text file, can I just remove the -text
part?
Or is it fine to leave it like this (don't want the OP's issue to affect me)?
I'm working on Ubuntu 18.04 now.
We have been seeing this same issue on several computers and several repositories after turning LFS on. The results are inconsistent - different files show up as modified on different computers. And sometimes after a fresh clone the problem doesn't occur at all. Sometimes the initial clone is clean, but we see files marked 'modified' after checkout out an existing branch.
It happens using Git Windows versions 2.22 - 2.25.
We'll see messages like "Encountered 19 file(s) that should have been pointers, but weren't:", but that's expected. The files that are listed in that message are the ones listed as "modified".
This appears to only happen on Windows platforms. Linux systems using the same repositories don't have the issue.
The "modification" seems to be the file mode:
$ cmp $ git diff Component/EventManager.doc
index 19ba14ff..5ba66436 100755
Binary files a/Component/EventManager.doc and b/Component/EventManager.doc differ
glenn.burkhardt@AI234087 MSYS /c/temp/ccs2
$ ll Component/EventManager.doc
-rw-r--r-- 1 glenn.burkhardt raptor 43K Feb 19 11:37 Component/EventManager.doc
But the same file (compares byte-wise equal) on a Linux system has:
glenn.burkhardt@armis-00:~/repos/ccs2$ ll Component/EventManager.doc
-rwxrwxr-x 1 glenn.burkhardt glenn.burkhardt 43520 Jan 22 2018 Component/EventManager.doc*
We stumbled on a fix, using this technique:
cp /dev/null .gitattributes
git status
git checkout .gitattributes
Oddly, if the intervening 'git status' step is skipped, it doesn't work. The status query reports:
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git restore ..." to discard changes in working directory)
modified: .gitattributes
If you see messages that say files should have been pointers but weren't, that's definitely the cause of your problems. You should run git lfs migrate import --no-rewrite
to fix those files, because they're going to show up as modified until you rewrite them as LFS files.
Generally that message means that someone checked in a file that's supposed to be tracked as LFS but didn't use LFS to do so, and doing that is going to cause problems.
If you see messages that say files should have been pointers but weren't, that's definitely the cause of your problems. You should run git lfs migrate import --no-rewrite
to fix those files, because they're going to show up as modified until you rewrite them as LFS files.
Generally that message means that someone checked in a file that's supposed to be tracked as LFS but didn't use LFS to do so, and doing that is going to cause problems.
Thanks - using the 'lfs migrate import' command is a nice workaround - it results in only a single commit if all the file names in question are passed to it, and it only affects the problem files. It does need to be applied to all the files in every branch that are reported as "modified".
But there's still a rat in that woodpile.
Although I'd see messages about files should have been pointers but weren't, not all of those files were marked 'modified'. Only a subset was incorrectly marked. After using the 'migrate' workaround, I'll still see those messages about files should have been pointers but weren't, but no longer see files marked modified.
It looks like the config setting "core.filemode=false" isn't being correctly obeyed for all files on a Windows 10 platform.
The workaround that zeros out .gitattributes shouldn't work. It's especially bizarre that if the status query is omitted, it doesn't work.
This problem doesn't appear when working on Linux systems using the same remote repository.
It's unclear to me why this behavior seems to occur only for certain files and only on Windows. I suspect it's because the files are being modified with line ending changes, which would not occur if they were actual LFS files (since those are not subject to line-ending changes). It's possible that it's also just a race condition in that some files are considered racily clean (due to matching timestamps) and Git processes them a second time, which causes them to be turned into pointer files and therefore marked as modified, and that Windows is more prone to this race condition.
Regardless, the message that tells you that files should be pointers but are not is a serious issue in your repo that you need to fix by migrating all of those files to LFS. Ignoring it leads to bizarre behaviors like this, and there's nothing we can do to address that because of how Git LFS is invoked by Git.
I'm not sure why you think core.filemode
isn't being honored, but that is also generally handled by Git, not Git LFS. The diff you've shown indicates that the files share the same mode but have different contents and at least one set of contents is binary. What's likely the case is that one of those files is the actual file and the other is the pointer file, which happens when Git realizes that the file is modified.
Changing .gitattributes
affects how and whether Git LFS is invoked by Git, and so if you empty the file, you're disabling Git LFS and therefore it won't be invoked by Git to turn files into pointer files.
You're welcome to post your .gitattributes
file and the output of git config -l
for the affected machine and I'll take a look and see if there's something obvious that stands out about why this is occurring, but I suspect the issue will solve itself if you either run the migrate invocation or run git add --renormalize .
and commit the result.
Thank you @bk2204 for your response.
re: line ending changes: The files that are marked modified are binary, e.g., jar, zip files. And, I have 'core.autocrlf=false'.
Agreed that it we should migrate the files to LFS.
The behavior does indicate an anomaly in processing somewhere, and the it's not readily apparent that the two identified workarounds should be executed. It's particularly curious that after zeroing out the .gitattributes file, 'git status' no longer reports any files to be modified other than the .gitattributes file, and that if the 'git status' step is omitted, the other files are still shown as modified.
The inconsistent results we see on different computers, or even on the same computer at different times, does point to a race condition.
Just for the record, here are the data you requested:
.gitattributes.txt
config.txt
Posting to report similar behavior with Git 2.26.2 / git-lfs 2.11.0 on macOS 10.15.5.
I'm cloning a repo of a bunch of similarly-sized audio files using git-subrepo into a repo that uses git-lfs and a few get flagged as modified after the clone.
Initially it was 31 of 1868 files. Then I did a hard reset and ran again and it was 638 modified of 1868. Just did a hard reset again and cloned the repo: 57 of 1868. Something's fishy.
Running git lfs migrate import --fixup --everything
appears to be an important first step to fixing things. That will at least get all the pointers in place.
I'll upgrade to git 2.30.0, git-lfs 2.13.1 and git-subrepo 0.4.3 and keep my eyes open for recurrence in the future. Leaving the above "fix" for others.
I'm experiencing this exact behavior (lfs files randomly being reported as modified after a clone) in a Linux docker image running as part of a pipeline build.
I've tried checking for the CR LF problem, I've tied running git lfs migrate import --fixup --everything
Nothing as worked to resolve the problem.
@bk2204 I had just run a pipeline build with the git diff --ws-error-highlight=all
command along with some other commands.
Here is the output, (hint the output it is not helpful)
+ git --version
git version 2.17.1
+ git lfs --version
Sorry, no usage text found for "git-lfs"
Error: unknown flag: --version
+ git status
On branch feature/resolve-pipeline-dirty-issue
Your branch is up to date with 'origin/feature/resolve-pipeline-dirty-issue'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: binaries/u-boot.elf
modified: design_1_xsa/design_1_wrapper.xsa
no changes added to commit (use "git add" and/or "git commit -a")
+ git lfs status
On branch feature/resolve-pipeline-dirty-issue
Git LFS objects to be pushed to origin/feature/resolve-pipeline-dirty-issue:
Git LFS objects to be committed:
Git LFS objects not staged for commit:
binaries/u-boot.elf (LFS: 173ff22 -> File: 173ff22)
design_1_xsa/design_1_wrapper.xsa (LFS: 75a6c0d -> File: 75a6c0d)
+ git diff --ws-error-highlight=old
diff --git a/binaries/u-boot.elf b/binaries/u-boot.elf
index 247501c..0885601 100644
Binary files a/binaries/u-boot.elf and b/binaries/u-boot.elf differ
diff --git a/design_1_xsa/design_1_wrapper.xsa b/design_1_xsa/design_1_wrapper.xsa
index 0975809..ade4637 100644
Binary files a/design_1_xsa/design_1_wrapper.xsa and b/design_1_xsa/design_1_wrapper.xsa differ
@bk2204 Here is the contents of the .gitattributes file:
* text=auto
*.bit filter=lfs diff=lfs merge=lfs -text
*.xsa filter=lfs diff=lfs merge=lfs -text
*.elf filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
@bk2204 with respect to your question about git add --renormalize .
do you want me to run it when git is reporting one or more of the files as modified or just run the command in general?
I want you to run it in your local copy on an otherwise clean working tree and see if it causes any files to show as modified in the git status
output. If it does, I want you to commit that change and then see if it fixes the problem. This is the documented fix in the Git FAQ for this case.
Your problem is that you don't have Git LFS installed in the container or it's not appropriately configured. When you push data into a container, all the information in the index is incorrect because the device and inode numbers change, so Git re-reads all of the data anytime you refresh the index.
When you do that, Git asks to clean the LFS files, but since Git LFS isn't enabled in the container, those files are no longer reflected as LFS pointer files, but as Git files, which is different from what's in the index, and they're marked as modified.
Either don't refresh the index in the container (that is, avoid running Git) or install Git LFS and make sure that git lfs install
has run (or, if your distro sets things up properly, like Debian's packages, then you don't have to).
@bk2204 So git-lfs is installed in the container, it was done as part of the docker image build using apt install like so:
apt-get upgrade -y -qq && \
apt-get install -y -qq --no-install-recommends \
git \
git-lfs \
git-flow \
apt-get clean && \
apt-get autoclean
The image does not explicitly run git lfs install
as I was under the impression that the package install handled that but I can add that to the image build if it will fix the problem.
Also note that in the log I posted above the pipeline ran git lfs status
and it did report results consistent with the results produced by git status
so I'm relatively certain that git-lfs is installed and at least partially working.
You shouldn't see this output when you run git diff
:
+ git diff --ws-error-highlight=old
diff --git a/binaries/u-boot.elf b/binaries/u-boot.elf
index 247501c..0885601 100644
Binary files a/binaries/u-boot.elf and b/binaries/u-boot.elf differ
diff --git a/design_1_xsa/design_1_wrapper.xsa b/design_1_xsa/design_1_wrapper.xsa
index 0975809..ade4637 100644
Binary files a/design_1_xsa/design_1_wrapper.xsa and b/design_1_xsa/design_1_wrapper.xsa differ
Diff output when you have a file that's correctly being tracked as LFS looks like this:
diff --git a/test10.bin b/test10.bin
new file mode 100644
index 0000000..f4808fe
--- /dev/null
+++ b/test10.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:917df3320d778ddbaa5c5c7742bc4046bf803c36ed2b050f30844ed206783469
+size 3
In other words, what you should see in the diff is the pointer file, not binary content. If you're seeing that, then either the file already in the repository is not LFS file, in which case you need to use git add --renormalize .
to fix that, or the file in the working tree appears to be modified to Git (for whatever reason) and LFS is not properly configured.
This is also slightly concerning:
+ git lfs --version
Sorry, no usage text found for "git-lfs"
Error: unknown flag: --version
You may have a very old version, in which case git lfs version
would be helpful. I don't know what OS you're using here, but it looks like it may be Ubuntu 18.04, in which case you may have found an issue which is fixed in a newer version.
This output indicates that it looks like your file is an LFS file in the repository, and something is trying to turn it into something else here:
Git LFS objects not staged for commit:
binaries/u-boot.elf (LFS: 173ff22 -> File: 173ff22)
design_1_xsa/design_1_wrapper.xsa (LFS: 75a6c0d -> File: 75a6c0d)
Without a reproducible example against the latest release, I can't really say what the problem is for certain.
OK thanks for the info
The docker image is based on Ubuntu 18.04, I am rebuilding my docker image
and installing the latest git via source (2.35.1) and installing git lfs
from the packagecloud repository. I am hopeful that this resolves the
issue.
Matt S.
On Wed, Feb 2, 2022 at 1:43 PM brian m. carlson ***@***.***> wrote:
You shouldn't see this output when you run git diff:
+ git diff --ws-error-highlight=old
diff --git a/binaries/u-boot.elf b/binaries/u-boot.elf
index 247501c..0885601 100644
Binary files a/binaries/u-boot.elf and b/binaries/u-boot.elf differ
diff --git a/design_1_xsa/design_1_wrapper.xsa b/design_1_xsa/design_1_wrapper.xsa
index 0975809..ade4637 100644
Binary files a/design_1_xsa/design_1_wrapper.xsa and b/design_1_xsa/design_1_wrapper.xsa differ
Diff output when you have a file that's correctly being tracked as LFS
looks like this:
diff --git a/test10.bin b/test10.bin
new file mode 100644
index 0000000..f4808fe
--- /dev/null
+++ b/test10.bin
@@ -0,0 +1,3 @@
+version
https://git-lfs.github.com/spec/v1
+oid sha256:917df3320d778ddbaa5c5c7742bc4046bf803c36ed2b050f30844ed206783469
+size 3
In other words, what you should see in the diff is the pointer file, not
binary content. If you're seeing that, then either the file already in the
repository is not LFS file, in which case you need to use git add
--renormalize . to fix that, or the file in the working tree appears to
be modified to Git (for whatever reason) and LFS is not properly configured.
This is also slightly concerning:
+ git lfs --version
Sorry, no usage text found for "git-lfs"
Error: unknown flag: --version
You may have a very old version, in which case git lfs version would be
helpful. I don't know what OS you're using here, but it looks like it may
be Ubuntu 18.04, in which case you may have found an issue which is fixed
in a newer version.
This output indicates that it looks like your file is an LFS file in the
repository, and something is trying to turn it into something else here:
Git LFS objects not staged for commit:
binaries/u-boot.elf (LFS: 173ff22 -> File: 173ff22)
design_1_xsa/design_1_wrapper.xsa (LFS: 75a6c0d -> File: 75a6c0d)
Without a reproducible example against the latest release, I can't really
say what the problem is for certain.
Reply to this email directly, view it on GitHub
<
#3787 (comment)>,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ADYXOSRTJJPBUAGNLQ7KUL3UZGQRXANCNFSM4IO4XVKQ>
Triage notifications on the go with GitHub Mobile for iOS
<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
@bk2204 Updating the version of git and git-lfs in docker seems to have resolved the issue.
Thank you for the help.