xz/bzip/gzip fails with hardlink errors on ntfs partition (files with > 1 link)

got the same problem in lucid

Status changed to 'Confirmed' because the bug affects multiple users.

Same problem in Precise (12.04.2).

Also happens for files created on a Windows 7 computer. In my case the drive was converted from FAT using `convert` on Windows.

Same thing happens with bzip2:

user@host:/media/DISK$ bzip2 --keep FILE
bzip2: Input file FILE has 1 other link.

I haven't tried gzip since it doesn't have a --keep option, and I don't want my file deleted.

lzop works as expected.

Status changed to 'Confirmed' because the bug affects multiple users.

This is the normal (yet unexplained AFAIK) behavior of many compression tools like gzip, bzip2 and xz, to refuse to compress a file with a link count > 1, unless you give the --force option. The actual questions here are "why do my NTFS files have a link count of 2?" (see COMP6366_V01_02_15_2011.wmv in the OP) and "why does this link count drop to 1 when I rename the file?"

This is due to the DOS name mangling system (see https://en.wikipedia.org/wiki/Filename_mangling and https://en.wikipedia.org/wiki/8.3_filename). When a file (say "TextFile.Mine.txt") was copied on the NTFS partition with a 8.3 mangled filename (say "TEXTFI~1.TXT"), then it is presented by the ntfs-3g driver only once, in its long form. But the file continues to be accessible by both names: "TEXTFI~1.TXT" (hidden) and "TextFile.Mine.txt" (displayed). Hence the link count of 2, one for each name. And this is also why the link count drops to 1 when you rename the file: the mangled version disappear.

You may want to reassign this bug to the ntfs-3g package, arguing that the link count should be 1. Personally I don't think this is an obvious bad choice but others may disagree.

So, what else? Should we close this bug? Actually, I don't think so. I think it should be renamed to "Gzip / bzip2 / xz refuse to compress files with a link count > 1". As said above, I have yet to read a good reason for gzip/bzip2/xz to _refuse_ to compress a file with more than one hard link. Even with the --keep option! Why not just a mere warning? Imposing a --force flag on e.g. script writers for the sole reason that files may have many hard links, does not seem quite safe to me.

gzip is a very old program (25 years old?). My guess is that this is an unfortunate legacy of a formerly legitimate behavior (for security reasons for example?) which has absolutely no raison d'être nowadays. bzip2 and xz have just blindly copied gzip's interface.

I updated the title - thanks for the info! Great job tracking down the root cause of the issue.

Thanks. IMHO, the title should no longer mention NTFS. The behavior can be observed on any filesystem.

Try this, for example, on an ext* FS:
# touch foo
# ln foo bar
# gzip bar
gzip: bar has 1 other link -- unchanged
# bzip2 bar
bzip2: Input file bar has 1 other link.
# xz bar
xz: bar: Input file has more than one hard link, skipping
# lzma bar
lzma: bar: Input file has more than one hard link, skipping

Note: I have also affected this bug to the gzip project