du on ecryptfs reports incorrect file sizes

Hello, thanks for the bug report.

I'll leave the definitive answer to Tyler Hicks, the upstream kernel maintainer, but I'll try to field this question as best as I can.

Basically, there is a bit of time where the new file exists only in kernel memory, before it gets synced to disk. You might try running "sync", and seeing if it gets flushed.

Tyler, can you provide some more information?

:-Dustin

I just wanted to add to this discussion/report my own experience with this bug. It seems that the issue occurs not just between sync, but even after a reboot/remount.

I have an ecryptfs-mounted home directory, within which everything appears to be 0 bytes. However, I *can* read the files and they appear to have valid contents, it's just that they're misreported by "du" etc

However, as a test I did a similar "dd" from /dev/zero and created a file of approx 2mb. Even after rebooting the machine and remounting the cryptfs partition, "du" still displays the filesize as 0 bytes. As such, it would appear that this issue isn't related to just new files still in memory, as it was long since flushed to disk.

The machine I'm testing on at this particular moment has a kernel that's a few versions behind the latest, so I'll re-post here if the bug goes away after I finish compiling/upgrading.

Cheers,

Tyler

I'm also having this problem on Jaunty:

$ mount | grep home
/dev/mapper/MAIN_VG-HOME_LV_crypt on /home type ext3 (rw)
/home/user/.Private on /home/user type ecryptfs (ecryptfs_sig=ace5d3dc7966c556,ecryptfs_fnek_sig=c0bd3cca6d89c7fd,ecryptfs_cipher=aes,ecryptfs_key_bytes=16)
gvfs-fuse-daemon on /home/user/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=user)
$ dd if=/dev/zero of=test123 count=234333
234333+0 records in
234333+0 records out
119978496 bytes (120 MB) copied, 10.1632 s, 11.8 MB/s
$ du -sh test123
0 test123

Note, it happens with newly created files or directories. It doesn't matter what they're created by...I noticed this when I was creating a very large Lucene index yet du was telling me the directory was empty.

In my case, the index directory actually did show size after a reboot...but it took the reboot to make it happen.

Please, please, if there's any way, issue a patch for this. It's very frustrating, and I really don't have the option of redoing my home directory at the moment.

Thanks!

Also, please note, running "sync" doesn't fix the problem:

$ du -sh test123
0 test123
$ sync
$ sudo sync
[sudo] password for user:
$ du -sh test123
0 test123

The question expired, but IMO this is most definitely a bug, and a pretty important one at that.

FYI du with --apparent-size shows the right logical sizes, it's the sparse block etc support that tries to figure out the underlying storage taken that is the problem.

Oh, nice find!

Perhaps we can document our way around this bug...

One could argue that the current "du" output is correct because the ecryptfs really requires zero bytes for storing the files (because it's using the underlying file system for actual storage).

However, that prevents users from from using "du" or any similar tool for figuring out which files and/or directories take the most disk space. Because the encrypted files are much bigger than apparent size especially for small apparent file sizes it would make sense to allow normal user space programs to compute the real disk space used.

Perhaps one should figure out how "du" is computing its total count for real disk space taken and then implement required features for ecryptfs to correctly result in real disk space used in the underlying file system.

Sorry I haven't been actively updating this bug report. Here's a brain dump:

--apparent-size uses the st_size field returned from a call to stat(). It is set in the kernel with a call to i_size_read() on the eCryptfs inode. As Tv pointed out, it works well.

By default, du uses the st_blocks (512 byte blocks) field of the stat struct to show file size. It does this because most filesystems are smart enough to account for things like holes in the files, so st_blocks * 512 may be considerably less than what you'll see in the st_size field.

What I think eCryptfs should be doing is what Mikko is alluding to by calling the lower filesystem's getattr() to get the *actual* st_blocks. What I'm undecided on is whether or not the size of the header region should be subtracted from the lower filesystem's st_blocks count. It is a small difference - the header region is only going to be 8 KB on most systems.

I'll begin writing up the patch and decide as I'm going. Feel free to weigh-in in the meantime.

I'd vote for "show how much space the encrypted file is taking up on disk". That's what the field means ;)
And everything already knows to expect apparent and real sizes to not match (plus they're at different granularities, anyway).

Thanks for the explanation, Tyler. If the st_blocks field is changed to reflect the space taken by the underlying (lower) file system, I think that the header should be definately included in the count. The header may be "only" 8KB, but it's per file so if you have a huge collection of small files, the total space taken will be much higher with headers counted in. For example, the linux-2.6.32-rc6 kernel source contains 30477 files. With overhead of 8 KB per file, the total difference with and without headers will be 30477*8 KB or about 240MB (the apparent size for the kernel is 349MB). I'd hope differences that big to be reflected on the "du" output.

The masses have spoken :)

Thanks. I agree that reporting zero because the bytes are accounted for in the underlying FS is not the right answer, since this is a different mount point, and it is understood that overlay mounts can cause the same files or disk space to appear twice.

Any chance of having a bug fix in karmic, since you didn't get it out in time for the release?

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=f8f484d1b6677dd5cd5e7e605db747e8c30bbd47

Andy-

Any chance of getting this commit cherry picked into the Lucid kernel to fix this bug?

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=f8f484d1b6677dd5cd5e7e605db747e8c30bbd47

Patch http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=f8f484d1b6677dd5cd5e7e605db747e8c30bbd47 sent to <email address hidden>

This bug was fixed in the package linux - 2.6.32-14.19

---------------
linux (2.6.32-14.19) lucid; urgency=low

  [ Andy Whitcroft ]

  * ensure we build the source package contents when enabled
    - LP: #522308
  * [Config] enable CONFIG_X86_MCE_XEON75XX
  * SAUCE: AppArmor -- add linux/kref.h for struct kref
  * [Config] enable CONFIG_HID_ORTEK
  * enable udeb generation for arm versatile flavour
    - LP: #522515

  [ John Johansen ]

  * ubuntu: AppArmor -- update to mainline 2010-02-18
    - LP: #439560, #496110, #507069

  [ Johnathon Harris ]

  * SAUCE: HID: add support for Ortek WKB-2000
    - LP: #405390

  [ Upstream Kernel Changes ]

  * tpm_tis: TPM_STS_DATA_EXPECT workaround
    - LP: #490487
  * x86, mce: Xeon75xx specific interface to get corrected memory error
    information
  * x86, mce: Rename cpu_specific_poll to mce_cpu_specific_poll
  * x86, mce: Make xeon75xx memory driver dependent on PCI
  * drm/edid: Unify detailed block parsing between base and extension
    blocks
    - LP: #500999
  * (pre-stable) eCryptfs: Add getattr function
    - LP: #390833
 -- Andy Whitcroft <email address hidden> Thu, 18 Feb 2010 19:22:02 +0000

Uhoh, I think this broke dpkg and probably other userspace apps; see bug #524919.

Is there a chance to get this fix into karmics kernel too?