Trim doesn't work with lvm and dm-crypt

Bug #1213631 reported by dpslxoo
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Util-Linux-ng
New
Undecided
Unassigned
cryptsetup
New
Undecided
Unassigned
lvm2
New
Undecided
Unassigned
lvm2 (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

I want to use fstrim together with lvm and dm-crypt but it doesn't work (see test procedure below). The problem seems to be somewhere in the lvm layer. Is there any fix or workaround for this?

My configuration:

`cat /etc/crypttab`

    sda3_crypt UUID=[...] none luks,discard

and

`cat /etc/lvm/lvm.conf`

    # [...]
    devices {
          # [ ... ]
          issue_discards = 1
          # [ ... ]
       }
    # [...]

The SSD is a Samsung 840 Pro.

Here is the output of `sudo dmsetup table`

    lubuntu--vg-root: 0 465903616 linear 252:0 2048
    lubuntu--vg-swap_1: 0 33308672 linear 252:0 465905664
    sda3_crypt: 0 499222528 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 8:3 4096 1 allow_discards

Here is my `/etc/fstab`:

    # <file system> <mount point> <type> <options> <dump> <pass>
    /dev/mapper/lubuntu--vg-root / ext4 errors=remount-ro 0 1
    # /boot was on /dev/sda2 during installation
    UUID=f700d855-96d0-495e-a480-81f52b965bda /boot ext2 defaults 0 2
    # /boot/efi was on /dev/sda1 during installation
    UUID=2296-2E49 /boot/efi vfat defaults 0 1
    /dev/mapper/lubuntu--vg-swap_1 none swap sw 0 0
    # tmp
    tmpfs /tmp tmpfs nodev,nosuid,noexec,mode=1777 0 0

Testing procedure:

yes | dd iflag=fullblock bs=1M count=1 of=trim.test

filefrag -s -v trim.test

Filesystem type is: ef53
File size of trim.test is 1048576 (256 blocks, blocksize 4096)
 ext logical physical expected length flags
   0 0 2224384 256 eof
trim.test: 1 extent found

df trim.test

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/lubuntu--vg-root 229164268 38651596 178865084 18% /

sudo dd bs=4096 skip=2224384 count=256 if=/dev/mapper/lubuntu--vg-root | hexdump -C

00000000 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a |y.y.y.y.y.y.y.y.|
*
256+0 records in
256+0 records out
1048576 bytes (1,0 MB) copied, 0,00328209 s, 319 MB/s
00100000

rm trim.test

sync

 sudo fstrim -v /
/: 695181312 bytes were trimmed

sync

sudo dd bs=4096 skip=2224384 count=256 if=/dev/mapper/lubuntu--vg-root | hexdump -C
00000000 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a |y.y.y.y.y.y.y.y.|
*
256+0 records in
256+0 records out
1048576 bytes (1,0 MB) copied, 0,00166157 s, 631 MB/s
00100000

I.e. the y pattern persists which indicates that TRIM doesn't work. See also my post on unix-stackexchange: http://unix.stackexchange.com/questions/85865/trim-with-lvm-and-dm-crypt

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: lvm2 2.02.95-6ubuntu4
ProcVersionSignature: Ubuntu 3.8.0-27.40-generic 3.8.13.4
Uname: Linux 3.8.0-27-generic x86_64
ApportVersion: 2.9.2-0ubuntu8.3
Architecture: amd64
Date: Sun Aug 18 14:31:46 2013
InstallationDate: Installed on 2013-08-03 (14 days ago)
InstallationMedia: Lubuntu 13.04 "Raring Ringtail" - Release amd64 (20130423.1)
MarkForUpload: True
SourcePackage: lvm2
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
dpslxoo (beagle-trash-mail) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in lvm2 (Ubuntu):
status: New → Confirmed
Revision history for this message
dpslxoo (beagle-trash-mail) wrote :

Any progress on this issue?

Revision history for this message
frostschutz (frostschutz) wrote :

It turns out to be a caching issue.

When a region of the SSD is trimmed, that data is lost. However for some unfathomable reason, Linux keeps the data in its caches. As a result, the 2nd dd call returns wrong data, making it seem as if the TRIM didn't work regardless whether it really did or not.

I am not sure if this is a kernel bug or not. It should not have any ill effects in practice (after all who usually tries to read from a freshly TRIMmed region?) but even so it seems like a waste of cache memory if it's occupied by invalidated data.

Anyway, drop caches before issuing the dd command.

    echo 1 | sudo tee /proc/sys/vm/drop_caches

    sudo dd bs=4096 skip=2224384 count=256 if=/dev/mapper/lubuntu--vg-root | hexdump -C

Then see if it still returns y.y.y. or not. If it does not - trim has been working as it is supposed to be.

Also the issue_discards in LVM is not strictly necessary, it only affects lvremove, lvreduce, etc. Filesystem based TRIM seem to be passed on by LVM unconditionally.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.