Ubuntu
linux package

shiftfs: drop entries from cache on unlink

Bug #1841977 reported by Christian Brauner
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
Undecided
Christian Brauner
Disco
Fix Released
Medium
Christian Brauner

Bug Description

SRU Justification

Impact: LXD on Ubuntu runs on top of zfs by defaults. Users that make use of shiftfs for efficient id-shifting currently hit a bug where zfs is confused about the amount of space that is used in a dataset. For example, creating a file with 1GB of random data will increase the space used by the dataset by 1GB. When the file is removed via rm the space is not freed for zfs. This leads to zfs running out of space pretty quickly.
This bug has been observed, described, and reproduced here https://discuss.linuxcontainers.org/t/trying-out-shiftfs/5155/9 . Stéphane Graber observed related issues.

Regression Potential: Limited to shiftfs. This patch has been tested on various backends btrfs, dir, zfs to verify that it doesn't regress other workloads. Shiftfs now also aligns more closely with overlayfs on file deletion.

Test Case:
sudo snap install lxd
sudo snap set lxd shiftfs.enable=true
sudo systemctl restart snap.lxd.daemon
sudo lxd init # make sure to select zfs as backend
sudo lxc launch images:ubuntu/bionic b1
sudo lxc exec b1 -- dd if=/dev/urandom bs=1M count=1000 of=dummy.file
sudo zfs list default/containers/b1 # will show +1GB
sudo lxc exec b1 -- rm dummy.file
sudo zfs list default/containers/b1 # will show +1GB on a non-fixed kernel and -1GB on a fixed kernel

Target Kernels: All LTS kernels with shiftfs support.

Christian Brauner (cbrauner)
Changed in linux (Ubuntu):
assignee: nobody → Christian Brauner (cbrauner)
status: New → In Progress
Seth Forshee (sforshee)
Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 5.3.0-10.11

---------------
linux (5.3.0-10.11) eoan; urgency=medium

  * eoan/linux: 5.3.0-10.11 -proposed tracker (LP: #1843232)

  * No sound inputs from the external microphone and headset on a Dell machine
    (LP: #1842265)
    - SAUCE: ALSA: hda - Expand pin_match function to match upcoming new tbls
    - SAUCE: ALSA: hda - Define a fallback_pin_fixup_tbl for alc269 family

  * Horizontal corrupted line at top of screen caused by framebuffer compression
    (LP: #1840236)
    - SAUCE: drm/i915/fbc: disable framebuffer compression on IceLake

  * Add bpftool to linux-tools-common (LP: #1774815)
    - [Debian] package bpftool in linux-tools-common

  * Miscellaneous Ubuntu changes
    - update dkms package versions

  [ Upstream Kernel Changes ]

  * Rebase to v5.3-rc8

 -- Paolo Pisati <email address hidden> Mon, 09 Sep 2019 10:00:41 +0200

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Stefan Bader (smb)
Changed in linux (Ubuntu Disco):
importance: Undecided → Medium
Kleber Sacilotto de Souza (kleber-souza)
Changed in linux (Ubuntu Disco):
status: New → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco
Christian Brauner (cbrauner)
tags: added: verification-done-disco
removed: verification-needed-disco
Changed in linux (Ubuntu Disco):
assignee: nobody → Christian Brauner (cbrauner)
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (22.6 KiB)

This bug was fixed in the package linux - 5.0.0-32.34

---------------
linux (5.0.0-32.34) disco; urgency=medium

  * disco/linux: 5.0.0-32.34 -proposed tracker (LP: #1846097)

  * CVE-2019-14814 // CVE-2019-14815 // CVE-2019-14816
    - mwifiex: Fix three heap overflow at parsing element in cfg80211_ap_settings

  * CVE-2019-15505
    - media: technisat-usb2: break out of loop at end of buffer

  * CVE-2019-2181
    - binder: check for overflow when alloc for security context

  * Support Hi1620 zip hw accelerator (LP: #1845355)
    - [Config] Enable HiSilicon QM/ZIP as modules
    - crypto: hisilicon - add queue management driver for HiSilicon QM module
    - crypto: hisilicon - add hardware SGL support
    - crypto: hisilicon - add HiSilicon ZIP accelerator support
    - crypto: hisilicon - add SRIOV support for ZIP
    - Documentation: Add debugfs doc for hisi_zip
    - crypto: hisilicon - add debugfs for ZIP and QM
    - MAINTAINERS: add maintainer for HiSilicon QM and ZIP controller driver
    - crypto: hisilicon - fix kbuild warnings
    - crypto: hisilicon - add dependency for CRYPTO_DEV_HISI_ZIP
    - crypto: hisilicon - init curr_sgl_dma to fix compile warning
    - crypto: hisilicon - add missing single_release
    - crypto: hisilicon - fix error handle in hisi_zip_create_req_q
    - crypto: hisilicon - Fix warning on printing %p with dma_addr_t
    - crypto: hisilicon - Fix return value check in hisi_zip_acompress()
    - crypto: hisilicon - avoid unused function warning

  * xfrm interface: several kernel panic (LP: #1836261)
    - xfrm interface: fix memory leak on creation
    - xfrm interface: avoid corruption on changelink
    - xfrm interface: ifname may be wrong in logs
    - xfrm interface: fix list corruption for x-netns
    - xfrm interface: fix management of phydev

  * shiftfs: drop entries from cache on unlink (LP: #1841977)
    - SAUCE: shiftfs: fix buggy unlink logic

  * shiftfs: mark kmem_cache as reclaimable (LP: #1842059)
    - SAUCE: shiftfs: mark slab objects SLAB_RECLAIM_ACCOUNT

  * Suspend to RAM(S3) does not wake up for latest megaraid and mpt3sas
    adapters(SAS3.5 onwards) (LP: #1838751)
    - PCI: Restore Resizable BAR size bits correctly for 1MB BARs

  * No sound inputs from the external microphone and headset on a Dell machine
    (LP: #1842265)
    - ALSA: hda - Expand pin_match function to match upcoming new tbls
    - ALSA: hda - Define a fallback_pin_fixup_tbl for alc269 family

  * Add -fcf-protection=none when using retpoline flags (LP: #1843291)
    - SAUCE: kbuild: add -fcf-protection=none when using retpoline flags

  * Disco update: upstream stable patchset 2019-09-25 (LP: #1845390)
    - bridge/mdb: remove wrong use of NLM_F_MULTI
    - cdc_ether: fix rndis support for Mediatek based smartphones
    - ipv6: Fix the link time qualifier of 'ping_v6_proc_exit_net()'
    - isdn/capi: check message length in capi_write()
    - ixgbe: Fix secpath usage for IPsec TX offload.
    - net: Fix null de-reference of device refcount
    - net: gso: Fix skb_segment splat when splitting gso_size mangled skb having
      linear-headed frag_list
    - net: phylink: Fix flow control resolution
    - net: s...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (linux-bluefield/5.0.0-1003.12)

All autopkgtests for the newly accepted linux-bluefield (5.0.0-1003.12) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

fsprotect/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-bluefield

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.