Kernel 3.13.0-77 crashes (can be triggered by Samba)

Status changed to 'Confirmed' because the bug affects multiple users.

We've seen similar reports of (always) smbd getting stuck with our own 4.1.16 kernel: <https://forge.univention.org/bugzilla/show_bug.cgi?id=40558>

The common patch from my research so far looks like "unix: properly account for FDs passed over unix sockets", which might have introduced some suitable locking problem. It has been back-ported from 4.4 to several kernels by now.

I'm not able to reproduce the bug with samba3.raw.composite in my 8 vCPU KVM environment.
Some time ago I had a similar bug which I could only trigger on real-HW with 8 CPU.

Reverting the patch "unix: avoid use-after-free in ep_remove_wait_queue" in 4.1 fixes my problem (for now). The original patch went into 4.4, but was back-ported to several stable trees:

v3.2: a3b0f6e8a21ef02f69a15abac440572d8cde8c2a
v3.18: 72032798034d921ed565e3bf8dfdc3098f6473e2
v4.1: 5c77e26862ce604edea05b3442ed765e9756fe0f
v4.2: bad967fdd8ecbdd171f5f243657be033d2d081a7
v4.3: 58a6a46a036ce81a2a8ecaa6fc1537c894349e3f
v4.4: 7d267278a9ece963d77eefec61630223fce08c6c

See <https://lkml.org/lkml/2016/2/2/474>

https://forge.univention.org/bugzilla/show_bug.cgi?id=40558#c11

Indicates that commit
51cd3ed4c41b3895869925b99dd95a704bd2c91a unix: avoid use-after-free in ep_remove_wait_queue
should be reverted in order to avoid this regression.

Can we please get some progress on this?

Hello,

Am 23.02.2016 um 08:34 schrieb Stefan Metzmacher:
> https://forge.univention.org/bugzilla/show_bug.cgi?id=40558#c11
>
> Indicates that commit
> 51cd3ed4c41b3895869925b99dd95a704bd2c91a unix: avoid use-after-free in ep_remove_wait_queue
> should be reverted in order to avoid this regression.
>
> Can we please get some progress on this?

Rainer Weikusat sent a patch named
 [PATCH net] af_unix: Guard against other == sk in unix_dgram_sendmsg
 <https://patchwork.ozlabs.org/patch/582017/>
which fixes the problem.

For our distribution we released chose to revert the original patch as
we needed a working kernel as fast as possible, as several of our
customers were hit by that bug.

I tested the patch from Rainer and it also made the bug disappear.
David Miller also picked the patch for stable and we will do the same
when next be build a new kernel for our release.

Philipp

Ping!

This is the 4.5 commit: <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a5527dda344fff0514b7989ef7a755729769daa1>

For 4.4 it is in review right now for 4.4.4 as announced by greg k-h yesterday: <https://lkml.org/lkml/2016/3/1/828>

More broken kernel versions:
v3.14: 9d054f57adc981a5f503d5eb9b259aa450b90dc5
v3.12: 9964b4c4ee925b2910723e509abd7241cff1ef84
v3.10: da8db0830a2ce63f628150307a01a315f5081202
ckt/linux-3.13.y: 6505b15f7f7efde1853b5a7641e9ce675c2b1a96
v3.4: -
v3.2: a3b0f6e8a21ef02f69a15abac440572d8cde8c2a

Thanks Philipp!

I just hope to trigger some reaction from the ubuntu maintainers
in order get a usable kernel more than two week after breaking it.

Any news on this one?

This fix (a5527dd af_unix: Guard against other == sk in unix_dgram_sendmsg) has now been committed to Trusty, scheduled for the next kernel release (3.13.0-84.128).

Are the 3.13.0-84.128 packages available somewhere for testing?

The 3.13.0-84.128 packages (including this fix) are currently building, here:
https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+packages?field.series_filter=trusty

After the builds finish (several hours), the packages will enter the -proposed archive and an announcement will be posted here (likely tomorrow).

The Trusty 3.13.0-84.128 kernel packages (including this fix) are now available for testing in the -proposed archive. To test, select your architecture from the "Published Versions" here:

  https://launchpad.net/ubuntu/trusty/+package/linux-image-3.13.0-84-generic

Then download and install the "linux-image...deb" package under "Downloadable files".

Could not reproduce with 3.13.0-84-generic :-)

Thanks a lot!

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Already confirmed; Thanks @Karolin.

This bug was fixed in the package linux - 3.13.0-85.129

---------------
linux (3.13.0-85.129) trusty; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1558727

  [ Upstream Kernel Changes ]

  * Revert "Revert "af_unix: Revert 'lock_interruptible' in stream receive
    code""

linux (3.13.0-84.128) trusty; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1557596

  [ Upstream Kernel Changes ]

  * Revert "af_unix: Revert 'lock_interruptible' in stream receive code"
    - LP: #1540731
  * seccomp: cap SECCOMP_RET_ERRNO data to MAX_ERRNO
    - LP: #1496073
  * net/mlx4_en: Remove dependency between timestamping capability and
    service_task
    - LP: #1537859
  * net/mlx4_en: Fix HW timestamp init issue upon system startup
    - LP: #1537859
  * x86/mm: Fix slow_virt_to_phys() for X86_PAE again
    - LP: #1549601
  * iw_cxgb3: Fix incorrectly returning error on success
    - LP: #1557191
  * EVM: Use crypto_memneq() for digest comparisons
    - LP: #1557191
  * x86/entry/compat: Add missing CLAC to entry_INT80_32
    - LP: #1557191
  * iio: dac: mcp4725: set iio name property in sysfs
    - LP: #1557191
  * iommu/vt-d: Fix 64-bit accesses to 32-bit DMAR_GSTS_REG
    - LP: #1557191
  * PCI/AER: Flush workqueue on device remove to avoid use-after-free
    - LP: #1557191
  * libata: disable forced PORTS_IMPL for >= AHCI 1.3
    - LP: #1557191
  * mac80211: start_next_roc only if scan was actually running
    - LP: #1557191
  * mac80211: Requeue work after scan complete for all VIF types.
    - LP: #1557191
  * rfkill: fix rfkill_fop_read wait_event usage
    - LP: #1557191
  * crypto: shash - Fix has_key setting
    - LP: #1557191
  * drm/i915/dp: fall back to 18 bpp when sink capability is unknown
    - LP: #1557191
  * target: Fix WRITE_SAME/DISCARD conversion to linux 512b sectors
    - LP: #1557191
  * crypto: algif_hash - wait for crypto_ahash_init() to complete
    - LP: #1557191
  * iio: inkern: fix a NULL dereference on error
    - LP: #1557191
  * intel_scu_ipcutil: underflow in scu_reg_access()
    - LP: #1557191
  * ALSA: seq: Fix race at closing in virmidi driver
    - LP: #1557191
  * ALSA: rawmidi: Remove kernel WARNING for NULL user-space buffer check
    - LP: #1557191
  * ALSA: pcm: Fix potential deadlock in OSS emulation
    - LP: #1557191
  * ALSA: seq: Fix yet another races among ALSA timer accesses
    - LP: #1557191
  * ALSA: timer: Fix link corruption due to double start or stop
    - LP: #1557191
  * libata: fix sff host state machine locking while polling
    - LP: #1557191
  * cputime: Prevent 32bit overflow in time[val|spec]_to_cputime()
    - LP: #1557191
  * ASoC: dpcm: fix the BE state on hw_free
    - LP: #1557191
  * module: wrapper for symbol name.
    - LP: #1557191
  * ALSA: hda - Add fixup for Mac Mini 7,1 model
    - LP: #1557191
  * ALSA: Move EXPORT_SYMBOL() in appropriate places
    - LP: #1557191
  * ALSA: rawmidi: Make snd_rawmidi_transmit() race-free
    - LP: #1557191
  * ALSA: rawmidi: Fix race at copying & updating the position
    - LP: #1557191
  * ALSA: seq: Fix lockdep warnings due to double mutex locks
    - LP: #1557191
  * drivers/scsi/sg.c: mark VMA as VM_IO...