cp file from external HDD to NFS on Pi 3B causes ubuntu 19.10 and 20.04 kernel to hang

Bug #1882213 reported by David Woodgate
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux-raspi (Ubuntu)
Confirmed
Undecided
Unassigned
Eoan
Won't Fix
Undecided
Unassigned
Focal
Invalid
Undecided
Unassigned
Groovy
Invalid
Undecided
Unassigned

Bug Description

Copying a large file using cp on my Raspberry Pi 3B (not 3B+) from an externally-powered USB hard disk (Seagate Expansion 2TB) to an NFS-mounted filesystem across Ethernet causes the kernel of releases 19.10 and 20.04 to hang/freeze.

This happens every time the copy is attempted, although the exact duration from the start of the copy to the hang varies from a few seconds to a couple of minutes. The kernel does not recover even after long waits (30mins+), and a power reset (pulling the plug) is required. (Although I admit I haven't tried yet simply disconnecting the USB cable of the HDD during a hang. Update: I've now tried that - the system doesn't recover after pulling the USB cable out.)

Likewise, messages are erratic and I suspect are only symptoms, notably watchdog BUG soft lockup messages and RCU stall messages on different CPU cores for different processes/threads. There are no hardware messages that I've seen recorded in log files (or appearing on the console) matching the incidents.

When a hang happen, network connectivity is lost - ssh sessions fail, and the file copy stops (even when the cp is run on the console). Pings to the system from other hosts on the same Ethernet LAN (and switch) fail with destination host unreachable.

Although I have had a swapfile configured on the SD card, the hangs continue to happen doing the cp after "swapoff -a". I've also tried playing with taskset affinities and snappiness to no obvious effect - the hangs still keep happening.

Copying from the external HDD to the SD card is successful, as is copying from the SD card to the NFS.

Raspbian 4.19.0-8 (armmp-lpae) does not have this problem; it successfully copies the file from HDD to NFS across Ethernet without stalling.

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-1011-raspi 5.4.0-1011.11
ProcVersionSignature: Ubuntu 5.4.0-1011.11-raspi 5.4.34
Uname: Linux 5.4.0-1011-raspi aarch64
ApportVersion: 2.20.11-0ubuntu27
Architecture: arm64
CasperMD5CheckResult: skip
Date: Fri Jun 5 07:37:44 2020
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
SourcePackage: linux-raspi
UpgradeStatus: Upgraded to focal on 2020-05-21 (14 days ago)

Revision history for this message
David Woodgate (ub4pi) wrote :
David Woodgate (ub4pi)
description: updated
Revision history for this message
Francis Ginther (fginther) wrote :

I hit this during scp of a large file (> 2GB) from a usb thumbdrive to a local host on my network:

scp /media/usbstick/large-file.tar.gz my@myserver:/tmp/

There were no messages to the vga console. I tried it 3 times and it hung twice. Had to pull power to get the system back.

Revision history for this message
Juerg Haefliger (juergh) wrote :

I cannot reproduce this. Copied a 3GB file from a USB attached HD to an NFS mount. 10 times in a loop, works just fine.

$ uname -a
Linux rpi-3b-rev1d2-62e2 5.4.0-1011-raspi #11-Ubuntu SMP Fri May 8 07:43:33 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

$ cat /proc/device-tree/model
Raspberry Pi 3 Model B Rev 1.2

Revision history for this message
Brian Murray (brian-murray) wrote :

The Eoan Ermine has reached end of life, so this bug will not be fixed for that release

Changed in linux-raspi (Ubuntu Eoan):
status: New → Won't Fix
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux-raspi (Ubuntu Focal):
status: New → Confirmed
Changed in linux-raspi (Ubuntu):
status: New → Confirmed
Revision history for this message
krot (krot-mstu) wrote :

I have the same issue. I just copy files via network(via cable) with rsync from local PC to USB HDD (USB HDD is attached to raspberry). In 30-90 minutes Raspberry hangs every time. I made this test with 2 different hardware sets (Raspberry + SD card + external USB HDD). I had the same issue with different hardware.
I've added photo to attach.

Revision history for this message
krot (krot-mstu) wrote :
Revision history for this message
Juerg Haefliger (juergh) wrote :

@ub4pi @fginther @krot-mstu Is this still an issue with the latest kernel? Can you please retest?

Revision history for this message
Francis Ginther (fginther) wrote :

@juergh, I retested this with focal 5.4.0-1028-raspi and was unable to reproduce.

Revision history for this message
Juerg Haefliger (juergh) wrote :

Closing this as it's not reproduceable. Please open a new ticket if you run into the problem again.

Changed in linux-raspi (Ubuntu Focal):
status: Confirmed → Invalid
Changed in linux-raspi (Ubuntu Groovy):
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.