SRU: Stop kswapd consuming 100% CPU when highest zone is small

gregkh has declined Mel's patchset for 2.6.38.y as it is no longer supported as a stable release.

@Tim, does this invalid these patches for a SRU then? Or can we still run with them?

Colin - We could still SRU theses for Natty, but I'd definitely need a reproducer. I assume you have a reliable method for demonstrating the problem?

From the posting: The reproduction case is almost always during
copying large files that kswapd pegs at 100% CPU until the file is
deleted or cache is dropped.

I believe I've seen this behaviour on earlier hardware than sandybridge, I can trigger it with good reliability using abocks image-writer to create a liveusb version of the GNOME3 isos (from gnome.org) however the hardware is question unfortunately died recently. Any large file copy operation should trigger it on sandybridge according to Mel though..

Colin - In your copious spare time perhaps you could try building this Natty master-next kernel and testing it on an SB:

The following changes since commit 006d932cf137dfcc156647dc08a0dda6e3e0f890:
  Tao Ma (1):
        ext4: init timer earlier to avoid a kernel panic in __save_error_info, CVE-2011-2493

are available in the git repository at:

  git://kernel.ubuntu.com/rtg/ubuntu-natty.git lp808509-kswapd

Mel Gorman (5):
      mm: vmscan: correct check for kswapd sleeping in sleeping_prematurely
      mm: vmscan: kswapd should not free an excessive number of pages when balancing small zones
      mm: vmscan: do not apply pressure to slab if we are not applying pressure to zone
      mm: vmscan: evaluate the watermarks against the correct classzone
      mm: vmscan: only read new_classzone_idx from pgdat when reclaiming successfully

 include/linux/swap.h | 9 ++++++
 mm/vmscan.c | 75 ++++++++++++++++++++++++++++++++------------------
 2 files changed, 57 insertions(+), 27 deletions(-)

@Tim, Will do, however I will probably get around to this on Thursday as I'm off on sick leave until then. I have a good set of reproducer scripts + H/W that can faithfully trigger the bug.

@Tim, managed to get some overnight testing on this kernel, results are very good:

Did 1000 x test cycles, each copying ~795MB of data (~7 hours of soak testing), no lock-ups on a SNB laptop with 4GB memory. Also re-ran with 500 x test cycles with 2GB memory (as this *really* exercises the bug). My test script (which contains portions of the ubiquity copying script which originally tripped this bug) is attached. Testing - run script to copy data from one ext4 partition to another.

/dev/sda3, ext4, source, containing ~795MB of files from the Natty installer
/dev/sda4, ext4, destination

Summary: Exhaustive testing passed.

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-natty' to 'verification-done-natty'.

If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Also would be good to have a formal SRU justification here.

I've repeated the test scenario as described in comment #7 with 500 copy operations using the script (see comment #7 too) to give it nearly 4 hours of intensive file copying soak testing. Works fine, no lock-ups/hangs.

Marking it as verified.

SRU Justification

Impact: Sandy Bridge platforms with a small normal zone can apparently lockup when copying from a fast I/O source to a slower I/O destination, e.g., disk to USB stick.

Patch Description: A number of upstream changes have made accommodation for how pages are allocated when the normal zone is exhausted.

Patch: See the list of commits in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/808509/comments/5

This bug was fixed in the package linux - 2.6.38-11.48

---------------
linux (2.6.38-11.48) natty-proposed; urgency=low

  [Herton R. Krzesinski]

  * Release Tracking Bug
    - LP: #818175

  [ Upstream Kernel Changes ]

  * Revert "HID: magicmouse: ignore 'ivalid report id' while switching
    modes"
    - LP: #814250

linux (2.6.38-11.47) natty-proposed; urgency=low

  [Steve Conklin]

  * Release Tracking Bug
    - LP: #811180

  [ Keng-Yu Lin ]

  * SAUCE: Revert: "dell-laptop: Toggle the unsupported hardware
    killswitch"
    - LP: #775281

  [ Ming Lei ]

  * SAUCE: fix yama_ptracer_del lockdep warning
    - LP: #791019

  [ Stefan Bader ]

  * SAUCE: Re-enable RODATA for i386 virtual
    - LP: #809838

  [ Tim Gardner ]

  * [Config] Add grub-efi as a recommended bootloader for server and
    generic
    - LP: #800910
  * SAUCE: rtl8192se: Force a build for a 2.6/3.0 kernel
    - LP: #805494

  [ Upstream Kernel Changes ]

  * Revert "bridge: Forward reserved group addresses if !STP"
    - LP: #793702
  * Fix up ABI directory
  * bonding: Incorrect TX queue offset, CVE-2011-1581
    - LP: #792312
    - CVE-2011-1581
  * fs/partitions/efi.c: corrupted GUID partition tables can cause kernel
    oops
    - LP: #795418
    - CVE-2011-1577
  * usbnet/cdc_ncm: add missing .reset_resume hook
    - LP: #793892
  * ath5k: Disable fast channel switching by default
    - LP: #767192
  * mm: vmscan: correctly check if reclaimer should schedule during
    shrink_slab
    - LP: #755066
  * mm: vmscan: correct use of pgdat_balanced in sleeping_prematurely
    - LP: #755066
  * ALSA: hda - Use LPIB for ATI/AMD chipsets as default
    - LP: #741825
  * ALSA: hda - Enable snoop bit for AMD controllers
    - LP: #741825
  * ALSA: hda - Enable sync_write workaround for AMD generically
    - LP: #741825
  * cpuidle: menu: fixed wrapping timers at 4.294 seconds
    - LP: #774947
  * drm/i915: Fix gen6 (SNB) missed BLT ring interrupts.
    - LP: #761065
  * USB: ehci: remove structure packing from ehci_def
    - LP: #791552
  * drm/i915: disable PCH ports if needed when disabling a CRTC
    - LP: #791752
  * kmemleak: Do not return a pointer to an object that kmemleak did not
    get
    - LP: #793702
  * kmemleak: Initialise kmemleak after debug_objects_mem_init()
    - LP: #793702
  * Fix _OSC UUID in pcc-cpufreq
    - LP: #793702
  * CPU hotplug, re-create sysfs directory and symlinks
    - LP: #793702
  * Fix memory leak in cpufreq_stat
    - LP: #793702
  * net: recvmmsg: Strip MSG_WAITFORONE when calling recvmsg
    - LP: #793702
  * ftrace: Only update the function code on write to filter files
    - LP: #793702
  * qla2xxx: Fix hang during driver unload when vport is active.
    - LP: #793702
  * qla2xxx: Fix virtual port failing to login after chip reset.
    - LP: #793702
  * qla2xxx: Fix vport delete hang when logins are outstanding.
    - LP: #793702
  * powerpc/kdump64: Don't reference freed memory as pacas
    - LP: #793702
  * powerpc/kexec: Fix memory corruption from unallocated slaves
    - LP: #793702
  * x86, cpufeature: Fix cpuid leaf 7 feature detection
    - LP: #793702
  * ath9k_hw: do noise floor calibration only on required chain...