[Hyper-V] Rebase Hyper-V to 4.7.2 (stable)

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1616677

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Feature request, no logs required.

All patches are upstream

git://kernel.ubuntu.com/rtg/ubuntu-xenial.git hv-lp1616677

Test kernel at http://people.canonical.com/~rtg/hv-lp1616677/

Thanks, we'll run it through our tests.

Hi,

I tested SR-IOV feature on Hyper-V and it's not working.
Could you please include the following commits from git.kernel.org?
2f8214fe6811a246265629d81af2313695c63f4d
c6d45171d706c2b5efa3d5ee7a8260c14b6367c0
7921f4dc4c36e736d7a5b45dfa7b6a755a4fc012
b4363fbd8df2be23439e15a53b4040897228c481

Those should resolve this SR-IOV issue.

Thanks

@timg-tpi, the commits mentioned by Adrian above are actually to the ixgbevf driver, updating that VF driver for Hyper-V, not Hyper-V itself. It is a driver we can update as part of this process or should I file a separate LP? The commits have been upstream since April.

We've tested the kernel ~rtg/hv-lp1616677 provided and it passed the quality and functional testing.

There are 2 issues that are to be handled:
- missing SR-IOV patches - see comments #6 and #7. These should be included.
- kdump triggered through NMI doesn't work. We'll identify the missing patches and submit a separate request to include those.

Thank you!

Noted, though it may take a couple days to get to the SR-IOV patches.

We also passed functional and performance testing on the Azure side. This kernel looks good, and the SR-IOV driver ixgbevf is all that remains. Sorry for not including that in my initial request. We do performance testing with that driver in our performance scenarios.

New test kernel with the requested commits from #6:

http://people.canonical.com/~rtg/hv-lp1616677-b4125ba9d4ac428623484eb267c30ba2bd321df6/

Refreshed branch at 'git://kernel.ubuntu.com/rtg/ubuntu-xenial.git hv-lp1616677'

Joshua - are y'all ever gonna retest this ?

The SR-IOV test is still pending. In progress.

Thanks, --jrp

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Tim Gardner
Sent: Tuesday, September 13, 2016 08:41
To: Josh Poulson <email address hidden>
Subject: [Bug 1616677] Re: [Hyper-V] Rebase Hyper-V to 4.7.2 (stable)

Joshua - are y'all ever gonna retest this ?

--
You received this bug notification because you are subscribed to the bug report.
https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs.launchpad.net%2fbugs%2f1616677&data=02%7c01%7cjopoulso%40microsoft.com%7ca03647aed6164c1b8f1b08d3dbeddcfa%7c72f988bf86f141af91ab2d7cd011db47%7c1%7c0%7c636093787054915770&sdata=LQE2jeOCNxemBwzTe5t5eZdmBx7dgl3UhWA9olw8KZ8%3d

Title:
  [Hyper-V] Rebase Hyper-V to 4.7.2 (stable)

Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  In Progress
Status in linux source package in Yakkety:
  Fix Released

Bug description:
  Please rebase the Hyper-V drivers and tools in lts-xenial (and
  yakkity) to the final v4.7.2 stable upstream kernel.

  The following files comprise Hyper-V support in the upstream kernel:
  arch/x86/kernel/cpu/mshyperv.c
  arch/x86/include/asm/mshyperv.h
  arch/x86/include/uapi/asm/hyperv.h
  include/linux/hyperv.h
  drivers/hv/channel.c
  drivers/hv/channel_mgmt.c
  drivers/hv/connection.c
  drivers/hv/hv_balloon.c
  drivers/hv/hv.c
  drivers/hv/hv_fcopy.c
  drivers/hv/hv_kvp.c
  drivers/hv/hv_snapshot.c
  drivers/hv/hv_util.c
  drivers/hv/hv_utils_transport.c
  drivers/hv/hv_utils_transport.h
  drivers/hv/hyperv_vmbus.h
  drivers/hv/ring_buffer.c
  drivers/hv/vmbus_drv.c
  tools/hv/hv_fcopy_daemon.c
  tools/hv/hv_get_dhcp_info.sh
  tools/hv/hv_get_dns_info.sh
  tools/hv/hv_kvp_daemon.c
  tools/hv/hv_set_ifconfig.sh
  tools/hv/hv_vss_daemon.c
  tools/hv/lsvmbus
  drivers/input/serio/hyperv-keyboard.c
  drivers/net/hyperv/hyperv_net.h
  drivers/net/hyperv/netvsc.c
  drivers/net/hyperv/netvsc_drv.c
  drivers/net/hyperv/rndis_filter.c
  drivers/scsi/storvsc_drv.c
  drivers/hid/hid-hyperv.c
  drivers/pci/host/pci-hyperv.c
  drivers/video/fbdev/hyperv_fb.c

  While I know a number of post-4.6 commits are already included, I am
  including all of the commits from 4.6 to 4.7.2 for reference:

  channel_mgmt.c : commit cd95aad5579371ac332507fc946008217fc37e6c : Drivers: hv: vmbus: handle various crash scenarios
  connection.c : commit 5cc472477f928fb8584eb8e08245c9cf9002d74a : Drivers: hv: vmbus: Export the vmbus_set_event() API
  hv_balloon.c : commit 77c0c9735bc0ba5898e637a3a20d6bcb50e3f67d : Drivers: hv: balloon: don't crash when memory is added in non-sorted order
  hv_balloon.c : commit d19a55d6ed5bf0ffe553df2d8bf91d054ddf2d76 : Drivers: hv: balloon: reset host_specified_ha_region
  hv_kvp.c : commit 4dbfc2e68004c60edab7e8fd26784383dd3ee9bc : Drivers: hv: kvp: fix IP Failover
  hyperv_fb.c : commit 696ca5e82c057a272381ae6064d59eb97a578397 : drivers:hv: Use new vmbus_mmio_free() from client drivers.
  hyperv.h : commit 97fb77dc87582300fa3c141b63699f853576cab1 : drivers:hv: M...

SR-IOV test still pending, we're on it.

I'm acking this kernel. There are additional SR-IOV issues we are tracking down upstream that remain.

https://lists.ubuntu.com/archives/kernel-team/2016-September/079965.html

Hi Joshua-

Before applying this, we'd like more details about the "additional SR-IOV issues" you noted in comment #15. Can you clarify what those issues are, and what level of testing has been done to validate the ixgbevf changes? Thanks!

Hi,

The rebase is good, you can apply it.

Sorry for the (very) long wait, we had some issues with the test environment which gave us some false positives.

Thanks!

@kamalmostafa

Basically unexplained crashes in upstream 4.8-rc5 (but not with your test kernel)... but only related to SR-IOV. I generally only ask for stable, tested patches in rebases, the four patches above are targeted to SR-IOV and it's unclear if something else in the upstream is breaking them. RCs do have to get burned in.

Thanks for the clarification, Joshua. We have committed this rebase to Xenial.

I see the 4.4.0-38 SRU went out. Eagerly awaiting -proposed.

Just to confirm, were the SR-IOV drivers updated in Yakkety or should I follow up on those in a separate bug?

After Ignite we will turn up the heat on 16.10. (pun intended)

@josh

Should be in -proposed now (as soon as the publisher finishes)

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Thanks Brad!
We're doing a test pass on the proposed kernel and update this thread once we have the results.

Marking this as verification-done.
Functional testing are looking good and performance for STOR and NET are in par with similar kernels.

Functional and performance testing are also complete on the Azure side with the same results. Thanks!

This bug was fixed in the package linux - 4.4.0-42.62

---------------
linux (4.4.0-42.62) xenial; urgency=low

  * Fix GRO recursion overflow for tunneling protocols (LP: #1631287)
    - tunnels: Don't apply GRO to multiple layers of encapsulation.
    - gro: Allow tunnel stacking in the case of FOU/GUE

  * CVE-2016-7039
    - SAUCE: net: add recursion limit to GRO

linux (4.4.0-41.61) xenial; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
    - LP: #1628204

  * nvme drive probe failure (LP: #1626894)
    - (fix) NVMe: Don't unmap controller registers on reset

linux (4.4.0-40.60) xenial; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
    - LP: #1627074

  * Permission denied in CIFS with kernel 4.4.0-38 (LP: #1626112)
    - Fix memory leaks in cifs_do_mount()
    - Compare prepaths when comparing superblocks
    - SAUCE: Fix regression which breaks DFS mounting

  * Backlight does not change when adjust it higher than 50% after S3
    (LP: #1625932)
    - SAUCE: i915_bpo: drm/i915/backlight: setup and cache pwm alternate
      increment value
    - SAUCE: i915_bpo: drm/i915/backlight: setup backlight pwm alternate
      increment on backlight enable

linux (4.4.0-39.59) xenial; urgency=low

  [ Joseph Salisbury ]

  * Release Tracking Bug
    - LP: #1625303

  * thunder: chip errata w/ multiple CQEs for a TSO packet (LP: #1624569)
    - net: thunderx: Fix for issues with multiple CQEs posted for a TSO packet

  * thunder: faulty TSO padding (LP: #1623627)
    - net: thunderx: Fix for HW issue while padding TSO packet

  * CVE-2016-6828
    - tcp: fix use after free in tcp_xmit_retransmit_queue()

  * Sennheiser Officerunner - cannot get freq at ep 0x83 (LP: #1622763)
    - SAUCE: (no-up) ALSA: usb-audio: Add quirk for sennheiser officerunner

  * Backport E3 Skylake Support in ie31200_edac to Xenial (LP: #1619766)
    - EDAC, ie31200_edac: Add Skylake support

  * Ubuntu 16.04 - Full EEH Recovery Support for NVMe devices (LP: #1602724)
    - SAUCE: nvme: Don't suspend admin queue that wasn't created

  * ISST-LTE:pNV: system ben is hung during ST (nvme) (LP: #1620317)
    - blk-mq: Allow timeouts to run while queue is freezing
    - blk-mq: improve warning for running a queue on the wrong CPU
    - blk-mq: don't overwrite rq->mq_ctx

  * lsattr 32bit does not work on 64bit kernel (Inappropriate ioctl error)
    (LP: #1619918)
    - btrfs: bugfix: handle FS_IOC32_{GETFLAGS, SETFLAGS, GETVERSION} in
      btrfs_ioctl

  * radeon: monitor connected to onboard VGA doesn't work with Xenial
    (LP: #1600092)
    - drm/radeon/dp: add back special handling for NUTMEG

  * initramfs includes qle driver, but not firmware (LP: #1623187)
    - qed: add MODULE_FIRMWARE()

  * [Hyper-V] Rebase Hyper-V to 4.7.2 (stable) (LP: #1616677)
    - hv_netvsc: Implement support for VF drivers on Hyper-V
    - hv_netvsc: Fix the list processing for network change event
    - Drivers: hv: vmbus: Introduce functions for estimating room in the ring
      buffer
    - Drivers: hv: vmbus: Use READ_ONCE() to read variables that are volatile
    - Drivers: hv: vmbus: Export the vmbus_set_event() API
    - lcoking/barriers, arch: Use smp barriers...