[Hyper-V] Rebase Hyper-V to upstream 4.11 kernel plus some cherry-picks from linux-next

Bug #1690177 reported by Joshua R. Poulson
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux-azure (Ubuntu)
Fix Committed
Undecided
Marcelo Cerri
Xenial
Fix Committed
Undecided
Marcelo Cerri

Bug Description

Catching up with upstream 4.11 Hyper-V files:
arch/x86/kernel/cpu/mshyperv.c
arch/x86/include/asm/mshyperv.h
arch/x86/include/uapi/asm/hyperv.h
include/linux/hyperv.h
drivers/hv/channel.c
drivers/hv/channel_mgmt.c
drivers/hv/connection.c
drivers/hv/hv_balloon.c
drivers/hv/hv.c
drivers/hv/hv_fcopy.c
drivers/hv/hv_kvp.c
drivers/hv/hv_snapshot.c
drivers/hv/hv_util.c
drivers/hv/hv_utils_transport.c
drivers/hv/hv_utils_transport.h
drivers/hv/hyperv_vmbus.h
drivers/hv/ring_buffer.c
drivers/hv/vmbus_drv.c
tools/hv/hv_fcopy_daemon.c
tools/hv/hv_get_dhcp_info.sh
tools/hv/hv_get_dns_info.sh
tools/hv/hv_kvp_daemon.c
tools/hv/hv_set_ifconfig.sh
tools/hv/hv_vss_daemon.c
tools/hv/lsvmbus
drivers/input/serio/hyperv-keyboard.c
drivers/net/hyperv/hyperv_net.h
drivers/net/hyperv/netvsc.c
drivers/net/hyperv/netvsc_drv.c
drivers/net/hyperv/rndis_filter.c
drivers/scsi/storvsc_drv.c
drivers/hid/hid-hyperv.c
drivers/pci/host/pci-hyperv.c
drivers/video/fbdev/hyperv_fb.c
drivers/uio/uio_hv_generic.c

This is the delta from 4.11 to 4.10.3:
mshyperv.c : commit 63ed4e0c67df332681ebfef6eca6852da28d6300 : Drivers: hv: vmbus: Consolidate all Hyper-V specific clocksource code
mshyperv.c : commit 8de8af7e0873c4fdac2205327dff922819e16657 : Drivers: hv: vmbus: Move the extracting of Hypervisor version information
mshyperv.c : commit 8730046c1498e8fb8c9a124789893944e8ce8220 : Drivers: hv vmbus: Move Hypercall page setup out of common code
hyperv.h : commit d058fa7e98ff01a4b4750a2210fc19906db3cbe1 : Drivers: hv: vmbus: Move the crash notification function
hyperv.h : commit e4165a0fad0963bf8b4a59f54d3360ccb6a6d1ea : vmbus: constify parameters where possible
hyperv.h : commit f6c4391553573d592212e6624cfba5e2752cd5c8 : vmbus: remove no longer used signal_policy
hyperv.h : commit 3454323c954775098871559b5c23d877c3e23f77 : vmbus: remove unused kickq argument to sendpacket
hyperv.h : commit 6e47dd3e2938f41d75045bbcb64aa9df3a463b2a : vmbus: expose hv_begin/end_read
channel.c : commit 5c1bec61fdfcd056df909a712e2a86bbaeb0f942 : vmbus: use kernel bitops for traversing interrupt mask
channel.c : commit 5e030d5ce9d99a899b648413139ff65bab12b038 : Drivers: hv: vmbus: Don't leak memory when a channel is rescinded
channel_mgmt.c : commit e810e48c0c9a1a1ebb90cfe966bce6dc80ce08e7 : Drivers: hv: vmbus: Move the code to signal end of message
channel_mgmt.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
connection.c : commit 37cdd991fac810a727cd285629d1640fcf53cd19 : vmbus: put related per-cpu variable together
connection.c : commit 8de8af7e0873c4fdac2205327dff922819e16657 : Drivers: hv: vmbus: Move the extracting of Hypervisor version information
connection.c : commit a100d88df1e924e5c9678fabf054d1bae7ab74fb : hv: use correct order when freeing monitor_pages
hv_balloon.c : commit ad6d41253bf91eabb41626683c35a712ba27a20c : Drivers: hv: balloon: add a fall through comment to hv_memory_notifier()
hv.c : commit 6ab42a66d2cc10afefea9f9e5d9a5ad5a836d254 : Drivers: hv: vmbus: Move Hypercall invocation code out of common code
hv.c : commit d5116b4091ecca271c249ede43a49c1245920558 : Drivers: hv: vmbus: Restructure the clockevents code
hv_fcopy.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_fcopy.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_kvp.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_kvp.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_snapshot.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_snapshot.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_util.c : commit 3716a49a81ba19dda7202633a68b28564ba95eb5 : hv_utils: implement Hyper-V PTP source
hv_util.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_utils_transport.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_utils_transport.h : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hyperv_vmbus.h : commit 37cdd991fac810a727cd285629d1640fcf53cd19 : vmbus: put related per-cpu variable together
hyperv_vmbus.h : commit 63ed4e0c67df332681ebfef6eca6852da28d6300 : Drivers: hv: vmbus: Consolidate all Hyper-V specific clocksource code
hyperv_vmbus.h : commit d6f3609d2b4c6d0eec01f398cb685e50da3e6013 : Drivers: hv: restore hypervcall page cleanup before kexec
hyperv_vmbus.h : commit e810e48c0c9a1a1ebb90cfe966bce6dc80ce08e7 : Drivers: hv: vmbus: Move the code to signal end of message
ring_buffer.c : commit 6e47dd3e2938f41d75045bbcb64aa9df3a463b2a : vmbus: expose hv_begin/end_read
ring_buffer.c : commit b103a56f80878b4370522ec26017fcadd526f5ad : vmbus: drop no longer used kick_q argument
ring_buffer.c : commit e4165a0fad0963bf8b4a59f54d3360ccb6a6d1ea : vmbus: constify parameters where possible
vmbus_drv.c : commit 68db0cf10678630d286f4bbbbdfa102951a35faa : sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h>
vmbus_drv.c : commit 37cdd991fac810a727cd285629d1640fcf53cd19 : vmbus: put related per-cpu variable together
vmbus_drv.c : commit 5c1bec61fdfcd056df909a712e2a86bbaeb0f942 : vmbus: use kernel bitops for traversing interrupt mask
vmbus_drv.c : commit d6f3609d2b4c6d0eec01f398cb685e50da3e6013 : Drivers: hv: restore hypervcall page cleanup before kexec
hyperv-keyboard.c : commit f20b161f59b73d40043199b3bff362e656a9c673 : Input: serio - drop unnecessary calls to device_init_wakeup
hyperv_net.h : commit 23312a3be999c22ec70c0fdf9f01cdee05fac986 : netvsc: negotiate checksum and segmentation parameters
hyperv_net.h : commit dc54a08cd3620e6457382c0cd0c8f03513dd749a : netvsc: optimize receive path
hyperv_net.h : commit 793e39555511bccd73308c41205b72448d0077db : netvsc: account for packets/bytes transmitted after completion
hyperv_net.h : commit 962f3fee83a4ef9010ae84dc43ae7aecb572e2a9 : netvsc: add ethtool ops to get/set RSS key
hyperv_net.h : commit 2b01888d1b453096b5b13c0d4b73e630411198b4 : netvsc: allow more flexible setting of number of channels
hyperv_net.h : commit fdfb70d275223b9d69d5d3abe1f88507da579139 : netvsc: fix calculation of available send sections
netvsc.c : commit fdfb70d275223b9d69d5d3abe1f88507da579139 : netvsc: fix calculation of available send sections
netvsc.c : commit 3454323c954775098871559b5c23d877c3e23f77 : vmbus: remove unused kickq argument to sendpacket
netvsc.c : commit 7ce101246655935b014b11d81f815342921f5654 : netvsc: handle select_queue when device is being removed
netvsc.c : commit 2c7f83ca713fa0ac5c4698e4134b09a355f60263 : netvsc: don't pass void * to internal device_add
netvsc_drv.c : commit b8b835a89b2f7a7fe681983dfe5c489cb9ad9500 : netvsc: group all per-channel state together
netvsc_drv.c : commit dc54a08cd3620e6457382c0cd0c8f03513dd749a : netvsc: optimize receive path
netvsc_drv.c : commit 1130383c174499826a3f01486e574e89be17e2d2 : netvsc: call netif_receive_skb
netvsc_drv.c : commit ceaaea0483cf82a2c817781f3cbacf29f2c5cc97 : netvsc: remove unused variables
netvsc_drv.c : commit d0d7b10b05945f40fefd4e60f457c61aefa3e9a9 : net-next: treewide use is_vlan_dev() helper function.
rndis_filter.c : commit 962f3fee83a4ef9010ae84dc43ae7aecb572e2a9 : netvsc: add ethtool ops to get/set RSS key
rndis_filter.c : commit dc54a08cd3620e6457382c0cd0c8f03513dd749a : netvsc: optimize receive path
rndis_filter.c : commit 23312a3be999c22ec70c0fdf9f01cdee05fac986 : netvsc: negotiate checksum and segmentation parameters
rndis_filter.c : commit ceaaea0483cf82a2c817781f3cbacf29f2c5cc97 : netvsc: remove unused variables
rndis_filter.c : commit 46b4f7f5d1f7410de48128540ef2d1aab913a619 : netvsc: eliminate per-device outstanding send counter
rndis_filter.c : commit 2289f0aa706e5160e078f73c32fcbfb56a3ff1e2 : netvsc: simplify rndis_filter_remove
storvsc_drv.c : commit 183b8021fc0a5fadecdf9c0ccac1f48b5c326278 : scripts/spelling.txt: add "intialization" pattern and fix typo instances
storvsc_drv.c : commit f1c635b439a5c01776fe3a25b1e2dc546ea82e6f : scsi: storvsc: Workaround for virtual DVD SCSI version
storvsc_drv.c : commit d86adf482b843b3a58a9ec3b7c1ccdbf7c705db1 : scsi: storvsc: Enable multi-queue support
storvsc_drv.c : commit f64dad2628bdf62eac7ac145a6e31430376b65e4 : scsi: storvsc: Enable tracking of queue depth
storvsc_drv.c : commit b6a05c823fc573a65efc4466f174abf05f922e0f : scsi: remove eh_timed_out methods in the transport template
pci-hyperv.c : commit 4a9b0933bdfcd85da840284bf5a0eb17b654b9c2 : PCI: hv: Use device serial number as PCI domain
uio_hv_generic.c : commit b71e328297a3a578c482fb4814e737a0ec185839 : vmbus: add direct isr callback mode

In addition, the following patches are in linux-next that are needed for larger VM sizes, network performance, and PCI passthrough fixes found in SR-IOV testing:

mshyperv.c : commit a33fd4c27b3ad11c66bdadc5fe6075297ca87a6d : Drivers: hv: Issue explicit EOI when autoeoi is not enabled
hyperv.h : commit 4539673a6ac027e3ba95901379b5f36c5eb3c9dc : Drivers: hv: Fix a typo
hyperv.h : commit 6c248aad81c8903981db9f12fd74189fa01b3789 : Drivers: hv: Base autoeoi enablement based on hypervisor hints
hyperv.h : commit 4827ee1dca5691c9fc568883170a568db94f9b38 : vmbus: expose debug info for drivers
hyperv.h : commit 2a9d7de2038e87bb2a1085ac73c4246c260263f0 : vmbus: cleanup header file style
channel.c : commit bdc1dd47dbcd39f266f0a6c8727a2b1995b36ef2 : vmbus: fix spelling errors
channel_mgmt.c : commit e6242fa0fb50570acb6364446dec369d7b9b5b2c : vmbus: make channel_message table constant
connection.c : commit ada6eb11137be0baf846e00fca40fc096ba85a28 : vmbus: only reschedule tasklet if time limit exceeded
hv_balloon.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hv.c : commit 6c248aad81c8903981db9f12fd74189fa01b3789 : Drivers: hv: Base autoeoi enablement based on hypervisor hints
hv_fcopy.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hv_kvp.c : commit bdc1dd47dbcd39f266f0a6c8727a2b1995b36ef2 : vmbus: fix spelling errors
hv_kvp.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hv_snapshot.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hyperv_vmbus.h : commit 2a9d7de2038e87bb2a1085ac73c4246c260263f0 : vmbus: cleanup header file style
hyperv_vmbus.h : commit 4827ee1dca5691c9fc568883170a568db94f9b38 : vmbus: expose debug info for drivers
hyperv_vmbus.h : commit b103a56f80878b4370522ec26017fcadd526f5ad : vmbus: drop no longer used kick_q argument
hyperv_vmbus.h : commit e6242fa0fb50570acb6364446dec369d7b9b5b2c : vmbus: make channel_message table constant
ring_buffer.c : commit f3dd3f4797652c311df9c074436d420f1ad3566e : vmbus: introduce in-place packet iterator
ring_buffer.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
ring_buffer.c : commit 4827ee1dca5691c9fc568883170a568db94f9b38 : vmbus: expose debug info for drivers
ring_buffer.c : commit 2c616a8b6bd3b51b560d994f3412da998fe7ee48 : vmbus: remove unnecessary initialization
vmbus_drv.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
vmbus_drv.c : commit e6242fa0fb50570acb6364446dec369d7b9b5b2c : vmbus: make channel_message table constant
hyperv_net.h : commit 15a863bf7436124e799ba175a801e25f7b57191e : netvsc: implement NAPI
hyperv_net.h : commit 8db91f6a9b2ff2bb5355ad11c668fe63eb8ae0c3 : hv_netvsc: Fix the queue index computation in forwarding case
hyperv_net.h : commit 545a8e79bd1cc8774877a26275171a2ec8881c9e : netvsc: use RCU to protect inner device structure
hyperv_net.h : commit a0be450e19d397e9ff215e32ed31bc51339b460a : netvsc: uses RCU instead of removal flag
hyperv_net.h : commit 43c7bd1ffcd1621c64cedf1be52156e2f95bba9b : netvsc: use refcount_t for keeping track of sub channels
hyperv_net.h : commit 76f5ed881c8de0264283c5b934deb33bf87209a8 : netvsc: remove unused #define
netvsc.c : commit 545a8e79bd1cc8774877a26275171a2ec8881c9e : netvsc: use RCU to protect inner device structure
netvsc.c : commit 73e64fa4f417b22d8d5521999a631ced8e2d442e : netvsc: Deal with rescinded channels correctly
netvsc.c : commit ebc1dcf6008e562a8f88a6b1f4a4705f4d4c4fdd : netvsc: eliminate unnecessary skb == NULL checks
netvsc.c : commit f4f1c23d6e41f5ee4973a6da65cba1e1c536ec29 : netvsc: fix NAPI performance regression
netvsc_drv.c : commit 545a8e79bd1cc8774877a26275171a2ec8881c9e : netvsc: use RCU to protect inner device structure
netvsc_drv.c : commit e91e7dd71dedbf4508513c8a66212248a853ecbe : netvsc: replace netdev_alloc_skb_ip_align with napi_alloc_skb
netvsc_drv.c : commit a0be450e19d397e9ff215e32ed31bc51339b460a : netvsc: uses RCU instead of removal flag
netvsc_drv.c : commit 5e8456fdd23c23bbfa06e0d647c90758102a4410 : net: hyperv: use new api ethtool_{get|set}_link_ksettings
netvsc_drv.c : commit 8db91f6a9b2ff2bb5355ad11c668fe63eb8ae0c3 : hv_netvsc: Fix the queue index computation in forwarding case
netvsc_drv.c : commit f3c9d40ee12926f330a1dfebce0bebadd1406ba6 : hv_netvsc: change netvsc device default duplex to FULL
netvsc_drv.c : commit 3071ada4916e26a8961c1b99f7766a73b9007bfc : netvsc: change max channel calculation
rndis_filter.c : commit ce12b81061a0a2647ca90c04d131d90edd5c5063 : netvsc: fix and cleanup rndis_filter_set_packet_filter
rndis_filter.c : commit 00ecfb3b34b69dd702dee1bd6de6fc100be384db : netvsc: remove unnecessary lock on shutdown
storvsc_drv.c : commit daf0cd445a218314f9461d67d4f2b9c24cdd534b : scsi: storvsc: Add support for FC rport.
storvsc_drv.c : commit e04085285b829dc922e78768a9abb390a8caed31 : scsi: storvsc: Prefer kcalloc over kzalloc with multiply
storvsc_drv.c : commit 8d4208c1a73ab30e05da142e0f05b2b02fc26fc9 : scsi: storvsc: remove return at end of void function
pci-hyperv.c : commit 24196f0c7d4bba093dfa8074507f31509970319f : PCI: hv: Convert hv_pci_dev.refs from atomic_t to refcount_t
pci-hyperv.c : commit 433fcf6b7b31f1f233dd50aeb9d066a0f6ed4b9d : PCI: hv: Specify CPU_AFFINITY_ALL for MSI affinity when >= 32 CPUs
pci-hyperv.c : commit d3a78d8bf759d8848339dcc367c4c1678b57a08b : PCI: hv: Properly handle PCI bus remove
pci-hyperv.c : commit 59c58ceeea9cdc6144d7b0303753e6bd26d87455 : PCI: hv: Allocate interrupt descriptors with GFP_ATOMIC
pci-hyperv.c : commit 414428c5da1c71986727c2fa5cdf1ed071e398d7 : PCI: hv: Lock PCI bus on device eject

Revision history for this message
Joshua R. Poulson (jrp) wrote :

Some of these commits have already been applied, but are included for completeness.

Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Joshua R. Poulson (jrp) wrote :

Since 4.10 did not apply cleanly to 4.4, I'm requesting these for 4.8, 4.10, and Azure kernel.

tags: added: kernel-da-key kernel-hyper-v
Marcelo Cerri (mhcerri)
affects: linux (Ubuntu) → linux-azure (Ubuntu)
Revision history for this message
Marcelo Cerri (mhcerri) wrote :

I submitted the first version of the backport for review:

https://lists.ubuntu.com/archives/kernel-team/2017-May/084242.html

I also compiled it and I made the deb files available for testing on the following link:

http://kernel.ubuntu.com/~mhcerri/azure/lp1690177-hyperv-backport/

I boot tested it on azure with a "Standard DS1 v2" instance (with 1 core and 3.5 GB of memory). Can you perform some additional tests with it to ensure it's working properly?

Changed in linux-azure (Ubuntu):
status: Confirmed → In Progress
Revision history for this message
Joshua R. Poulson (jrp) wrote :

First cut doesn't boot on local hyper-v still, but I'll try in Azure soon.

Revision history for this message
Joshua R. Poulson (jrp) wrote :

I'm definitely seeing issues with PCI interrupts with PCI passthrough, probably related to

pci-hyperv.c : commit 24196f0c7d4bba093dfa8074507f31509970319f : PCI: hv: Convert hv_pci_dev.refs from atomic_t to refcount_t
pci-hyperv.c : commit 59c58ceeea9cdc6144d7b0303753e6bd26d87455 : PCI: hv: Allocate interrupt descriptors with GFP_ATOMIC

I'm okay with leaving those two out but keeping the others. I suspect that since my first request was for a 4.11 base for everything we have a mismatch.

Symptom:
May 19 17:54:04 jrpcudau kernel: [ 87.156879] hv_pci f4d80956-1c02-4ac3-9063-2708944eab59: Request for interrupt failed: 0xc0350005

Revision history for this message
Marcelo Cerri (mhcerri) wrote :

I removed these two commits and prepared a new kernel for testing:

http://kernel.ubuntu.com/~mhcerri/azure/lp1690177-hyperv-backport_v2+local-hyperv/

Investigating the issue with local hyper-v, that seems to be caused by the missing EFI support. I re-enabled it in this kernel so you probably can test it locally.

I created the bug #1692920 to track the issue with local hyper-v.

Revision history for this message
Marcelo Cerri (mhcerri) wrote :

Second version of the pull request submitted:

https://lists.ubuntu.com/archives/kernel-team/2017-May/084389.html

Changed in linux-azure (Ubuntu):
assignee: nobody → Marcelo Cerri (mhcerri)
Marcelo Cerri (mhcerri)
Changed in linux-azure (Ubuntu):
status: In Progress → Fix Committed
Marcelo Cerri (mhcerri)
Changed in linux-azure (Ubuntu Xenial):
status: New → Fix Committed
Revision history for this message
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial
Marcelo Cerri (mhcerri)
Changed in linux-azure (Ubuntu Xenial):
assignee: nobody → Marcelo Cerri (mhcerri)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.