Please backport open-vm-tools 2:10.2.0-3 (main) from bionic

Bug #1741390 reported by evade
40
This bug affects 9 people
Affects Status Importance Assigned to Milestone
open-vm-tools (Ubuntu)
Fix Released
High
Unassigned
Xenial
Fix Released
High
Unassigned
Artful
Fix Released
High
Unassigned

Bug Description

[Impact]

 * Without SRUing the never version users get issues running on more
   recent hipervisors.

 * See comment #6 for where we decided to consider this SRU-worthy

 * This is not backporting a single fix, but the version of a latter
   Ubuntu release

[Test Case]

 * This is special in this case, and there are two things

 * a) If one doesn't have a vmware hipervisor he can take get a free 30
      day trial and test the upgrade in there

 * b) more important there is a while matrix of potential hipervisor
      versions to test and none of us has them available. Therefore
      vmware being the owner of the hipservisor as well as the open-vm-
      tools upstream project volunteered to verify the upload (already
      done on the PPA ,will be done on the SRU again)

[Regression Potential]

 * This is quite a bump of versions, so if the new version has issues
   the old one had not there might be regressions. There are no known
   issues like that atm and this also is why we pulled VMWare into
   helping us with the verification (as they are more an authority to
   call it good)

[Other Info]

 * This was meant to go to backports, but after several discussions a
   release via SRU is preferred as this will benefit all users way more.

 * Down the road we intend to re-do so for the LTS releases >=Xenial of
   Ubuntu backporting from the most recent Dev release, so ~6 month
   cadence.

---

This is no more intended to be a backport, but instead an SRU
Keep the content if ever needed again.

Description:
Please backport open-vm-tools 2:10.2.0-3 (main) from bionic to xenial.

Reason for the backport:
========================
Newer versions of VMWare ESXi, especially newer features of those require the availability of newer open-vm-tools in the guest.
Making that available as an SRU is at the moment considered to be too much of a risk due to the lack of a vast array of ESXi verification setups.
But it seems good and somewhat tested, while OTOH important for several users, which is why it seems the backports pocket is just the right pocket to go to.

Testing:
========
A ppa is made available for testing at:
https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3152
You can test it with:
sudo add-apt-repository ppa:ci-train-ppa-service/3152
apt install open-vm-tools

* xenial - build:
[No - see below] Package builds without modification

* xenial - install:
(tested on VMWare Workstation 14 - upgrading to ppa version)
[X] open-vm-tools installs cleanly and runs
[X] open-vm-tools-desktop installs cleanly and runs
[X] open-vm-tools-dev installs cleanly and runs
(ppa builds with debug, so I enabled debug from ppa for these tests)
[X] open-vm-tools-desktop-dbgsym installs cleanly and runs
[X] open-vm-tools-dbgsym installs cleanly and runs

* Modifications:
The backport needs some modifications to build:
- run autoreconf/systemd dh helpers
- use the older debhelper B-D
- revert to compat 10 (which is the latest supported in xenial)
Modifications are made available as git tree for easy review.
=> https://code.launchpad.net/~paelzer/ubuntu/+source/open-vm-tools/+git/open-vm-tools/+ref/backport-xenial-lp1741390

You might need to get the tarballs first to build so I recommend:
$ pull-lp-source open-vm-tools
$ git clone -b backport-xenial-lp1741390 https://git.launchpad.net/~paelzer/ubuntu/+source/open-vm-tools
Then you can review/modify/build locally
$ cd open-vm-tools
$ dpkg-buildpackage -S -nc -d -sa -v2:10.0.7-3227872-5ubuntu1~16.04.1

If you are a "git ubuntu" user you could also do
$ git ubuntu clone open-vm-tools
$ git ubuntu remote add paelzer
$ git checkout backport-xenial-lp1741390

Reverse dependencies:
=====================
The following reverse-dependencies need to be tested against the new version of open-vm-tools. For reverse-build-dependencies (-Indep), please test that the package still builds against the new open-vm-tools. For reverse-dependencies, please test that the version of the package currently in the release still works with the new open-vm-tools installed. Reverse- Recommends, Suggests, and Enhances don't need to be tested, and are listed for completeness-sake.

open-vm-tools
-------------
* ubuntu-server
  -- the base version is seeded, nothing to test-rebuild
  [X] xenial (Reverse-Depends)
* open-vm-tools-dkms
  -- the newer version no more provides and no more depends on that
  -- the breaks/replaces is very old and no issue
  [X] xenial (Reverse-Recommends)
  [X] xenial (Reverse-Replaces)
  [X] xenial (Reverse-Breaks)
* procps
  -- very old break, no issue
  [X] xenial (Reverse-Breaks)
* usrmerge
  -- very old conflict, no issue
  [X] xenial (Reverse-Conflicts)

open-vm-tools-desktop-dbgsym
----------------------------

open-vm-tools-dbgsym
--------------------

open-vm-tools-desktop
---------------------
* open-vm-tools-dkms
  -- the -dkms package is no more provided - no issue
  [X] xenial (Reverse-Suggests)

open-vm-tools-dev
-----------------

----- original bug -----

The latest Xenial open-vm-tools package is almost two years out of date.

I'm building Xenial systems in an up-to-date VMware environment and I need VMware tools updated to the latest stable version. We have experienced problems with VMware products when different components were mismatched.

We also require open-vm-tools-desktop updated as a bug in either it or this package are causing problems with a VMware's Horizon view virtual desktop environment.

Version 10.0.7 of open-vm-tools was first released in February 2016. The current stable is 10.2.0 released in December last year.

See: https://github.com/vmware/open-vm-tools
And: https://github.com/vmware/open-vm-tools/releases/tag/stable-10.2.0

Could these packages please be updated for Xenial? We're relying on them.

Thank you!

Related branches

Revision history for this message
evade (evade) wrote :

PS: There are multiple significant bugfixes for Linux in the updated version.

Here are the release notes, I believe the latest does not include the fixes from previous versions (all these releases are newer than the Xenial version)

https://docs.vmware.com/en/VMware-Tools/10.2/rn/vmware-tools-1020-release-notes.html
https://docs.vmware.com/en/VMware-Tools/10.1/rn/vmware-tools-1015-release-notes.html
https://docs.vmware.com/en/VMware-Tools/10.1/rn/vmware-tools-1010-release-notes.html

Revision history for this message
evade (evade) wrote :

I can't find any security bugs fixed, but there are good fixes like this:

"Kernel modules were not upgraded after upgrading OSPs using the recommended procedure
The package vmware-tools-esx-kmods is a meta package that depends on the kernel module package. Installing it with yum/apt/zypper gets the kmod packages. An upgrade of vmware-tools-esx-kmods might not automatically upgrade the kmod packages because the dependency was not versioned.
This issue is fixed in this release."

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi evade,
I understand your case but this has to be very carefully evaulated.
In the sense that you describe it about every package in Xenial is now "two years out of date", because the policy to not break on what users already use has a lot of implicatons [1]

One tries to address bug-fixes in an isolated testable way as good as possible, but such a major update is rare. Even minor release updates (less bumps than this case) have to follow a very strict process with [2] as examples.

The way out of this is [3] where people can prepare newer versions without affecting the world as it is mostly opt-in and thereby not affecting the majority of users who consider themselves safe by the SRU policy.

I currently have no cycles to spare on this, so if this is urgent for you I wanted to ask if you are willing and able to start driving this bug along the Ubuntu Backport Process [4]?

[1]: https://wiki.ubuntu.com/StableReleaseUpdates
[2]: https://wiki.ubuntu.com/StableReleaseUpdates#Documentation_for_Special_Cases
[3]: https://help.ubuntu.com/community/UbuntuBackports
[4]: https://wiki.ubuntu.com/UbuntuBackports

Revision history for this message
evade (evade) wrote :

Hello Christian,

Thank you for your fast response and the explanation.

Despite the difference in release time, I don't see this as a major upgrade but instead a required bugfix to maintain compatibility with another vendor's products.

I work in an enterprise environment and am working on a project building a new network.

We specifically chose an LTS release for our virtual developer desktops because we didn't want developers to have to upgrade their desktops on the normal release cycle.

Sadly we are stuck with VMware's Horizon View (which I wouldn't recommend). As Horizon View's Linux support is still maturing and we're still in the build phase of our Network, we've had to keep up with the latest version of that product. (For example, the latest version allowed us to offer KDE instead of only a GNOME desktop!)

We have to patch VMware software, especially including the ESXi hypervisors for security updates. However VMware don't offer security-only patches, only whole updates.

We're working with a whole matrix of software which we've learned the hard way has to be kept on equal supported versions. Also, as I mentioned previously, we've had experience with out of date VMware tools causing difficult to diagnose problems (like with the virtual NIC driver, for example).

I raised this ticket because a horrid combination of old VMware tools, NVIDIA proprietary graphics driver (Tesla acceleration is required for the number of monitors used) and VMware's Horizon view agent is causing desktop sessions to crash regularly. I've updated all the other software, only VMware tools is now old.

(I'm not convinced that older versions of VMware tools are more "stable" anyway, just that their problems are more likely known!)

I recognise the situation might be very different, but as an example, Red Hat provide the latest stable open-vm-tools and open-vm-tools-desktop packages for RHEL even though RHEL has a much older kernel and (many) older packages than 16.04. (We do of course pay them well for their support though)

I'll ask my project manager about investing time in the Backport Process. I'm afraid I have 0 personal time which I could invest in this.

Thanks for your time!

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thank you for your explanation - and even before so I was guessing just that.
I've been there as well in the past.

I understand you estimation of: I'm not convinced that older versions of VMware tools are more "stable"

I've been there as well (for other projects), but hard lessons learned (on my side) boils down on my favorite https://xkcd.com/1172/

There are certain exceptions, but these follow [2] in my former post and are generally projects adopted by Canonical in Main [1].

Long story short, backports as suggested are just the right way to do this.
I'll stay subscribed to support you when and where I can.

[1]: https://help.ubuntu.com/community/Repositories/Ubuntu#The_Four_Main_Repositories

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 1741390] Re: Package two years out of date

Hi Christian,

On Fri, Jan 05, 2018 at 07:25:16AM -0000, ChristianEhrhardt wrote:
> In the sense that you describe it about every package in Xenial is now
> "two years out of date", because the policy to not break on what users
> already use has a lot of implicatons [1]

> One tries to address bug-fixes in an isolated testable way as good as
> possible, but such a major update is rare. Even minor release updates
> (less bumps than this case) have to follow a very strict process with
> [2] as examples.

> The way out of this is [3] where people can prepare newer versions
> without affecting the world as it is mostly opt-in and thereby not
> affecting the majority of users who consider themselves safe by the SRU
> policy.
>
> I currently have no cycles to spare on this, so if this is urgent for
> you I wanted to ask if you are willing and able to start driving this
> bug along the Ubuntu Backport Process [4]?
>
> [1]: https://wiki.ubuntu.com/StableReleaseUpdates
> [2]: https://wiki.ubuntu.com/StableReleaseUpdates#Documentation_for_Special_Cases
> [3]: https://help.ubuntu.com/community/UbuntuBackports
> [4]: https://wiki.ubuntu.com/UbuntuBackports

With my SRU team hat on, I will say that open-vm-tools clearly falls into
the class of packages that have a "platform enablement" (née "hardware
enablement") exception to the usual bugfix-only rule.

Care must of course still be taken to test the updates and avoid
regressions, but in cases where the package must be updated from upstream to
maintain compatibility with the moving target of the OS's substrate (whether
that's hardware, or a cloud platform, or a VM platform), the requirement to
selectively cherry-pick bugfixes is waived.

Revision history for this message
evade (evade) wrote : Re: Package two years out of date

Thank you both for your responses!

What needs to happen for a "platform enablement" bugfix?

Revision history for this message
evade (evade) wrote :

Also, I hate to burden you more, but VMware tools seems to be updated every 3/4 months in line with ESXi patches. IMO it should be updated at least quarterly.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [Bug 1741390] Re: Package two years out of date

Thank Steve, you are right under that POV it would fall under the
(virtual) HW enablement category.
So not backports but a real SRU, that is beneficial for more people if
done right.
Thanks for sharing your SRU expert hat thoughts.

So it comes down to a matter of:
- developing some more extended regression checks to have more
confidence in such updates
- allocating time to do the regular updates (maybe quarterly, but even
less frequent is better than no updates)

Looking at my next things I need to do - I don't see me doing these
open-vm-tools updates very soon (mostly for the lack of good
regression checks atm) so any help is welcome, but I keep it on my
todo-list to pick it up when I can.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: Package two years out of date

The maintainer scripts have not a lot (actually none) version dependent special cases that sometimes wreaking havok for such backports.

OTOH unfortunately no integrated dep8 tests.
And as I mentioned at least I don't have a matrix of vmware hosts 10-14 around or any such which I'd like to see at least somewhat tested - and the SRU Team as well I'd guess.
I tried manually the current version, installing VMware-Workstation-Full-14.1.0-7370693.x86_64.bundle
Note: to get rid of it afterwards: sudo vmware-installer -u vmware-player
And Installed Xenial iso http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.iso
But I couldn't see total breakage without the new version just by installing and basic usage.

I have set up a ppa to test new versions that would be made available as SRU to be tested, see [1] for the packages.

@Evade:
- please help testing those packages as much as you can
- I know you referenced 10.2, but since 10.1.15 is the latest we have I won't go ahead of that.
- if you would have a few examples what fails on the old but would work on the new version with
  steps to reproduce (with only free vmware versions) - that would be helping the case a lot.

P.S: currently the Ubuntu build farm is disabled for maintenance, so you might need to wait a few days until the ppa is ready - that can be seen on the web view of the ppa on [2] where the packages need little "green checkmarks".

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3099
[2]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3099/+packages

Revision history for this message
evade (evade) wrote :

Thank you very much!
I will try the package from your ppa as soon as I'm able.

Please note that the related open-vm-tools-desktop package also needs to be kept at the same version as open-vm-tools. As I mentioned we're using VMware's "Horizon View" product so I require this also.
Do I need to raise a bug against that package also?

I linked open-vm-tools version 10.2.0 as it is the stable latest available. My thinking is that if an update is necessary (and with the testing that involves) why not go to the latest now?!

Could you please clarify this:
"
The maintainer scripts have not a lot (actually none) version dependent special cases that sometimes wreaking havok for such backports.
"
as I don't understand. The statement seems to contradict itself.

Thank you again!

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [Bug 1741390] Re: Package two years out of date

[...]
> Please note that the related open-vm-tools-desktop package also needs to be kept at the same version as open-vm-tools. As I mentioned we're using VMware's "Horizon View" product so I require this also.
> Do I need to raise a bug against that package also?

The source package open-vm-tools which I thrown in a test backport
here builds all the binary packages you mentioned, so
open-vm-tools-desktop is already in the ppa.

>
> I linked open-vm-tools version 10.2.0 as it is the stable latest available. My thinking is that if an update is necessary (and with the testing that involves) why not go to the latest now?!

Sure eventually we might go for that.
But usually for any such work you'd not go ahead of the latest
available version - otherwise people upgrading from former Ubuntu
releases would downgrade the package.
For the current level of "giving it a try and evaluate" 10.1.15 is just as good.

>
> Could you please clarify this:
> "
> The maintainer scripts have not a lot (actually none) version dependent special cases that sometimes wreaking havok for such backports.
> "

That mostly is a note to myself that there is not a lot of special
magic making these backports harder, sorry for being misleading.
TL;DR - Note: it has none of the bad stuff

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: Package two years out of date

Artful built, but on Xenial there were various issues for compat 10, I need to revert more changes.
A new build was uploaded but will again take a while.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Xenial ok as well in the PPA now.

@evade - having this in the archive is kind of a comfort feature, but for your case where e.g. the absolute sync of tools and host is so important - can't you rely on the implicit availability of guest additions via the host like in [1] ? If not what are the issues - script-ability, ease of deployment, ... ?

[1]: https://kb.vmware.com/s/article/1018414

Changed in open-vm-tools (Ubuntu):
status: New → Incomplete
Revision history for this message
evade (evade) wrote :

Hello Christian,
Thank you for your work on this! Sorry to hear about build problem. I hope it's not difficult to resolve. I am looking forward to trying this.

I do not feel this is a "comfort feature" but instead a compatibility requirement for Xenial to be supported on the latest ESXi 6.5 update(s). If there are any problems on our Ubuntu VMs and I raise a support ticket with VMware, they will insist we update VMware tools before working further. I also have a specific problem I mentioned in the original post, but I won't know the impact on that until I can try an updated version.

The VMware article you linked is out of date. Note that even though the whole page was updated on May 30th 2017, if you read the "Update History" section, the actual content changed last in December 2015.

The advice I've read from VMware [1] is to always use a vendor's open-vm-tools package.
That advice is linked from their first announcement of planning to release Open-VM-tools [2]. Apparently the same advice is also displayed if you try to install VMware Tools directly from ESXi [3]!

I briefly tried to compile the open-vm-tools package from VMWare's github page, but it failed. I'm a sysadmin responsible for multiple systems aside from these Ubuntu desktops. As soon as I recognized that I needed an updated open-vm-tools package I knew either I'd be maintaining the package for our Ubuntu systems forever, or I could ask the distribution to update this vendor-required component. I cannot be the only person affected by this [4]!

Thank you again.

[1]: https://kb.vmware.com/s/article/2073803 (VMware support policy section)
[2]: https://blogs.vmware.com/vsphere/2015/09/open-vm-tools-ovt-the-future-of-vmware-tools-for-linux.html
[3]: https://www.thomas-krenn.com/en/wiki/VMware_Tools_or_open-vm-tools (VMware Tools section)
[3]: https://communities.vmware.com/thread/547675

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [Bug 1741390] Re: Package two years out of date

On Thu, Jan 11, 2018 at 10:35 PM, evade <email address hidden> wrote:
> Hello Christian,
> Thank you for your work on this! Sorry to hear about build problem. I hope it's not difficult to resolve. I am looking forward to trying this.

I can't spend like a full day on it, but if you are ok with me trying
to improve it step by step until it works then it is ok :-)

[...]

> The advice I've read from VMware [1] is to always use a vendor's open-vm-tools package.

Thanks for all of that.
I had assumed they'd prefer "their own", but I'm ok either way.
I kicked of an discussion about - if that is the stance taken - vmware
maybe assisting to qualify the backports to give the confidence needed
to SRU them.

Revision history for this message
evade (evade) wrote : Re: Package two years out of date

Thank you again.

If you have something specific you'd like VMware to act on, I can raise a support ticket with them but it will take a few days before there's any chance of action. I'm not sure I can get much traction, but it's worth a try.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [Bug 1741390] Re: Package two years out of date

> If you have something specific you'd like VMware to act on, I can raise
> a support ticket with them but it will take a few days before there's
> any chance of action. I'm not sure I can get much traction, but it's
> worth a try.

thanks for the offer. I have contacts, so this will go on - yet these
mills grind slowly so it will take a while.
Until then I need to get the ppa building for you so that you can at
least test for now.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: Package two years out of date

The build is kind of messy, as upstream moved some sources around and building that "nicely" required features in debhelper only avaialble in later Ubuntu versions. It failed a few times now and this week is really bad in terms of free time for this :-/

Also we discussed a potential SRU into Xenial a bit more and it came out as rather unlikely - as we really lack a way to check for regressions on a vast array of VMWare ESXi versions. If I get a build running on my weekend I might prep something for the backports pocket (which is opt in and therefore not under the same constraints as the SRU), but I doubt I could do that on a regular schedule to keep it always up to date.

For the "I need to be totally in sync with the host" it might come down as a case for the host provided tools still - despite the kb article declaring the OS vendor content as preferred. I wonder about the mentioned "VMware provides assistance to operating system vendors and communities with the integration" as I would not know at all where/how.

The package is on community level support, but if VMWare itself would step up and maintain it (as they are who can check and populate [1] against the new version) that would be great. Yet I'd not expect you to get a great answer if you open up a support ticket for that, but who knows - you might try.

Revision history for this message
evade (evade) wrote :

Sorry to hear that the build is messy.

Thanks for putting the time in that you have. I've got plenty of other things to do so I'm just checking every few days for the build to succeed.

Would this be much easier if the new packages were in backports only? In hindsight that is OK with me.
If so does it mean the priority of building new releases of these packages would also be lower? This wouldn't be ideal, since we'll need the packages in a timely fashion. (I'm also thinking ahead of when we move to the 2018 LTS release)

Could you please explain what you would like me to ask VMware to do?

As much as we'd both like it, I highly doubt they would be willing to directly maintain a package of the software they provide for any linux distro. They do say that they will provide "assistance" but whatever I ask will need to be more specific.

Thanks again!

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI - I completed 10.2 into Bionic now, it was in proposed and will soon be available there.

But the backport to Xenial drove me nuts (for stupid reasons), and since I can't even qualify it for the SRU I gave up the backport for not.
Yes for a xenial-backports only upload it would not be "that much" qualification needed. But still I'd not like to ship it untested.

Several people are going on with an discussion if/who/how to provide newer versions of this into older releases. But things are going (very) slow and it seems other than me (and you) nobody really wants/likes it and considers things good as they are :-/

For the question you could ask to vmware (so that it comes from different angles) I think it would be:
"if one would provide an Ubuntu ppa with such a backport, would vmware be willing to qualify this against their current set of supported vmware versions and post that on a bug like this?"
Because without that offer all my work for a backport would be kind of in vain as no one would release it.

Revision history for this message
evade (evade) wrote :

Thanks again. It will be a few days until I'm able to do it, but I will raise a job and request assistance from VMware.

Revision history for this message
evade (evade) wrote :

It's take a few days to get through some internal delays about raising the ticket. It should be acted on soon, although as I said before I'm unsure if it will get the right result.

Have you tried to contact the VMware people who maintain the open-vm-tools repo in Github by any chance? I wonder if that might be a better way to get to the right people.
At least you could ask them about the build problem?

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I asked around way more, but the TL;DR summary is that it never worked and that no group is able or willing to do the verification for a full SRU.

Backports would therefore be the option to go, but due to the (probably trivial once you know the fix) complex build issue it not feasible for me to do at the moment.
I'm open to help by anyone and willing to assist, but right now I just can't spend the time - therefore community help for the backport is welcome.

Revision history for this message
evade (evade) wrote :

Thank you again for your help.

I'm very disappointed by the overall response to this issue. If you've already contacted the VMware staffers working on the GitHub repo, then I wouldn't expect much from the business critical support ticket I've raised.

We need to know that the distros we use will maintain packages necessary for integration. I imagine there are few in the community using VMware (I use KVM when I have a choice) who are so dependent on running the latest ESXi release.

If you could please get this to compile that would be very helpful. At this stage I'd consider using a test version in production!

Revision history for this message
evade (evade) wrote :

I've had an unofficial response that VMware will continue to include Xenial in the supported OSes list for ESXi if VMware tools is updated. However I'm waiting for the support person to get an official statement, which I hope I can quote for you.

They're still talking internally

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Dropping it on/off any list won't help.
As I outlined before - I'm even for updating, but need some confirmation that it works right.
If they'd commit to verify a ppa that I provide against all their ESXi versions that would be great!
If you could ask them for that explicitly that would be great.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I have a build now working for xenial now (local sbuild).
Unless we will get a confirmation for extra tests by vmware or someone else able to cover the current supported esxi we will (as discussed) "only" target backports but not an SRU (this can change if one confirms he is willing and able to do the verifications).

I have a ppa [1] that is open to be tested already and will kick off the processing to make this a xenial-backport upload now (I never done such a backport so I hope I get the process right from [2]).

@Evade - if you could try out the build from that ppa that would be great

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3152
[2]: https://wiki.ubuntu.com/UbuntuBackports#Requesting_a_Backport

summary: - Package two years out of date
+ Please backport open-vm-tools 2:10.2.0-3 (main) from bionic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I added Xenial-Backports for an ack of the backports Team.
I updated the bug description with a full backport-request template for this case (as you'd do on an SRU with the SRU template).

This is my first backport to the actual -backports pocket, please let me know who of us eventually uploads what exactly to where.
I built the source as I assumed it will be needed, but need your experience if you need modifications.

@Backport Team - please review and ack or let me know what needs to be changed.

description: updated
Changed in open-vm-tools (Ubuntu):
status: Incomplete → Triaged
importance: Undecided → High
Revision history for this message
evade (evade) wrote :

Hi,

VMware Tools Product Management Team (indirectly) responded to me. They are also disappointed that Canonical is not rebasing open-vm-tools for LTS releases. (I too would suggest it is in Canonical's interest to keep open-vm-tools updated for LTS releases)

I don't know what discussions you folks are having internally, but what happened to the response Steve Langasek made above (#6)?

Also, I wonder if your expectation of them testing Ubuntu is a bit unrealistic.

Perhaps a compromise might be reached between the two organisations where they assist you to set up automated testing?
Perhaps they might at least provide you some cloud instances on their Hypervisors so you don't have to bear the hardware/compute cost?

Thanks again!

Revision history for this message
evade (evade) wrote :

PS: IMO this ticket should address the issue of rebasing OVT across LTS releases, not just backporting one package from bionic to xenial.

We need to rely on the latest stable OVT being available in all LTS releases when we upgrade to 18.04 later this year.

Revision history for this message
evade (evade) wrote :

Update: There are security vulnerabilities in the current version OVT 10.0.7. Can you please update this ticket appropriately?

I don't yet know where the CVE's are publicly published but VMware support tell me:

"
Later releases of VMware Tools specifically 10.0.9 for CVE listed in VMSA - VMSA-2017-0013 and 10.2.0 has fixes for CVE Listed in VMSA - VMSA-2018-0003
"

Revision history for this message
Emily Ratliff (emilyr) wrote :

VMSA-2017-0013 describes the following CVEs: CVE-2017-4921, CVE-2017-4922, CVE-2017-4923, CVE-2015-5191. Of these, only CVE-2015-5191 is applicable to open-vm-tools and it is partially mitigated via symlink restrictions. It is on the list to be fixed, but is currently rated low.

VMSA-2018-0003 describes CVE-2017-4945, CVE-2017-4946, and CVE-2017-4948. CVE-2017-4945 is applicable to VM tools, but only for Windows guests, so it is not applicable to the open-vm-tools package. CVE-2017-4946 and CVE-2017-4948 are not applicable to open-vm-tools.

You can see the CVE status for the package at
http://people.canonical.com/~ubuntu-security/cve/pkg/open-vm-tools.html

Revision history for this message
evade (evade) wrote :

Hi Emily,

I'm sorry, I posted my previous message in a hurry without checking out what the vulnerabilities involved.

Thanks for your response and the CVE link for open-vm-tools. That's helpful!

Can you please tell me the URL for the companion open-vm-tools-desktop package? It wasn't obvious.

Although this bug has been turned in to one about a specific package in Xenial, I see this as a bigger issue for all LTS releases. If an LTS release won't be patched to resolve a low priority vulnerability, what level of vulnerability will trigger a patch?

If such a patch is required, will the maintainer(s) attempt to write a mitigation or back-port a fix, or will they upgrade these packages in the process anyway?

Unlike many packages used in an LTS, Open VM Tools does not have a long-term stable release, it's always moving forward.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I now have several leads which raise my mood on this.
I already had one party testing the locally which made us spot an issue with the latest version in regard to xenials systemd (bug 1750780). But that is fixed already in the proposed ppa.

We now also got vmware itself taking a look at the proposed changes in the ppa which should update here in a bit.

Since this is the first time this is formally introduced it might take a while, but I think if all that works together we can work out a formal SRU exception that will allow to do that regularly.

Next Steps:
1. smoke test ack on the ppa 3152 by VMware (result: comment on this bug)
--- the following are follow on actions in my mind, not yet discussed/agreed with the others ---
2. create SRU exception (I'll do that) agreed by VMware on how/who verifies them (result: SRU exception wiki page)
3. ack by the SRU Team on the exception (result: ack by SRU team)
4. upload verified open-vm-tools versions under above SRU exception regularly (result: up to date open-vm-tools in Ubuntu)

Revision history for this message
David Britton (dpb) wrote :

@John, @Oliver, @vmware-gos-qa

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3152

Please give this PPA a test and let us know if its a good candidate for an SRU to xenial.

Revision history for this message
vmware-gos-Yuhua (yhzou) wrote :

> [1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3152

> Please give this PPA a test and let us know if its a good candidate for an SRU to xenial.

Env:
VMs:ubuntu-16.04-desktop-amd64 and ubuntu-16.04.1-server-amd64
open-vm-tools version: 10.2.0.1608(build-7253323) from ppa:ci-train-ppa-service/3152

Result:
Two errors are found when install open-vm-tools related packages from untrusted PPA by adding ppa:ci-train-ppa-service/3152

1) VGAuathServices is not running because /lib/systemd/system/vgauth.service is missing
   when update to / install open-vm-tools 10.2.0

   (Note: refer https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1677196 for the fix)

2) get error message as follows when type "apt-get install open-vm-tools-desktop"
   /usr/bin/deb-systemd-helper: error:unable to read run-vmblock-fuse.mount
   Failed to get unit file state for run-vmblock-fuse.mount:No such file or directory.

   (Note: This is a known bug at Canonical: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1639222.
   In fact this issue has been fixed in ubuntu 17.10 with bundled open-vm-tools-desktop version 2:10.1.10-3ubuntu0.1 and ubuntu 18.04 with bundled open-vm-tools-desktop 10.2.0-3ubuntu2)

Steps:
1) add repository
sudo add-apt-repository ppa:ci-train-ppa-service/3152
sudo apt-get update

2) update to / install open-vm-tools 10.2.0, and then check tools, vgauth, vmware-toolbox-cmd

3) reboot guestOS and check tools, vgauth, vmware-toolbox-cmd

4) uninstall open-vm-tools and then check tools, vgauth, vmware-toolbox-cmd

5) reboot guestOS and check tools, vgauth, vmware-toolbox-cmd

6) install/uninstall open-vm-tools-desktop

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

For #1 VGAuathServices this is already built against libxmlsec1-dev (was the fix for bug 1677196 ).
It must be an artifact of the backport, maybe the older version lacks something.

I found in the build log:
checking xmlsec/xmlsec.h usability... no
checking xmlsec/xmlsec.h presence... no
checking for xmlsec/xmlsec.h... no
checking for xmlsec1-config... /usr/bin/xmlsec1-config

OTOH a newer build (e.g. the same on Bionic) has this as well:
checking xmlsec/xmlsec.h usability... no
checking xmlsec/xmlsec.h presence... no
checking for xmlsec/xmlsec.h... no
checking for xmlsec1-config... /usr/bin/xmlsec1-config

Ignoring the above I clearly confirm that the service is missing.
./lib/systemd/system/vgauth.service is in
[1] bionic, but not [2] backport build.

[1]: https://launchpadlibrarian.net/358778573/buildlog_ubuntu-bionic-amd64.open-vm-tools_2%3A10.2.0-3ubuntu2_BUILDING.txt.gz
[2]: https://launchpadlibrarian.net/358001805/buildlog_ubuntu-xenial-amd64.open-vm-tools_2%3A10.2.0-3ubuntu0.16.04.1~ppa2_BUILDING.txt.gz

I found dh_installsystemd used in the new code which didn't exist back then.
Therefore the override in d/rules effectively became a no-op and by that forgetting most of the services vgauth, fus.mount and also some renames due to the special char in those services.

This needs some more cleanups, I'm currently iterating on the ppa and will post here once it is ready for retest.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Latest build in the ppa looks fine from a build POV.
I'm going to test this tomorrow morning for the open issues, but please everybody that can easily test things (e.g. automation available) feel free to retest already.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Checks for reported deficiencies:
1. vmware-toolbox-cmd works (e.g. timesync status)
2. run-vmblock\x2dfuse.mount exists and starts without errors
3. vgauth.service exists and starts without errors

That said at everybody, please re-evaluate the ppa [1].

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3152

Revision history for this message
vmware-gos-Yuhua (yhzou) wrote :

> Checks for reported deficiencies:
> 1. vmware-toolbox-cmd works (e.g. timesync status)
> 2. run-vmblock\x2dfuse.mount exists and starts without errors
> 3. vgauth.service exists and starts without errors

> That said at everybody, please re-evaluate the ppa [1].

>[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3152

still get error message as follows when type "apt-get install open-vm-tools-desktop" in ubuntu-16.04-desktop-64bit and ubuntu-16.04.1-Server-64bit.
   Failed to get unit file state for run-vmblockx2dfuse.mount: No such file or directory
   run-vmblock\x2dfuse.mount is a disabled or a static unit, not starting it.

The following is detail message when install open-vm-tools-desktop in ubuntu-16.04-desktop-64bit

virtual-machine:~$ sudo apt-get install open-vm-tools-desktop
[sudo] password for vmware:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  open-vm-tools-desktop
0 upgraded, 1 newly installed, 0 to remove and 607 not upgraded.
Need to get 128 kB of archives.
After this operation, 464 kB of additional disk space will be used.
Get:1 http://ppa.launchpad.net/ci-train-ppa-service/3152/ubuntu xenial/main amd64 open-vm-tools-desktop amd64 2:10.2.0-3ubuntu0.16.04.1~ppa5 [128 kB]
Fetched 128 kB in 4s (26.3 kB/s)
Selecting previously unselected package open-vm-tools-desktop.
(Reading database ... 172417 files and directories currently installed.)
Preparing to unpack .../open-vm-tools-desktop_2%3a10.2.0-3ubuntu0.16.04.1~ppa5_amd64.deb ...
Unpacking open-vm-tools-desktop (2:10.2.0-3ubuntu0.16.04.1~ppa5) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up open-vm-tools-desktop (2:10.2.0-3ubuntu0.16.04.1~ppa5) ...
Failed to get unit file state for run-vmblockx2dfuse.mount: No such file or directory
run-vmblock\x2dfuse.mount is a disabled or a static unit, not starting it.
vmware@vmware-virtual-machine:~$

best regards
Yuhua Zou

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hmm, I had the service running in my test, but I started it on my own.
Maybe the remaining issue now is that it is not started by default - we know there were some issues due to the special name of this service making dh tools stumble over it (and those changed in different versions).
I'll take a look at the auto-start and install-messages of it in particular and thanks for taking a look again.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Ok, I can reproduce the issue with a full purge and reinstall.
The enable part is good on install - the service is enabled and not just dropped on disk as a file.
But the follow on start is what fails - which also explains why it worked after a reboot without further modification.
That is enough for me to reiterate until I have a fix.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

File is \x2d
Name has to be escaped as \\x2d for e.g. systemctl

This is exactly as we have it in later releases with the fix - so the file names are "ok" despite still feeling odd to have it named that way.

It turns out that the old 'deb-systemd-helper' is fine with the name - that is the reason it is installed and enabled corretcly.
The failing part is 'deb-systemd.invoke', that would need other escapes.
This call is no more existing in newer Ubuntu releases as dh_installsystemd has taken over there but does not exist on Xenial yet.

The newer version of deb-systemd-invoke is no more used in Bionic, but it would be fixed - not worth for this case but nice to know.

The older Xenial version failes the same way at the moment btw.
So we are fixing a real bug in Xenial, not just removing backport artifacts.
Essentially it is not started on install and stopped on upgrade until reboot.

The escaping in /usr/bin/deb-systemd-invoke is inconsistent.
For the first check we would need to pass: run-vmblock\\\\x2dfuse.mount
But for the second we would need to pass: run-vmblock\\x2dfuse.mount

The "real" fix would be in /usr/bin/deb-systemd-invoke:
my $enabled_output = `/bin/systemctl is-enabled -- '$unit'`;
to
my $enabled_output = `/bin/systemctl is-enabled -- $unit`;
But that has too many regression risks on an SRU.

But for open-vm-tools alone we can fix this up by overriding dh_systemd_start.
Generated:
if [ -d /run/systemd/system ]; then
        systemctl --system daemon-reload >/dev/null || true
        deb-systemd-invoke start run-vmblock\\x2dfuse.mount >/dev/null || true
fi
Has to be replaced with a custom snipped to handle the esacping.
I checked the other calls to deb-systemd-invoke, e.g. the stop - they work fine with the names.

I have fixed the issues and ran test builds.
The now:
- do not report the warnigns on install/upgrade they did before
- the service is active right after install of open-vm-tools-desktop

The new version in the ppa 10.2.0-3ubuntu0.16.04.1~ppa6 is good for a retest.

Revision history for this message
vmware-gos-Yuhua (yhzou) wrote :

  > The new version in the ppa 10.2.0-3ubuntu0.16.04.1~ppa6 is good for a retest.

verified. It works well with the new package version.

Env:
VMs:ubuntu-16.04-desktop-amd64 and ubuntu-16.04.1-server-amd64
open-vm-tools/open-vm-tools-desktop version: 10.2.0-3ubuntu0.16.04.1~ppa6 from ppa:ci-train-ppa-service/3152

Changed in open-vm-tools (Ubuntu Xenial):
status: New → In Progress
Changed in open-vm-tools (Ubuntu):
status: Triaged → Fix Released
Changed in open-vm-tools (Ubuntu Xenial):
importance: Undecided → High
description: updated
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I prepared the SRU Template for this, there are two things to complete before pushing:

1. VMWare will run more regression testing using the ovt automation package over the next days
2. I added an MP with the packaging changes to be reviewed [1]

Once both complete we can push it for the actual SRU.

[1]: https://code.launchpad.net/~paelzer/ubuntu/+source/open-vm-tools/+git/open-vm-tools/+merge/341797

Revision history for this message
Charles Crouch (ccrouch) wrote :

FWIW we've been trying out open-vm-tools version 2:10.2.0-3ubuntu0.16.04.1~ppa6 and its been working great for us. Thanks very much!

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I did some quick testing with vsphere 6 and esx for the merge proposal, copied below:
- upgrade from previous version, including dev and desktop subpackages
- used an ESXi xenial vm for testing
- some actions in the vsphere GUI failed with the previous version, worked out-of-the box with the new one ("view guest user mappings"). Well, that's the only one I managed to try actually, but it's progress.
- removal and purging of the packages
- vmblock fuse mounted after install

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The upload is in the SRU queue [3] for xenial to be reviewed one last time (by the SRU Team now).
We have plenty of tests on the PPA already which is the same as what we propose, once it is in proposed [1][2] (there will be an update here) I'd ask you to test that instead.

[1]: https://wiki.ubuntu.com/ProposedMigration
[2]: https://wiki.ubuntu.com/Testing/EnableProposed
[3]: https://launchpad.net/ubuntu/xenial/+queue?queue_state=1

Revision history for this message
evade (evade) wrote :

Thank you all for your work on this!

Christian, my original bug (which this came out of) was about needing updates for both open-vmtools AND the open-vm-tools-desktop package. I understand that the open-vm-tools-desktop packages needs to be updated in step with OVT. Is it being addressed please?

Thank you

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi evade,
yes it is taken care of.

In Packaging there are source and binary packages.
When we say "we update X" we mean we upload a new src:X
But from your consumption point of view you use binary packages (that is what you install).
And src:open-vm-tools creates open-vm-tools-dev, open-vm-tools-desktop, open-vm-tools
So you'll be good once this completes.

Afterwards I hope to be able to do kind of a retrospective with all involved people and make this a regular action to backport the new version that goes out with an Ubuntu Release back to the last LTS at least - but lets first complete this work here :-)

Revision history for this message
Charles Crouch (ccrouch) wrote :

I spoke to soon :-( https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1741390/comments/47

We're having to back this version out from our deploy. More details coming soon...

Revision history for this message
Charles Crouch (ccrouch) wrote :
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

@paelzer Seeing Charle's comment, could you re-assess if the SRU is still valid for upload? Should I reject it from the queue?

Revision history for this message
Oliver Kurth (okurth-1) wrote :

We do see weird issues with that build similar to bug #1758428, so please hold on with the upload.

Revision history for this message
Oliver Kurth (okurth-1) wrote :

I found a fix for bug #1758428 and updated the issue. It's related to the use of PrivateTmp=yes in the service file.

Since this is a backport from 18.04, I expect the same issue there, and in Debian.

Revision history for this message
vmware-gos-Yuhua (yhzou) wrote :

> ChristianEhrhardt (paelzer) wrote on 2018-03-21: #46
> I prepared the SRU Template for this, there are two things to complete before pushing:

> 1. VMWare will run more regression testing using the ovt automation package over the next
         days

> 2. I added an MP with the packaging changes to be reviewed [1]

> Once both complete we can push it for the actual SRU.

> [1]: https://code.launchpad.net/~paelzer/ubuntu/+source/open-vm-tools/+git/open-vm->
           tools/+merge/341797

Done for regression test with 10.2.0-3ubuntu0.16.04.1~ppa6 in Ubuntu 16.04-Server64.

Result:
Except the issue about "PrivateTmp=yes" in open-vm-tools.service file mentioned , No others issue exists.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@SRU Team

The xenial-unapproved queue has now a fixed version in regard to our findings in the prechecks:
=> 2:10.2.0-3ubuntu0.16.04.2

No need to cancel the old one, just consider accepting the new upload.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Due to SRU regulations - thanks rbasak for pointing it out, we have to prep the upload for Artful as well.
Given how much backport-special cases we hit I'm already happy :-/

Also the state of 1758428 in Bionic has to be resolved.

So for now tasks:
1. for me - prepare an Artful SRU upload as well
2. for those involved in 1758428 please let me know if Bionic is affected as well (there)

Revision history for this message
Robie Basak (racb) wrote :

I've reviewed the SRU upload to Xenial in the Unapproved queue.

SRU policy requires fixes to be in Bionic first. I understand the spirit of this to be not just to both to stop users regressing upon future upgrade, but also for us to gain some confidence in quality in the development release to mitigate SRU regression risk before backporting. Bug 1758428 concerns me. We don't know if it's fixed in Bionic, which suggests to me that we don't have a well-defined test case. I understand that in this case the nature of the fix is that it can easily be applied to Bionic, and the reason for not doing so is because other factors have changed such that it may be unnecessary. Nevertheless, I think this should be settled in Bionic first both to avoid user regressions and to ensure quality in Bionic prior to complicating things by SRUing. And besides, Bionic will be the next LTS so we should presumably have this settled well in Bionic anyway.

SRU policy also requires that for feature backports to LTSs, intermediary releases are also updated: "To avoid regressions on upgrade, any such feature must then also be added to any newer supported Ubuntu release". Note that one severe class of regression is that failure to do this will make any future security updates for this package in Artful invisible to users who have upgraded, so I believe this to be an important requirement. I'm also not sure that the SRU team has the remit to make an exception here. Finally, my opinion is that if an exception is warranted in this case, the implications are large enough that we should be making a general policy statement for the project so users can be clear on the consequences (eg. "cloud instance release upgrades to non-LTS releases are no longer supported").

I haven't reviewed the diff itself, since I'd want to do it again before accepting a future upload anyway. Based on the changelog entries, the backport itself looks reasonable in principle though.

A couple of minor points which don't necessarily block upload, but would be useful to fix in future uploads. These caused me some confusion to follow what was going on:

2:10.2.0-3ubuntu0.16.04.1 has never been published, so 2:10.2.0-3ubuntu0.16.04.2 is unnecessary and could be squashed into 2:10.2.0-3ubuntu0.16.04.1.

2:10.2.0-3ubuntu0.16.04.1 is based on 2:10.2.0-3 which has been superseded in Bionic. That it has been superseded since the SRU was prepared is fine. But at that time, an upgrade from Xenial to Bionic wouldn't work as expected because 2:10.2.0-3 is lower than 2:10.2.0-3ubuntu0.16.04.1. Since Bionic has moved on, this won't cause a problem now, assuming that Artful will be updated the same as Xenial. I suggest using a pattern like 2:10.2.0-3~ubuntu0.16.04.1 in the future.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

While waiting for a Bionic confirm/deny to bug 1758428:
- X version squashed to 2:10.2.0-3~ubuntu0.16.04.1 for the next upload (thanks for the hint).
- Start preparing Artful now ...

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI: PPA build with the fix for privateTmp is available in a PPA
Versioning changes slightly (better to do these SRUs regularly).
So the new version is below the old one, be sure to update correctly when testing this.
Consider using
  sudo ppa-purge ppa:ci-train-ppa-service/3152
Before using the new PPA:
=> https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3226

I have verified that Artful builds and correctly picks up all we did so far (a.k.a. watching out if some of our backport efforts need to be reverted there).

So the builds LGTM, so from now on for testing please use this PPA.
Once bug 1758428 is resolved in Bionic (or confirmed to not be needed) we can go on with the SRU again.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

bug 1758428 unlocked, it's fix is on the way into Bionic.
Once that completes I can propose the changes as prepared to Artful and Xenial.

Changed in open-vm-tools (Ubuntu Artful):
status: New → In Progress
importance: Undecided → High
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Ok, all dependent bugs were sorted out:
- 1758428 is in Bionic, ready to be included in the backport SRU
- 1750780 is only an issue pre-Bionic, ready to be included in the backport SRU
- 1741390 (this bug) was well tested along with the others

Version numbers for Xenial and Artful were adapted so that there will be no issues upgrading forward from them.
For correctness I also inserted the changelog history that happened in Bionic since we started this. This has no functional change, but is correct in terms of reading the changes.

Build with the right -v against what is in artful/xenial updates that gives quite a long changes, but it is what is correct.

That said, upload made available in -unapproved for SRU Team review:
X: open-vm-tools_10.2.0-3~ubuntu0.16.04.1_source.changes
A: open-vm-tools_10.2.0-3~ubuntu0.17.10.1_source.changes

After all the tests I'm rather confident we don't regress anything.
If any we might face issues we had before as well, we have to be careful when verifying these to not block for new but only regressing issues - or this will never complete :-)

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Is https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1668244 still an issue with this update? The udev rules look different

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks Andreas for the ping, yes this update will include the upstream fix to bug 1668244 as well.
If it fixes the issue of the reporter I'm not 100% sure, but I'll update and subscribe to the other bug if not. We really don't need another unrelated bug stopping this backport :-)

Actually we only wait for the SRU Team to action for a while now, I'm afraid this SRU being more complex is punted by many for its complexity in favor of other SRUs - I'll ask them once more.

Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello evade, or anyone else affected,

Accepted open-vm-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.2.0-3~ubuntu0.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in open-vm-tools (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
Revision history for this message
Chris Halse Rogers (raof) wrote :

Hello evade, or anyone else affected,

Accepted open-vm-tools into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.2.0-3~ubuntu0.17.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in open-vm-tools (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed-artful
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

1. Upgrade from proposed
Binaries usually installed are open-vm-tools and open-vm-tools-desktop
I see 10.2.0-3~... in Xenial and Artful, checking if the upgrade works without issues.

On the upgrade one sees a list of new dependencies libdrm2, libudev, libxml2 libxmlsec1 libxmlsec1-openssl pciutils and iproute2 - fortunately all are in main - so no undetected component mismatch.

Note: as host I used VMWare Workstation 14.1.1 and systems installed from 16.04.4 and 17.10.1 iso as of today.

Artful: upgraded without issues
Xenial: upgraded withous issues

Note: the default console setup can't copy and paste, so no full log here :-/

2. This bug in particular is about making the new version available.
Check vmtoolsd -v for the version being on 10.2
Artful: ok 10.2.0.1608
Xenial: ok 10.2.0.1608

Per above (and the extensive precheck on the content-equal ppa by VMWare) - setting to verified

tags: added: verification-done verification-done-artful verification-done-xenial
removed: verification-needed verification-needed-artful verification-needed-xenial
Revision history for this message
evade (evade) wrote :

Thank you all for your help!
I'll get this version and test as soon as I'm able.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The Upload was also tested by the reporter against another bug that will be fixed by the backport.
So I dup'ed 1668244 onto this bug here.

+1 on the backport adressing more issues :-)

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (10.0 KiB)

This bug was fixed in the package open-vm-tools - 2:10.2.0-3~ubuntu0.16.04.1

---------------
open-vm-tools (2:10.2.0-3~ubuntu0.16.04.1) xenial; urgency=medium

  * backport Bionic open-vm-tools to Xenial (LP: #1741390)
    - d/control: B-D for dh-autoreconf and dh-systemd
    - d/rules: re-add autoreconf and systemd
    - d/control: B-D to debhelper version of xenial supporting compat 10
    - d/compat: compat level 10 is latest xenial supported level
    - d/rules: go back from override_dh_installsystemd to
      override_dh_systemd_enable and override_dh_systemd_start as needed on
      xenials debhelper version.
    - d/rules: drop --no-restart-after-upgrade which only exists in debhelper
      10, the version 9 behavior defaults to what we need.
    - debian/rules: dh_systemd_start in xenial has issues with the escaping of
      the run-vmblock\\x2dfuse.mount job, so drop this call
    - debian/open-vm-tools-desktop.postinst: add a fixed version of what
      dh_systemd_start would have added
    - d/open-vm-tools-desktop.prerm, d/open-vm-tools-desktop.postrm: add what
      dh_systemd_start would have added (as-is since those sections worked)
    - d/control: update maintainers
  * d/open-vm-tools.service: Add After=local-fs.target dependency ensuring
    filesystems are ready to fix a race on startup (LP: #1750780)

open-vm-tools (2:10.2.0-3ubuntu3) bionic; urgency=medium

  * Disable PrivateTmp for the open-vm-tools.service as it triggers issues
    when triggering processes that need tmp through VMOMI API (LP: #1758428)

open-vm-tools (2:10.2.0-3ubuntu2) bionic; urgency=medium

  * Revert change in d/open-vm-tools.service that added After=local-fs.target.
    It turned out that the systemd in bionic already implicitly fixes
    this issue (the change is still needed for backports) (LP: 1750780)

open-vm-tools (2:10.2.0-3ubuntu1) bionic; urgency=medium

  * d/open-vm-tools.service: Add After=local-fs.target dependency ensuring
    filesystems are ready to fix a race on startup (LP: #1750780)

open-vm-tools (2:10.2.0-3) unstable; urgency=medium

  * [47e50a1] Fix debhelper dep for backports
  * [34538a5] Make tools.conf useful.
    Thanks to Dariusz Gadomski (Closes: #889884)

open-vm-tools (2:10.2.0-2) unstable; urgency=medium

  * [249d54c] Fix wayland segfault.
    Adding a patch from Fedora to fix a wayland/gnome related segfault.
    Thanks to Oliver Kurth (Closes: #887755)

open-vm-tools (2:10.2.0-1) unstable; urgency=medium

  * [f0bf956] Add .travis.yml which was removed by gbp.
  * [892e2f6] Build with gtk3.
  * [03a655b] Check if debhelper handles \ in systemd units.
    Thanks to Oliver Kurth (Closes: #886191)
  * [5bf9301] Drop -dkms package.
    Thanks to Christian Ehrhardt (Closes: #884656)
  * [236cdba] Update upstream source from tag 'upstream/10.2.0'
    Update to upstream version '10.2.0'
    with Debian dir d5190e486b6beb65ee7ed31c0c23a789b8f60cab
    (Closes: #884496)
  * [692beff] snapshot changelog.
  * [45aa743] Add .travis.yml which was removed by gbp.
  * [aabeded] Fix dpkg --compare-versions call
  * [0697425] Better debhelper version parsing.
  * [390ec09] fix even more makefile bugs.
  * [6e5fa38] Refreshing...

Changed in open-vm-tools (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for open-vm-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (3.7 KiB)

This bug was fixed in the package open-vm-tools - 2:10.2.0-3~ubuntu0.17.10.1

---------------
open-vm-tools (2:10.2.0-3~ubuntu0.17.10.1) artful; urgency=medium

  * backport Bionic open-vm-tools to Artful (LP: #1741390)
    - d/control: B-D for dh-autoreconf and dh-systemd
    - d/rules: re-add autoreconf and systemd
    - d/control: B-D to debhelper version of xenial supporting compat 10
    - d/compat: compat level 10 is latest xenial supported level
    - d/rules: go back from override_dh_installsystemd to
      override_dh_systemd_enable and override_dh_systemd_start as needed on
      xenials debhelper version.
    - d/rules: drop --no-restart-after-upgrade which only exists in debhelper
      10, the version 9 behavior defaults to what we need.
    - debian/rules: dh_systemd_start in xenial has issues with the escaping of
      the run-vmblock\\x2dfuse.mount job, so drop this call
    - debian/open-vm-tools-desktop.postinst: add a fixed version of what
      dh_systemd_start would have added
    - d/open-vm-tools-desktop.prerm, d/open-vm-tools-desktop.postrm: add what
      dh_systemd_start would have added (as-is since those sections worked)
    - d/control: update maintainers
  * d/open-vm-tools.service: Add After=local-fs.target dependency ensuring
    filesystems are ready to fix a race on startup (LP: #1750780)

open-vm-tools (2:10.2.0-3ubuntu3) bionic; urgency=medium

  * Disable PrivateTmp for the open-vm-tools.service as it triggers issues
    when triggering processes that need tmp through VMOMI API (LP: #1758428)

open-vm-tools (2:10.2.0-3ubuntu2) bionic; urgency=medium

  * Revert change in d/open-vm-tools.service that added After=local-fs.target.
    It turned out that the systemd in bionic already implicitly fixes
    this issue (the change is still needed for backports) (LP: 1750780)

open-vm-tools (2:10.2.0-3ubuntu1) bionic; urgency=medium

  * d/open-vm-tools.service: Add After=local-fs.target dependency ensuring
    filesystems are ready to fix a race on startup (LP: #1750780)

open-vm-tools (2:10.2.0-3) unstable; urgency=medium

  * [47e50a1] Fix debhelper dep for backports
  * [34538a5] Make tools.conf useful.
    Thanks to Dariusz Gadomski (Closes: #889884)

open-vm-tools (2:10.2.0-2) unstable; urgency=medium

  * [249d54c] Fix wayland segfault.
    Adding a patch from Fedora to fix a wayland/gnome related segfault.
    Thanks to Oliver Kurth (Closes: #887755)

open-vm-tools (2:10.2.0-1) unstable; urgency=medium

  * [f0bf956] Add .travis.yml which was removed by gbp.
  * [892e2f6] Build with gtk3.
  * [03a655b] Check if debhelper handles \ in systemd units.
    Thanks to Oliver Kurth (Closes: #886191)
  * [5bf9301] Drop -dkms package.
    Thanks to Christian Ehrhardt (Closes: #884656)
  * [236cdba] Update upstream source from tag 'upstream/10.2.0'
    Update to upstream version '10.2.0'
    with Debian dir d5190e486b6beb65ee7ed31c0c23a789b8f60cab
    (Closes: #884496)
  * [692beff] snapshot changelog.
  * [45aa743] Add .travis.yml which was removed by gbp.
  * [aabeded] Fix dpkg --compare-versions call
  * [0697425] Better debhelper version parsing.
  * [390ec09] fix even more makefile bugs.
  * [6e5fa38] Refreshing...

Read more...

Changed in open-vm-tools (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
yanhuih@vmware.com (yanhuih) wrote :

Env:
VMs:
* ubuntu-16.04.4-desktop-amd64
* ubuntu-16.04.4-server-amd64
* ubuntu-17.10-desktop-amd64
* ubuntu-17.10-server-amd64

open-vm-tools version: 10.2.0.1608(build-7253323) from ppa:ci-train-ppa-service/3152 or using the default repository

Result:
open-vm-tools can be installed and uninstalled successfully except the following 2 problems.

Problem:
There is 2 problems.
1) After install open-vm-tools, ubuntu 17.10 desktop needs reboot to validate vmtoolsd
2) During install open-vm-tools, ubuntu 17.10 server needs to ask this question about "tools.conf" to continue. (I'm not sure it's necessary.)
----------
......
Installing new version of config file /etc/vmware-tools/scripts/vmware/network ...

Configuration file '/etc/vmware-tools/tools.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
      D : show the differences between the versions
      Z : start a shell to examine the situation
 The default action is to keep your current version.
*** tools.conf (Y/I/N/O/D/Z) [default=N] ? y
Installing new version of config file /etc/vmware-tools/tools.conf ...
Processing triggers for libc-bin (2.26-0ubuntu2) ...
Processing triggers for systemd (234-2ubuntu12) ...
Processing triggers for man-db (2.7.6.1-2) ...
----------

Steps:
1) add repository
sudo add-apt-repository ppa:ci-train-ppa-service/3152
sudo apt-get update

2) update to / install open-vm-tools 10.2.0, and then check tools, vgauth, vmware-toolbox-cmd

3) reboot guestOS and check tools, vgauth, vmware-toolbox-cmd

4) uninstall open-vm-tools and then check tools, vgauth, vmware-toolbox-cmd

5) reboot guestOS and check tools, vgauth, vmware-toolbox-cmd

6) install/uninstall open-vm-tools-desktop

If ignore anything, please kindly tell me.

Thanks!
Yanhui

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Yanhui,
thanks for the retest the issues are not part of the ppa/SRU.
Also this is released already, so no need for the ppa anymore.

#2 is intentional if you had a modified tools.conf due to conffile handling

#1 we can look at when working on the next version.
   If you could outline - best in a new bug - the symptom for "reboot to validate vmtoolsd" then we can check that when pulling an even newer version for Ubuntu 18.10 and later the backport for 18.04.

no longer affects: xenial-backports
Revision history for this message
yanhuih@vmware.com (yanhuih) wrote :

Hi Christian,

I have filed a new bug as below.

https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1773931

If have any problems, please kindly tell me!

Thanks!
Yanhui

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.