Comment 12 for bug 1750780

Installed another Xenial and Bionic in vmware to take a deper look.
- Xenial (with backported open-vm-tools): affected
- Bionic (with the interim fix reverted): no hit in several retries, explanation below

Systemd fixed it (via our assumed implicit dependency).
In Bionic the PrivateTmp gives it a dependency on systemd-tmpfile-setup.service (seen in systemd analyze, there might be more but not on crit path).
This is configured by default to include /var/tmp in /usr/lib/tmpfiles.d/tmp.conf.

In regard to your thoughts about later on changing cloud-init ordering that won't help you, as the dependency is there (implicit or explicit doesn't matter).

For the xenial case where I reliably hit the issue instead of stracing I cut things short.
A service with the following exposes exactly the same error:
[Unit]
Description=foo
DefaultDependencies=no

[Service]
PrivateTmp=yes
ExecStart=/bin/true

[Install]
WantedBy=multi-user.target

So back on Xenial it is privateTmp + too early that breaks it.

Xenial vs Bionic critical-chain according to "systemd-analyze critical-chain open.vm-tools.service"

Xenial with fix:
open-vm-tools.service @3.482s
└─local-fs.target @3.460s
  └─local-fs-pre.target @3.460s
    └─systemd-remount-fs.service @3.442s +9ms
      └─system.slice @220ms
        └─-.slice @204m

Xenial without fix:
└─run-vmblock\x2dfuse.mount @6.076s +390ms
  └─sys-fs-fuse-connections.mount @5.510s +375ms
    └─systemd-modules-load.service @1.996s +75ms
      └─system.slice @1.984s
        └─-.slice @1.966s

Bionic
open-vm-tools.service @3.566s
└─systemd-tmpfiles-setup.service @3.421s +100ms
  └─systemd-journal-flush.service @3.054s +342ms
    └─systemd-journald.service @825ms +2.219s
      └─syslog.socket @808ms
        └─system.slice @621ms
          └─-.slice @613ms

To Summarize, we can:
- revert the fix for Bionic (or later) - just make it a sync when convenient down the road, it doesn't hurt for now as it is (almost) the same as the implicit dependency)
- add a xenials systemd bug task (probably too complex to fix as -upstream)
- until said systemd bug is fixed a backport of open-vm-tools needs this fix