cloud-init disables user on azure at second reboot

Hi,
Please attach the tarball created by running 'cloud-init collect-logs'.
Then set the status back to New.

As it is, there is not enough information available here to see what happened.

Thanks for filing a bug.
Scott

Hello,

I know that /sys/devices/virtual/dmi/id/product_uuid gives the unicity of the instance to cloud-init on Azure platform.
In this case, after upstream kernel install, that UUID value gets from upper case to lower case.

Example:
before kernel install: B016A6CF-B90D-B743-8379-902BBF5E0037
after upstream kernel install: b016a6cf-b90d-b743-8379-902bbf5e0037

I checked the linux kernel tree and I found the commit responsible for this change:
https://github.com/torvalds/linux/commit/712ff25450bd01366301eef81c33e865d901e7b7#diff-f2bd14bc67b5e2da67116bca971bbd0b

It seems that the current 16.04/18.04 Azure kernels do not have this patch and when I install the upstream kernel, the UUID changes -> cloud-init reprovisions the instance.

This issue is similar to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1551419.

Thank you,
Adrian Vladu

Marking this New so we can re-look at it as this feels like case insensitivity in UUID matching is something we really should be handling.

Hi.
I added a 'linux' package task here. It seems that in bug 1551419 patches were added
to the ubuntu kernel to do the right thing. Possibly those patches were inadvertantly
dropped? Either way, it seems like the [Ubuntu] kernel should promise this consistently
so that multiple consumers do not need to all know about this issue and handle it
themselves.

That said, we will fix cloud-init to handle it.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1803173

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Hi Adrian,

In a system where this recreates, could you please run:
  cloud-init collect-logs
and attach the output?

Thank you.

Hi,
I recreated the issue using the Ubuntu upstream kernel builds at
  http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/

After launching a 18.04 instance, and then installing those kernels
and rebooting (generic_4.20.0-999.201811252100) I saw the issue.

I noticed that cloud-init's /var/lib/cloud data was getting messed up,
which didn't make any sense. The problem that I noticed was
/var/lib/cloud/instance was a directory rather than a symlink.
It turns out that the problem was walinuxagent was deleting the
/var/lib/cloud during boot. somewhere before cloud-init modules that
was getting deleted and was wreaking havoc on cloud-init.

It looks like this is at least identified as not the best idea at:
 https://github.com/Azure/WALinuxAgent/commit/f42d2e75617bb54

I verified that cloud-init was working properly by itself with:
  systemctl disable walinuxagent
before the reboot into the new kernel. All was well.
On reboot, cloud-init still used the azure datasource and had a single
entry in /var/lib/cloud/instances/

So, I'm marking this 'Invalid' for cloud-init. The fix needs to be
to have walinuxagent stop deleting state from other programs.

Thanks All.

A newer Azure Agent which no longer touches cloud-init data is already in the pipeline.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3279