tripleo

NetworkManager reload changes content of resolv.conf

Bug #1958989 reported by Emilien Macchi on 2022-01-25

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Released	Medium	Emilien Macchi	tripleo yoga-3

Bug Description

In https://opendev.org/openstack/tripleo-ansible/commit/6881079d12a9fb2cf752a5f56116c1b4d3a7b75d, we already tried to prevent NetworkManager from updating resolv.conf but it seems like it wasn't enough.

I've reproduced an issue where on CentOS8-Stream / TripleO master (25th of January 2022) the deployment will fail because /etc/resolv.conf is updated to only contain the searchdomain and all the previous content is nuked.

Looking at the timestamps & logs, it's caused by the NetworkManager reload; which no matter what will trigger a copy of /var/run/NetworkManager/resolv.conf into /etc/resolv.conf even with dns-none in the config.

The alternative is to set rc-manager=unmanaged, which from the doc says:

unmanaged: don't touch /etc/resolv.conf.

I think this is what we want in this case.

Tags:

Emilien Macchi (emilienm) on 2022-01-25

Changed in tripleo:
status:	New → Triaged
importance:	Undecided → Medium
assignee:	nobody → Emilien Macchi (emilienm)
milestone:	none → yoga-3

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-01-25: Fix proposed to tripleo-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/826287

Changed in tripleo:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-01-26: Fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/826287
Committed: https://opendev.org/openstack/tripleo-ansible/commit/9d0da2b383239d296b28ecb3b2458543dc0f0cb4
Submitter: "Zuul (22348)"
Branch: master

commit 9d0da2b383239d296b28ecb3b2458543dc0f0cb4
Author: Emilien Macchi <email address hidden>
Date: Tue Jan 25 08:39:09 2022 -0500

Force NetworkManager to unmanage /etc/resolv.conf

    The patch done via I989e3f1d14fd33e97933032111cf48166dc5f50c was
    apparently not enough and we have seen cases where the /etc/resolv.conf
    is still managed by NetworkManager (and DNS config nuked, causing
    deployment issues).

This patch will use the `rc-manager` option and set to `unmanaged` which
will really force NM to unmanage this file.

Closes-Bug: #1958989
Change-Id: Ib6a3f384f32c6c4fbd577a5bee62db620401f850

Changed in tripleo:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-01-27: Fix proposed to tripleo-ansible (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/826554

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-01-27: Fix proposed to tripleo-ansible (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/826555

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-01-27: Fix proposed to tripleo-ansible (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/826556

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-02-03: Fix merged to tripleo-ansible (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/826554
Committed: https://opendev.org/openstack/tripleo-ansible/commit/8ec777f4a55fa8111d781a73943b6c3f3d3d78c0
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 8ec777f4a55fa8111d781a73943b6c3f3d3d78c0
Author: Emilien Macchi <email address hidden>
Date: Tue Jan 25 08:39:09 2022 -0500

Force NetworkManager to unmanage /etc/resolv.conf

This patch will use the `rc-manager` option and set to `unmanaged` which
will really force NM to unmanage this file.

    Closes-Bug: #1958989
    Change-Id: Ib6a3f384f32c6c4fbd577a5bee62db620401f850
    (cherry picked from commit 9d0da2b383239d296b28ecb3b2458543dc0f0cb4)

tags:

added: in-stable-wallaby

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-02-03: Fix merged to tripleo-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/826556
Committed: https://opendev.org/openstack/tripleo-ansible/commit/57422a1ac557c2a5211a4d421e11484faf27634f
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 57422a1ac557c2a5211a4d421e11484faf27634f
Author: Emilien Macchi <email address hidden>
Date: Tue Jan 25 08:39:09 2022 -0500

Force NetworkManager to unmanage /etc/resolv.conf

This patch will use the `rc-manager` option and set to `unmanaged` which
will really force NM to unmanage this file.

    Closes-Bug: #1958989
    Change-Id: Ib6a3f384f32c6c4fbd577a5bee62db620401f850
    (cherry picked from commit 9d0da2b383239d296b28ecb3b2458543dc0f0cb4)

tags:	added: in-stable-ussuri
tags:	added: in-stable-victoria

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-02-03: Fix merged to tripleo-ansible (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/826555
Committed: https://opendev.org/openstack/tripleo-ansible/commit/9411786d6d0fad51eaed903561c04575ab2c18af
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 9411786d6d0fad51eaed903561c04575ab2c18af
Author: Emilien Macchi <email address hidden>
Date: Tue Jan 25 08:39:09 2022 -0500

Force NetworkManager to unmanage /etc/resolv.conf

This patch will use the `rc-manager` option and set to `unmanaged` which
will really force NM to unmanage this file.

    Closes-Bug: #1958989
    Change-Id: Ib6a3f384f32c6c4fbd577a5bee62db620401f850
    (cherry picked from commit 9d0da2b383239d296b28ecb3b2458543dc0f0cb4)