openvswitch is installed by tripleo but ovn-controller is failing

Bug #1921097 reported by Kamil Sambor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
In Progress
Undecided
Kamil Sambor

Bug Description

Description of problem:
openvswitch is installed by tripleo but not started on a controller containing just neutron_api which leads to failure during the OVS->OVN migration.
Attempt migration from OVS->OVN

Actual results:
ovn_controller container failing to start on non existent /run/openvswitch directory:

2021-03-17 14:05:47,314 INFO: 35043 -- Image already exists: director-c.infra.prod.upshift.rdu2.redhat.com:8787/rhosp-rhel8/openstack-neutron-server-ovn:16.1.3\n2021-03-17 14:05:50,411 ERROR: 35042 -- ['/usr/bin/podman', 'run', '--user', '0', '--name', 'container-puppet-ovn_controller', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', 'NAME=ovn_controller', '--env', 'HOSTNAME=ctrl-net-c-04', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpi9ii6nqu:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--volume', '/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'director-c.infra.prod.upshift.rdu2.redhat.com:8787/rhosp-rhel8/openstack-ovn-controller:16.1.3'] run failed after Error: error checking path \"/run/openvswitch\": stat /run/openvswitch: no such file or directory\n attempt(s): 1\n2021-03-17 14:05:50,412 WARNING: 35042 -- Retrying running container: ovn_controller\n2021-03-17 14:05:53,559 ERROR: 35042 -- ['/usr/bin/podman', 'run', '--user', '0', '--name', 'container-puppet-ovn_controller', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', 'NAME=ovn_controller', '--env', 'HOSTNAME=ctrl-net-c-04', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpi9ii6nqu:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--volume', '/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'director-c.infra.prod.upshift.rdu2.redhat.com:8787/rhosp-rhel8/openstack-ovn-controller:16.1.3'] run failed after Error: error checking path \"/run/openvswitch\": stat /run/openvswitch: no such file or directory\n attempt(s): 2

Expected results:
OVN migration succeeds and openvswitch is started out of the box

Kamil Sambor (ksambor)
Changed in tripleo:
assignee: nobody → Kamil Sambor (ksambor)
summary: - openvswitch is installed by tripleo but ovn-controller is failingr
+ openvswitch is installed by tripleo but ovn-controller is failing
Kamil Sambor (ksambor)
Changed in tripleo:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786968

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/ussuri)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786968
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/48c44479616b5a2a971b27c5b4d196397d8ea58d
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 48c44479616b5a2a971b27c5b4d196397d8ea58d
Author: Kamil Sambor <email address hidden>
Date: Wed Mar 24 11:46:35 2021 +0100

    Add systemd dependency to openvswitch to ovn-controller

    Ovn controller need openvswitch to be running properly. This
    patch adds the required depends_on so paunch can add the dependency to
    the systemd service file.

    Closes-bug: #1921097
    Change-Id: I2e54771f0a01d22ce95530bef146ea368189db24
    (cherry picked from commit 51059676b6b6d73648a6ffb3c6364a6f3b5ed42f)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786969
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/d81c5544d24b6567f1615f6065d340278f317604
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit d81c5544d24b6567f1615f6065d340278f317604
Author: Kamil Sambor <email address hidden>
Date: Wed Mar 24 11:46:35 2021 +0100

    Add systemd dependency to openvswitch to ovn-controller

    Ovn controller need openvswitch to be running properly. This
    patch adds the required depends_on so paunch can add the dependency to
    the systemd service file.

    Closes-bug: #1921097
    Change-Id: I2e54771f0a01d22ce95530bef146ea368189db24
    (cherry picked from commit 51059676b6b6d73648a6ffb3c6364a6f3b5ed42f)

tags: added: in-stable-ussuri
tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786970
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/c9fa94dd52cc20855586fb13e3ba51c2c2662f06
Submitter: "Zuul (22348)"
Branch: stable/train

commit c9fa94dd52cc20855586fb13e3ba51c2c2662f06
Author: Kamil Sambor <email address hidden>
Date: Wed Mar 24 11:46:35 2021 +0100

    Add systemd dependency to openvswitch to ovn-controller

    Ovn controller need openvswitch to be running properly. This
    patch adds the required depends_on so paunch can add the dependency to
    the systemd service file.

    Closes-bug: #1921097
    Change-Id: I2e54771f0a01d22ce95530bef146ea368189db24
    (cherry picked from commit 51059676b6b6d73648a6ffb3c6364a6f3b5ed42f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 14.1.0

This issue was fixed in the openstack/tripleo-heat-templates 14.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 13.3.0

This issue was fixed in the openstack/tripleo-heat-templates 13.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.6.0

This issue was fixed in the openstack/tripleo-heat-templates 11.6.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 12.4.4

This issue was fixed in the openstack/tripleo-heat-templates 12.4.4 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.