etcdmain: initial cluster has duplicate url

Bug #1890071 reported by alpha23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Invalid
Undecided
Unassigned
Rocky
Triaged
Medium
Unassigned

Bug Description

Using rocky/stable centos/source

The following error occurs in etcd container logs on the new compute/storage host (<SERVER3>/<IP3> below) when deploying services on the new host (--limit <NEW_HOST>). This results in that etcd container continuously restarting. There are no issues with the etcd container on the existing hosts.

Running command: 'etcd'
+ exec etcd
2020-08-02 12:01:13.450356 I | pkg/flags: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://<IP3>:2379
2020-08-02 12:01:13.450445 I | pkg/flags: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd
2020-08-02 12:01:13.450465 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://<IP3>:2380
2020-08-02 12:01:13.450471 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER=<SERVER1>=http://<IP1>:2380,<SERVER2>=http://<IP2>:2380,<SERVER1>=http://<IP1>:2380,<SERVER3>=http://<IP3>:2380
2020-08-02 12:01:13.450475 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=new
2020-08-02 12:01:13.450479 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_TOKEN=h5PyOEojNQfYBAAklH6Wb1SG6w3AwPHVzrb446JG
2020-08-02 12:01:13.450485 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://<IP3>:2379
2020-08-02 12:01:13.450490 I | pkg/flags: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://<IP3>:2380
2020-08-02 12:01:13.450501 I | pkg/flags: recognized and used environment variable ETCD_NAME=<SERVER3>
2020-08-02 12:01:13.450526 W | pkg/flags: unrecognized environment variable ETCD_OUT_FILE=/var/log/kolla/etcd/etcd.log
2020-08-02 12:01:13.450568 I | etcdmain: etcd Version: 3.3.11
2020-08-02 12:01:13.450574 I | etcdmain: Git SHA: 2cf9e51
2020-08-02 12:01:13.450577 I | etcdmain: Go Version: go1.10.3
2020-08-02 12:01:13.450580 I | etcdmain: Go OS/Arch: linux/amd64
2020-08-02 12:01:13.450583 I | etcdmain: setting maximum number of CPUs to 24, total number of available CPUs is 24
2020-08-02 12:01:13.450704 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2020-08-02 12:01:13.450824 I | embed: listening for peers on http://<IP3>:2380
2020-08-02 12:01:13.450884 I | embed: listening for client requests on <IP3>:2379
etcdmain: initial cluster <SERVER1>=http://<IP1>:2380,<SERVER1>=http://<IP1>:2380,<SERVER2>=http://<IP2>:2380,<SERVER3>=http://<IP3>:2380 has duplicate url

It may be related to: https://github.com/scality/metalk8s/issues/2083 however, how can this be resolved in a kolla-ansible reconfiguration?

Tags: rocky
Revision history for this message
Mark Goddard (mgoddard) wrote :

I think this is caused by using hosts which are in both the control and compute groups:

ETCD_INITIAL_CLUSTER: "{% for host in etcd_hosts %}{{ hostvars[host]['ansible_hostname'] }}={{ internal_protocol }}://{{ hostvars[host]['ansible_' + hostvars[host]['api_interface']]['ipv4']['add ress'] }}:{{ etcd_peer_port }}{% if not loop.last %},{% endif %}{% endfor %}"

etcd_hosts: "{{ groups['control'] + (groups['compute'] if enable_kuryr | bool else []) }}"

There probably needs to be some uniqueness enforcement, e.g. the unique filter in etcd_hosts.

Note that this probably doesn't affect Stein & later releases, which just iterate over the etcd group.

As a workaround you could override the etcd_hosts variable.

Changed in kolla:
status: New → Triaged
importance: Undecided → Medium
affects: kolla → kolla-ansible
no longer affects: kolla/rocky
Changed in kolla-ansible:
status: Triaged → Invalid
importance: Medium → Undecided
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.