Check IPs specified in undercloud.conf aren't going to cause issues or conflict

Bug #1832168 reported by Brendan Shephard
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Takashi Kajinami

Bug Description

Description
===========
If the same IP is used for any of these options in undercloud.conf:
local_ip undercloud_admin_host undercloud_public_host

It will cause issues with the endpoints and haproxy for services. In this specific case, the zaqar_ws haproxy service was using the same bind and server IP address.

Steps to reproduce
==================
1. Define the same IP address for local_ip and undercloud_public_host as the same address:
local_ip = 192.168.24.2/24
undercloud_public_host = 192.168.24.2
undercloud_admin_host = 192.168.24.3

2. openstack undercloud install
3. Try to deploy the overcloud.

Expected result
===============
Undercloud install should throw an error advising that this can't be done.

Actual result
=============
Traceback:
clean_up DeployOvercloud: Handshake status 502 Bad Gateway
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 136, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 281, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 176, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 401, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 32, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action
    self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir
    new_tht_root, tht_root)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 386, in _deploy_tripleo_heat_templates
    plans = plan_management.list_deployment_plans(self.clients)
  File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/plan_management.py", line 129, in list_deployment_plans
    with tripleoclients.messaging_websocket() as ws:
  File "/usr/lib/python2.7/site-packages/tripleoclient/plugin.py", line 211, in messaging_websocket
    cacert=self._instance.cacert)
  File "/usr/lib/python2.7/site-packages/tripleoclient/plugin.py", line 89, in __init__
    sslopt=OS_CACERT)
  File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 511, in create_connection
    websock.connect(url, **options)
  File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 223, in connect
    self.handshake_response = handshake(self.sock, *addrs, **options)
  File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 79, in handshake
    status, resp = _get_resp_headers(sock)
  File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 152, in _get_resp_headers
    raise WebSocketBadStatusException("Handshake status %d %s", status, status_message)
WebSocketBadStatusException: Handshake status 502 Bad Gateway

Checking haproxy.cfg:
listen zaqar_ws
  bind 192.168.24.2:9000 ssl crt /etc/pki/tls/private/overcloud_endpoint.pem <<---- This IP here
  bind 192.168.24.3:9000
  mode http
  option forwardfor
  redirect scheme https code 301 if { hdr(host) -i 192.168.24.2 } !{ ssl_fc }
  rsprep ^Location:\ http://(.*) Location:\ https://\1
  timeout connect 5s
  timeout client 25s
  timeout server 25s
  timeout tunnel 14400s
  server undercloud 192.168.24.2:9000 check fall 5 inter 2000 rise 2 <<--- Is the same IP as here

Environment
===========
python2-tripleoclient-11.5.0-0.20190604104358.b44d6f8.el7.noarch

Changed in tripleo:
assignee: nobody → Brendan Shephard (bshephar)
status: New → In Progress
Changed in tripleo:
milestone: none → train-2
importance: Undecided → Medium
Changed in tripleo:
milestone: train-2 → train-3
Changed in tripleo:
assignee: Brendan Shephard (bshephar) → Takashi Kajinami (kajinamit)
Changed in tripleo:
milestone: train-3 → ussuri-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (master)

Reviewed: https://review.opendev.org/664224
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=2600260be5ad99da36775e470758591d8fcd2ca9
Submitter: Zuul
Branch: master

commit 2600260be5ad99da36775e470758591d8fcd2ca9
Author: Brendan <email address hidden>
Date: Mon Jun 10 16:52:25 2019 +1000

    Ensure no duplicate IPs are used in undercloud.conf

    If duplicate IPs are used in undercloud.conf it will cause
    issues with haproxy.

    Closes-Bug: #1832168
    Change-Id: I35547080aee943438d34b2d62632795af32f8462

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/681878

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (stable/stein)

Reviewed: https://review.opendev.org/681878
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=00fe507ed61a664ba5d2d8bc7275221d8a66ec68
Submitter: Zuul
Branch: stable/stein

commit 00fe507ed61a664ba5d2d8bc7275221d8a66ec68
Author: Brendan <email address hidden>
Date: Mon Jun 10 16:52:25 2019 +1000

    Ensure no duplicate IPs are used in undercloud.conf

    If duplicate IPs are used in undercloud.conf it will cause
    issues with haproxy.

    Closes-Bug: #1832168
    Change-Id: I35547080aee943438d34b2d62632795af32f8462
    (cherry picked from commit 2600260be5ad99da36775e470758591d8fcd2ca9)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 12.2.0

This issue was fixed in the openstack/python-tripleoclient 12.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 11.5.2

This issue was fixed in the openstack/python-tripleoclient 11.5.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/746671

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/746672

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.opendev.org/746698

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-tripleoclient (master)

Change abandoned by Harald Jensås (<email address hidden>) on branch: master
Review: https://review.opendev.org/746672
Reason: I think we can go for Takashi's patch instead: https://review.opendev.org/746698

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/746671
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=94997223b6173be29fb621d827f8a7ecfa4b6bfa
Submitter: Zuul
Branch: master

commit 94997223b6173be29fb621d827f8a7ecfa4b6bfa
Author: Harald Jensås <email address hidden>
Date: Tue Aug 18 11:36:20 2020 +0200

    Fix endpoint map tls - zaqar wss port

    We open port: `` 3000 #SSL for websocket`` in the
    zaqar service defenition:
      deployment/zaqar/zaqar-container-puppet.yaml:L130

    But SSL environment files use port 9000 for the public
    endpoint.

    Using 9000 also for SSL can cause issues in haproxy.
    We may want to revert or relax the check in
    https://review.opendev.org/664224 as duplicate IP's
    is'nt the actual problem.

    Related-Bug: #1832168
    Related-RHBZ: #1868910
    Change-Id: I05f31885ade46d47ff5d384dabbd5561f4df9278

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (master)

Reviewed: https://review.opendev.org/746698
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=5b246e4f1aa86f8fe2cadb5fccf087dafa36523e
Submitter: Zuul
Branch: master

commit 5b246e4f1aa86f8fe2cadb5fccf087dafa36523e
Author: Takashi Kajinami <email address hidden>
Date: Tue Aug 18 21:30:16 2020 +0900

    Allow usage of duplicated IPs for undercloud config

    Change 2600260be5ad99da36775e470758591d8fcd2ca9 introduced validation
    to avoid duplicated usage of undercloud IPs, but in fact we don't need
    to ensure complete unieqness of local_ip, undercloud_admin_host and
    undercloud_public_host, but it should be enough if local_ip and
    undercloud_admin_host are different, because public endpoints should
    listen on different ports when ssl is enabled.

    This patch eases the validation introduced by the previous patch, so
    that we allow the deployment especially with the same ip used for
    public_host and admin_host, which was allowed in old releases.

    Depends-On: https://review.opendev.org/746671
    Change-Id: I932482e097d62f46e02eb035435d2bc0d5548b2a
    Related-Bug: #1832168
    Related: RHBZ#1868910

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/747479

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/747480

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/747508

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/747509

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (stable/ussuri)

Reviewed: https://review.opendev.org/747479
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=91c17d54ecdc2451cc76df1481fa40012fba8d2e
Submitter: Zuul
Branch: stable/ussuri

commit 91c17d54ecdc2451cc76df1481fa40012fba8d2e
Author: Harald Jensås <email address hidden>
Date: Tue Aug 18 11:36:20 2020 +0200

    Fix endpoint map tls - zaqar wss port

    We open port: `` 3000 #SSL for websocket`` in the
    zaqar service defenition:
      deployment/zaqar/zaqar-container-puppet.yaml:L130

    But SSL environment files use port 9000 for the public
    endpoint.

    Using 9000 also for SSL can cause issues in haproxy.
    We may want to revert or relax the check in
    https://review.opendev.org/664224 as duplicate IP's
    is'nt the actual problem.

    Related-Bug: #1832168
    Related-RHBZ: #1868910
    Change-Id: I05f31885ade46d47ff5d384dabbd5561f4df9278
    (cherry picked from commit 94997223b6173be29fb621d827f8a7ecfa4b6bfa)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/747480
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=1848fa025e2b405c4d04a428b9668e70eda4dbbf
Submitter: Zuul
Branch: stable/train

commit 1848fa025e2b405c4d04a428b9668e70eda4dbbf
Author: Harald Jensås <email address hidden>
Date: Tue Aug 18 11:36:20 2020 +0200

    Fix endpoint map tls - zaqar wss port

    We open port: `` 3000 #SSL for websocket`` in the
    zaqar service defenition:
      deployment/zaqar/zaqar-container-puppet.yaml:L130

    But SSL environment files use port 9000 for the public
    endpoint.

    Using 9000 also for SSL can cause issues in haproxy.
    We may want to revert or relax the check in
    https://review.opendev.org/664224 as duplicate IP's
    is'nt the actual problem.

    Related-Bug: #1832168
    Related-RHBZ: #1868910
    Change-Id: I05f31885ade46d47ff5d384dabbd5561f4df9278
    (cherry picked from commit 94997223b6173be29fb621d827f8a7ecfa4b6bfa)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (stable/train)

Reviewed: https://review.opendev.org/747509
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=8e7a8e76b1db36e1e0c9bebfdceb0d73d72f459d
Submitter: Zuul
Branch: stable/train

commit 8e7a8e76b1db36e1e0c9bebfdceb0d73d72f459d
Author: Takashi Kajinami <email address hidden>
Date: Tue Aug 18 21:30:16 2020 +0900

    Allow usage of duplicated IPs for undercloud config

    Change 2600260be5ad99da36775e470758591d8fcd2ca9 introduced validation
    to avoid duplicated usage of undercloud IPs, but in fact we don't need
    to ensure complete unieqness of local_ip, undercloud_admin_host and
    undercloud_public_host, but it should be enough if local_ip and
    undercloud_admin_host are different, because public endpoints should
    listen on different ports when ssl is enabled.

    This patch eases the validation introduced by the previous patch, so
    that we allow the deployment especially with the same ip used for
    public_host and admin_host, which was allowed in old releases.

    Depends-On: https://review.opendev.org/747480
    Change-Id: I932482e097d62f46e02eb035435d2bc0d5548b2a
    Related-Bug: #1832168
    Related: RHBZ#1868910
    (cherry picked from commit 5b246e4f1aa86f8fe2cadb5fccf087dafa36523e)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (stable/ussuri)

Reviewed: https://review.opendev.org/747508
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=c2a770648933e9ac1cef7eae33662aa2653ab5c4
Submitter: Zuul
Branch: stable/ussuri

commit c2a770648933e9ac1cef7eae33662aa2653ab5c4
Author: Takashi Kajinami <email address hidden>
Date: Tue Aug 18 21:30:16 2020 +0900

    Allow usage of duplicated IPs for undercloud config

    Change 2600260be5ad99da36775e470758591d8fcd2ca9 introduced validation
    to avoid duplicated usage of undercloud IPs, but in fact we don't need
    to ensure complete unieqness of local_ip, undercloud_admin_host and
    undercloud_public_host, but it should be enough if local_ip and
    undercloud_admin_host are different, because public endpoints should
    listen on different ports when ssl is enabled.

    This patch eases the validation introduced by the previous patch, so
    that we allow the deployment especially with the same ip used for
    public_host and admin_host, which was allowed in old releases.

    Depends-On: https://review.opendev.org/747479
    Change-Id: I932482e097d62f46e02eb035435d2bc0d5548b2a
    Related-Bug: #1832168
    Related: RHBZ#1868910
    (cherry picked from commit 5b246e4f1aa86f8fe2cadb5fccf087dafa36523e)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/748673

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (master)

Reviewed: https://review.opendev.org/748673
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=4a52809a9676f3058863882b529f88c5f078c9ee
Submitter: Zuul
Branch: master

commit 4a52809a9676f3058863882b529f88c5f078c9ee
Author: Harald Jensås <email address hidden>
Date: Fri Aug 28 15:30:11 2020 +0200

    Fix tls - zaqar wss default port

    In https://review.opendev.org/746671 we updated the
    endpoint map. It seems this broke the undercloud as
    haproxy still listen on port 9000.

    Update the default zaqar wss port in puppet as well.

    Also set the request header Host to use the service
    port so that handshake does not fail on port missmatch.

    Change-Id: I0c9dd4e9457c7cfa8cd54b5eaa167bf4e8e6bcc1
    Closes-Bug: #1893621
    Related-Bug: #1832168
    Related-RHBZ: #1868910

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/749161

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/749162

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (stable/ussuri)

Reviewed: https://review.opendev.org/749161
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=e8dfd034c51500a8fa362a30a60615ae8efca8c0
Submitter: Zuul
Branch: stable/ussuri

commit e8dfd034c51500a8fa362a30a60615ae8efca8c0
Author: Harald Jensås <email address hidden>
Date: Fri Aug 28 15:30:11 2020 +0200

    Fix tls - zaqar wss default port

    In https://review.opendev.org/746671 we updated the
    endpoint map. It seems this broke the undercloud as
    haproxy still listen on port 9000.

    Update the default zaqar wss port in puppet as well.

    Also set the request header Host to use the service
    port so that handshake does not fail on port missmatch.

    Change-Id: I0c9dd4e9457c7cfa8cd54b5eaa167bf4e8e6bcc1
    Closes-Bug: #1893621
    Related-Bug: #1832168
    Related-RHBZ: #1868910
    (cherry picked from commit 4a52809a9676f3058863882b529f88c5f078c9ee)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (stable/train)

Reviewed: https://review.opendev.org/749162
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=72ad205fa85377663b72d9dc7d297452098a41cb
Submitter: Zuul
Branch: stable/train

commit 72ad205fa85377663b72d9dc7d297452098a41cb
Author: Harald Jensås <email address hidden>
Date: Fri Aug 28 15:30:11 2020 +0200

    Fix tls - zaqar wss default port

    In https://review.opendev.org/746671 we updated the
    endpoint map. It seems this broke the undercloud as
    haproxy still listen on port 9000.

    Update the default zaqar wss port in puppet as well.

    Also set the request header Host to use the service
    port so that handshake does not fail on port missmatch.

    Change-Id: I0c9dd4e9457c7cfa8cd54b5eaa167bf4e8e6bcc1
    Closes-Bug: #1893621
    Related-Bug: #1832168
    Related-RHBZ: #1868910
    (cherry picked from commit 4a52809a9676f3058863882b529f88c5f078c9ee)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.