kolla-ansible

[fluentd] Elasticsearch output enabled even when service disabled

  1. Series wallaby
  2. Bug #1927880
Bug #1927880 reported by Scott Shambarger
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
Medium
Scott Shambarger
Ussuri
Fix Released
Medium
Unassigned
Victoria
Fix Released
Medium
Unassigned
Wallaby
Fix Committed
Medium
Unassigned
Xena
Fix Released
Medium
Scott Shambarger

Bug Description

Seen on v11, but appears in v12/master too...

Deploying all-in-one with the following in global.yml (elasticsearch disabled by default):

kolla_internal_vip_address: "192.168.60.10"
kolla_internal_fqdn: "cluster.test.lan"

fluentd container contains td-agent.conf with elasticsearch output <store> entries, and fluentd repeatedly fails to connect to it, from fluentd.log:

#0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 192.168.60.10:9200 (Errno::ECONNREFUSED)

THE FIX:
It appears that there's an error in ansible/roles/common/tasks/config.yml (about line 105):

- name: Copying over td-agent.conf
  vars:
    log_direct_to_elasticsearch: >-
      {{ ( enable_elasticsearch | bool or
           ( elasticsearch_address != kolla_internal_vip_address )) and
         not enable_monasca | bool }}

log_direct_to_elasticsearch is treated as True as ansible/group_vars/all.yml (about line 748) has:

elasticsearch_address: "{{ kolla_internal_fqdn }}"

Clearly a name is being compared to an IP. It appears the comparison should be:

    elasticsearch_address != kolla_internal_fqdn

... that, or elasticsearch_address is not using the correct default?

Revision history for this message
Mark Goddard (mgoddard) wrote :

Looks like a good bug with an easy solution, as proposed - compare with the FQDN. Can you propose a fix to gerrit?

Changed in kolla-ansible:
importance: Undecided → Medium
status: New → In Progress
status: In Progress → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/791321

Changed in kolla-ansible:
status: Triaged → In Progress
Scott Shambarger (sshambar)
Changed in kolla-ansible:
assignee: nobody → Scott Shambarger (sshambar)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/791321
Committed: https://opendev.org/openstack/kolla-ansible/commit/283b0dec639c032945f5d1d87a66f0c7541c0562
Submitter: "Zuul (22348)"
Branch: master

commit 283b0dec639c032945f5d1d87a66f0c7541c0562
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac

Changed in kolla-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/804334

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/804357

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/804337

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/804334
Committed: https://opendev.org/openstack/kolla-ansible/commit/25ee0b693a89f44a09df1b5c522c553b1f8717fb
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 25ee0b693a89f44a09df1b5c522c553b1f8717fb
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/804357
Committed: https://opendev.org/openstack/kolla-ansible/commit/085a0852c9f70c4149eccd68dc8b8e89be25850b
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 085a0852c9f70c4149eccd68dc8b8e89be25850b
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac
    (cherry picked from commit 283b0dec639c032945f5d1d87a66f0c7541c0562)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/804337
Committed: https://opendev.org/openstack/kolla-ansible/commit/c65e1035b0406807d22f944f534138733a07c451
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit c65e1035b0406807d22f944f534138733a07c451
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac
    (cherry picked from commit 283b0dec639c032945f5d1d87a66f0c7541c0562)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 12.2.0

This issue was fixed in the openstack/kolla-ansible 12.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 13.0.0.0rc1

This issue was fixed in the openstack/kolla-ansible 13.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 10.4.0

This issue was fixed in the openstack/kolla-ansible 10.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 11.2.0

This issue was fixed in the openstack/kolla-ansible 11.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.