fluentd rewrite rules are using deprecated syntax

Bug #1730664 reported by Piotr Parczewski
38
This bug affects 9 people
Affects Status Importance Assigned to Milestone
kolla
Fix Released
High
Chason Chan
Ocata
Fix Released
Undecided
Jeffrey Zhang
Pike
Fix Released
Medium
Chason Chan
kolla-ansible
Fix Released
Critical
Chason Chan
Ocata
Fix Released
Critical
Jeffrey Zhang
Pike
Fix Released
Critical
Chason Chan
Queens
Fix Released
Critical
Chason Chan

Bug Description

Since version 2.0.0 of fluent rewrite filter, 'rewriteruleXX' is no longer valid syntax in 01-rewrite.conf. Therefore, after deployment fluentd container is in restarting state.

Revision history for this message
Piotr Parczewski (pparczewski) wrote :
Changed in kolla-ansible:
status: New → Confirmed
Changed in kolla-ansible:
importance: Undecided → Critical
milestone: none → queens-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.openstack.org/518490

Changed in kolla-ansible:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla-ansible (master)

Change abandoned by Chason Chan (<email address hidden>) on branch: master
Review: https://review.openstack.org/518490
Reason: Abandon this one because of the other sample way to fix the bug, see https://review.openstack.org/#/c/518820/3

Revision history for this message
Chason Chan (chen-xing) wrote :

This bug can be fixed in Kolla project, see https://review.openstack.org/518820

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.openstack.org/518490
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=419a2fc9fd206447f37fbcf62da8e15ab96f8530
Submitter: Zuul
Branch: master

commit 419a2fc9fd206447f37fbcf62da8e15ab96f8530
Author: chenxing <email address hidden>
Date: Wed Nov 8 08:39:35 2017 +0000

    Fix the deprecated syntax in 01-rewrite.conf

    The current syntax of Ubuntu fluentd rewrite rules are not supported
    anymore, reference to this commit[1].
    According to the build of this patch[2], The Centos has no such isuue.
    Only Ubuntu needs to be upgrade to use the <rule> section.

    * Centos use 01-rewrite-0.12.conf.j2
    * Ubuntu use 01-rewrite-0.14.conf.j2

    backport: pike

    [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter/commit/248ed8e97def0a60121199c0d0e2fa8785f0d076
    [2] https://review.openstack.org/#/c/517907

    Depends-on: I47985113fe732569d640a262ca832c6edc8c2bb6
    Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
    Closes-Bug: #1730664

Changed in kolla-ansible:
status: In Progress → Fix Released
Changed in kolla:
milestone: none → queens-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/521433

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/ocata)

Reviewed: https://review.openstack.org/521337
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=7ba2d7a383775bb8224c070bccfc4ee4edade08f
Submitter: Zuul
Branch: stable/ocata

commit 7ba2d7a383775bb8224c070bccfc4ee4edade08f
Author: chenxing <email address hidden>
Date: Wed Nov 8 08:39:35 2017 +0000

    Fix the deprecated syntax in 01-rewrite.conf

    Before pike, kolla use upstream fluentd repo. and it is now bump to
    v0.14. So need update the rewite syntax.

    Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
    Closes-Bug: #1730664
    (cherry picked from commit 419a2fc9fd206447f37fbcf62da8e15ab96f8530)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (stable/ocata)

Reviewed: https://review.openstack.org/521336
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=9c76488694c8f7e3251a8ee45c928d0a7960f469
Submitter: Zuul
Branch: stable/ocata

commit 9c76488694c8f7e3251a8ee45c928d0a7960f469
Author: Jeffrey Zhang <email address hidden>
Date: Sun Nov 19 11:34:02 2017 +0800

    Pin fluent-plugin-grok-parser to 2.1.4

    We are using upstream fluentd, which has bump to 0.14.0. Need use high
    version of plugin.

    Change-Id: I8bef4e153621be9f2983f05a045e7ce0849483a1
    Closes-Bug: #1730664

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (stable/pike)

Reviewed: https://review.openstack.org/521045
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=ed669e22c830d7044c99891faf8ce9ae199f63b7
Submitter: Zuul
Branch: stable/pike

commit ed669e22c830d7044c99891faf8ce9ae199f63b7
Author: chenxing <email address hidden>
Date: Mon Nov 20 03:04:11 2017 +0000

    Fix the Ubuntu Fluentd image

    For ubuntu, we actually use fluentd 0.14.22 and ruby 2.1.10 now.
    Thus, we should use the specific gem version for the plugins:

     - fluent-plugin-rewrite-tag-filter:2.0.0
    According to the README.md of this plugin[1]
    It shows:
    ----------------------------------------------------------
    | fluent-plugin-rewrite-tag-filter | Fluentd | Ruby |
    |----------------------------------|------------|--------|
    | >= 2.0.0 | >= v0.14.2 | >= 2.1 |
    | < 2.0.0 | >= v0.12.0 | >= 1.9 |
    ----------------------------------------------------------

     - fluent-plugin-grok-parser:2.1.4
    When using the fluent-plugin-rewrite-tag-filter 2.0.0 to fix this bug, another
    bug[1] which is the same as this one[2] was triggered.
    According to https://github.com/fluent/fluent-plugin-grok-parser, it shows:
    ---------------------------------------------------
    | fluent-plugin-grok-parser | fluentd | Ruby |
    |---------------------------|------------|--------|
    | >= 1.0.0 | >= v0.14.0 | >= 2.1 |
    | < 1.0.0 | >= v0.12.0 | >= 1.9 |
    ---------------------------------------------------

    We actually use fluentd 0.14.22 now, but the version of fluent-plugin-grok-parser
    is still pinned in 0.3.1. So we need to upgrade the fluent-plugin-grok-parser to
    2.1.4.

    [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter
    [2] http://paste.openstack.org/show/626145/
    [3] https://github.com/fluent/fluent-plugin-grok-parser/issues/32

    Change-Id: I47985113fe732569d640a262ca832c6edc8c2bb6
    Partial-Bug: #1730664

tags: added: in-stable-pike
Chason Chan (chen-xing)
Changed in kolla:
status: New → Fix Released
importance: Undecided → High
assignee: nobody → Chason Chan (chen-xing)
Revision history for this message
Serge Radinovich (srad015) wrote :

This does not appear to have fixed the issue:

https://git.openstack.org/cgit/openstack/kolla/commit/?id=ed669e22c830d7044c99891faf8ce9ae199f63b7

I built stable/pike kolla and built images with -t source -b ubuntu.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/pike)

Reviewed: https://review.openstack.org/521433
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=444b0e2b5527951d6e3d95e11e0e375283e7bb7a
Submitter: Zuul
Branch: stable/pike

commit 444b0e2b5527951d6e3d95e11e0e375283e7bb7a
Author: chenxing <email address hidden>
Date: Mon Nov 20 06:33:36 2017 +0000

    Fix the deprecated syntax in 01-rewrite.conf

    The current syntax of Ubuntu fluentd rewrite rules are not supported
    anymore, reference to this commit[1].
    According to the build of this patch[2], The Centos has no such isuue.
    Only Ubuntu needs to be upgrade to use the <rule> section.

    * Centos use 01-rewrite-0.12.conf.j2
    * Ubuntu use 01-rewrite-0.14.conf.j2

    [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter/commit/248ed8e97def0a60121199c0d0e2fa8785f0d076
    [2] https://review.openstack.org/#/c/517907

    Depends-on: ed669e22c830d7044c99891faf8ce9ae199f63b7
    Change-Id: I0b131c2f65652cf3c61b33d8162097b047603923
    Closes-Bug: #1730664

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 6.0.0.0b2

This issue was fixed in the openstack/kolla-ansible 6.0.0.0b2 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla 4.0.3

This issue was fixed in the openstack/kolla 4.0.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 4.0.3

This issue was fixed in the openstack/kolla-ansible 4.0.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 5.0.1

This issue was fixed in the openstack/kolla-ansible 5.0.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.