StackLight

Kibana displays incorrect timestamps when the cluster nodes are in another TZ than UTC

Bug #1633074 reported by Simon Pasquier
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StackLight
Fix Released
Undecided
Simon Pasquier
StackLight 1.0.0
0.10
Fix Released
Undecided
Simon Pasquier
StackLight 0.10.3

Bug Description

MOS 9 + StackLight plugins from master

When the operator configures the nodes with another TZ, Elasticsearch/Kibana assume UTC datetimes because the data pushed to Elasticsearch has no TZ information.

See original description
Simon Pasquier (simon-pasquier)
description: updated
Aleksei Shishkin (ashishkin)
tags: added: ct2 customer-found support
Revision history for this message
Simon Pasquier (simon-pasquier) wrote :

Workaround:

Override the timezone setting of Kibana and set it to 'UTC'. See https://drive.google.com/file/d/0B2Tm7Jz65CjVSGU4amdKTDFseDg/view?usp=sharing

Revision history for this message
Simon Pasquier (simon-pasquier) wrote :

Note that oslo.log has now a 'isotime' formatter that should probably be used instead of 'asctime'. Unfortunately it's only available starting from MOS 9.0 AFAICT.

Revision history for this message
Simon Pasquier (simon-pasquier) wrote :

In the same manner, the Syslog messages should be logged with the RFC3339 format (eg, '2016-11-04T15:16:39.967154+03:00') but unfortunately this isn't something which is of StackLight's responsibility.

Revision history for this message
Simon Pasquier (simon-pasquier) wrote :

For these reasons and also because other services (like MySQL, RabbitMQ, ...) don't use Syslog for logging, we have to let the Heka decoders know about the timezone to apply the correct offset. Also the Elasticsearch encoder will be updated to encode the timestamps following RFC3339.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-plugin-lma-collector (master)

Fix proposed to branch: master
Review: https://review.openstack.org/394311

Changed in lma-toolchain:
assignee: nobody → Simon Pasquier (simon-pasquier)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-plugin-lma-collector (master)

Reviewed: https://review.openstack.org/394311
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=737336a09cb978ad153562aab3d1347763e23042
Submitter: Jenkins
Branch: master

commit 737336a09cb978ad153562aab3d1347763e23042
Author: Simon Pasquier <email address hidden>
Date: Fri Nov 4 17:30:21 2016 +0100

    Enforce timezone setting in log processing

    Change-Id: I1fc5ecf8471c2effa1dadd72cf369c64bb11ec41
    Closes-Bug: #1633074

Changed in lma-toolchain:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-plugin-lma-collector (stable/0.10)

Fix proposed to branch: stable/0.10
Review: https://review.openstack.org/395669

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-plugin-lma-collector (stable/1.0)

Fix proposed to branch: stable/1.0
Review: https://review.openstack.org/395670

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-plugin-lma-collector (stable/1.0)

Reviewed: https://review.openstack.org/395670
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=b943f5b11d99a597a8f9cf2262921d88274d8346
Submitter: Jenkins
Branch: stable/1.0

commit b943f5b11d99a597a8f9cf2262921d88274d8346
Author: Simon Pasquier <email address hidden>
Date: Fri Nov 4 17:30:21 2016 +0100

    Enforce timezone setting in log processing

    Change-Id: I1fc5ecf8471c2effa1dadd72cf369c64bb11ec41
    Closes-Bug: #1633074
    (cherry picked from commit 737336a09cb978ad153562aab3d1347763e23042)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-plugin-lma-collector (stable/0.10)

Reviewed: https://review.openstack.org/395669
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=20c51d59912981c933d9c4ac05ed56b02c39811d
Submitter: Jenkins
Branch: stable/0.10

commit 20c51d59912981c933d9c4ac05ed56b02c39811d
Author: Simon Pasquier <email address hidden>
Date: Fri Nov 4 17:30:21 2016 +0100

    Enforce timezone setting in log processing

    Change-Id: I1fc5ecf8471c2effa1dadd72cf369c64bb11ec41
    Closes-Bug: #1633074
    (cherry picked from commit 737336a09cb978ad153562aab3d1347763e23042)

Revision history for this message
Vitaly Gusev (vgusev) wrote :

Verified on 0.10.3 version.

Simon Pasquier (simon-pasquier)
no longer affects: lma-toolchain/1.0
Changed in lma-toolchain:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.