use_syslog=True does not log to syslog via /dev/log anymore

Bug #1385295 reported by Bogdan Dobrelya
34
This bug affects 7 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
Medium
Unassigned
Juno
Fix Released
Medium
Pádraig Brady
oslo.log
Fix Released
High
John Stanford
nova (Ubuntu)
Invalid
Medium
Unassigned
python-oslo.log (Ubuntu)
Fix Released
High
Liang Chen

Bug Description

python-oslo.log SRU:
[Impact]

 * Nova services not able to write log to syslog

[Test Case]

 * 1. Set use_syslog to True in nova.conf/cinder.conf
   2. stop rsyslog service
   3. restart nova/cinder services
   4. restart rsyslog service
   5. Log is not written to syslog after rsyslog is brought up.

[Regression Potential]

 * none

Reproduced on:
https://github.com/openstack-dev/devstack 514c82030cf04da742d16582a23cc64962fdbda1
/opt/stack/keystone/keystone.egg-info/PKG-INFO:Version: 2015.1.dev95.g20173b1
/opt/stack/heat/heat.egg-info/PKG-INFO:Version: 2015.1.dev213.g8354c98
/opt/stack/glance/glance.egg-info/PKG-INFO:Version: 2015.1.dev88.g6bedcea
/opt/stack/cinder/cinder.egg-info/PKG-INFO:Version: 2015.1.dev110.gc105259

How to reproduce:
Set
 use_syslog=True
 syslog_log_facility=LOG_SYSLOG
for Openstack config files and restart processes inside their screens

Expected:
Openstack logs logged to syslog as well

Actual:
Nothing goes to syslog

Changed in oslo.log:
status: New → Confirmed
Revision history for this message
Ben Nemec (bnemec) wrote :

use_use=True <-- typo?

Changed in oslo.log:
status: Confirmed → Incomplete
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

yes, thanks.

description: updated
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Which info is missing, please elaborate?

Changed in oslo.log:
status: Incomplete → Confirmed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Thanks Dmitry Mescheryakov for help, here is the root cause: https://github.com/openstack/oslo-incubator/commit/1188d88a73e75da5c143d0dfad34e10dad0daee0
/dev/log was missed by that change

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Looks like /dev/log/ was removed by intentionally and this bug is invalid.
But I believe the commit 1188d88a73e75da5c143d0dfad34e10dad0daee0 with removed /dev/log should had had a DocImpact tag and the change should be documented anyway.
E.g. something like "/dev/log/ will not be used by Openstack services anymore due to portability reasons, please make sure syslog listens UDP:514 port in order to recieve logs from Openstack."

summary: - use_syslog=True does not log to syslog
+ use_syslog=True does not log to syslog via /dev/log anymore
Changed in openstack-manuals:
status: New → Confirmed
Revision history for this message
Anne Gentle (annegentle) wrote :

I think the best doc to place this in is the Cloud Administrator Guide.

Changed in openstack-manuals:
importance: Undecided → Medium
Revision history for this message
Daniel Gauthier (gaud) wrote :

I seem to have the same problem with cinder and nova . when those service have use_syslog set to True, nothing is sent to syslog even if rsyslog UDP:514 is accepting connection.

Revision history for this message
J-PMethot (jpmethot) wrote :

I understand that this "bug" is actually by design, for portability. However, it's not clear to me how openstack interacts with syslog instead of through /dev/log. Actually, from our tests, it seems to completely ignore the use_syslog=true now, just as the original report said.

I understand that this will be updated in the doc, but is there actually anything replacing the old way openstack used to interact with syslog? Or was there a conscious decision to drop syslog altogether?

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

I vote for return /dev/log and use another explicit ways to ensure portability as well. If ... else should do the job as well.
That 'drop' of /dev/log was not backwards compatible, for sure. Such changes first should go through a deprecation at least

Revision history for this message
Sergey Vasilenko (xenolog) wrote :

I vote for return /dev/log.

For portability will be better use if .. else statement.

Changed in oslo.log:
assignee: nobody → Roman Podoliaka (rpodolyaka)
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.log (master)

Fix proposed to branch: master
Review: https://review.openstack.org/133912

Changed in oslo.log:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/136842

Changed in oslo.log:
assignee: Roman Podoliaka (rpodolyaka) → Julien Danjou (jdanjou)
Revision history for this message
Tom Fifield (fifieldt) wrote :

Docs Triage: Looks like was a bug that is being fixed in oslo, so removing manuals

no longer affects: openstack-manuals
Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

This was resolved in the incubated version of the logging code by https://review.openstack.org/#/c/133638/

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

@Doug, could you please elaborate, should the patches be contributed both to the main and incubated versions of Oslo.log?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on oslo.log (master)

Change abandoned by Roman Podoliaka (<email address hidden>) on branch: master
Review: https://review.openstack.org/133912

Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

@Bogdan, at this point the logging code has fully graduated to oslo.log. Following our process for dealing with graduated libraries, changes should happen first in openstack/oslo.log and then be backported to relevant stable branches in openstack/oslo-incubator if needed.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.log (master)

Fix proposed to branch: master
Review: https://review.openstack.org/159606

Changed in oslo.log:
assignee: Julien Danjou (jdanjou) → John Stanford (e-john-6)
Revision history for this message
John Stanford (jxstanford) wrote :

I just resubmitted Roman's change since the related portability ticket is still a work in progress, and it seems reasonable to treat portability as a separate issue from a commit that broke a working, if not portable, feature.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.log (master)

Reviewed: https://review.openstack.org/159606
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=2bf8164a908b51c13d4709e63e7cb2905b5548c7
Submitter: Jenkins
Branch: master

commit 2bf8164a908b51c13d4709e63e7cb2905b5548c7
Author: John Stanford <email address hidden>
Date: Thu Feb 26 12:47:16 2015 -0800

    Make use_syslog=True log to syslog via /dev/log

    After upgrade from Icehouse to Juno OpenStack services don't log to
    syslog via /dev/log anymore. This behaviour was introduced in commit
    1188d88a73e75da5c143d0dfad34e10dad0daee0. While /dev/log is arguably
    not a very portable default, breaking the backwards compatibility is
    even worse.
    This commit restores the original behaviour. /dev/log portability
    issues will be addressed separately.
    Closes-Bug: #1385295
    Related-Bug: #1391428

    Change-Id: I5c450595fde7451a5bf9372430a53e75dd5773a3

Changed in oslo.log:
status: In Progress → Fix Committed
Changed in oslo.log:
milestone: none → 1.0.0
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/136842
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=33f5c6f94d94d5464fea38229d880f177b05aff7
Submitter: Jenkins
Branch: master

commit 33f5c6f94d94d5464fea38229d880f177b05aff7
Author: Julien Danjou <email address hidden>
Date: Mon Nov 24 18:35:36 2014 +0100

    Replace RFCSysLogHandler by a syslog() based one

    Rather than using the Python provided logging module that reimplents the
    whole syslog protocol and pointing it to /dev/log (which is not
    portable), we use the system calls from the syslog module, which are
    portable and default to the behaviour we actually want.

    Closes-Bug: 1385295
    Closes-Bug: 1391428

    Change-Id: I39a36316bd8778831b1bd5e7c5e2e12ede062a37

Liang Chen (cbjchen)
description: updated
Liang Chen (cbjchen)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.log (stable/kilo)

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/193633

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.log (stable/kilo)

Reviewed: https://review.openstack.org/193633
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=9a6c22663b57b7f2d9071767dadf28e59075a75e
Submitter: Jenkins
Branch: stable/kilo

commit 9a6c22663b57b7f2d9071767dadf28e59075a75e
Author: Julien Danjou <email address hidden>
Date: Mon Nov 24 18:35:36 2014 +0100

    Replace RFCSysLogHandler by a syslog() based one

    Rather than using the Python provided logging module that reimplents the
    whole syslog protocol and pointing it to /dev/log (which is not
    portable), we use the system calls from the syslog module, which are
    portable and default to the behaviour we actually want.

    Closes-Bug: 1385295
    Closes-Bug: 1391428

    Partial-Bug: 1459046

    (cherry picked from commit 33f5c6f94d94d5464fea38229d880f177b05aff7)

    Conflicts:
     oslo_log/log.py
     requirements.txt

    Change-Id: I2e47c896841cbd43d1f5504f2b8b897fc432cd0b

tags: added: in-stable-kilo
Liang Chen (cbjchen)
description: updated
Revision history for this message
Liang Chen (cbjchen) wrote :
Liang Chen (cbjchen)
Changed in python-oslo.log (Ubuntu):
assignee: nobody → Liang Chen (cbjchen)
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "vivid-kilo patch for python-oslo.log" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Revision history for this message
Pádraig Brady (p-draigbrady) wrote :

Fix proposed for nova Juno stable
https://review.openstack.org/202573

Changed in nova (Ubuntu):
status: New → In Progress
James Page (james-page)
Changed in cinder (Ubuntu):
importance: Undecided → Medium
Changed in nova (Ubuntu):
importance: Undecided → Medium
Changed in cinder (Ubuntu):
status: New → Invalid
Changed in nova (Ubuntu):
status: In Progress → Invalid
Changed in python-oslo.log (Ubuntu):
importance: Undecided → High
status: New → In Progress
Revision history for this message
Chris J Arges (arges) wrote :

For Ubuntu, Can you rebase this patch on the latest version of oslo.log? When done please re-subscribe sponsors.

Alan Pevec (apevec)
affects: nova (Ubuntu) → nova
Revision history for this message
George Shuklin (george-shuklin) wrote :

I think this patch is not merged to Juno https://review.openstack.org/#/c/202573/

Revision history for this message
George Shuklin (george-shuklin) wrote :

Sorry, My bad. I didn't saw merge message.

Mathew Hodson (mhodson)
affects: cinder (Ubuntu) → nova (Ubuntu)
Revision history for this message
Chuck Short (zulcss) wrote :

This should be fixed now.

Changed in python-oslo.log (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.log 1.0.0

This issue was fixed in the openstack/oslo.log 1.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.