Ceilometer

hbase should not use message_signature as unique id

Bug #1445227 reported by gordon chung
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Medium
ZhiQiang Fan
Ceilometer 5.0.0 "liberty"
Kilo
Fix Released
Medium
Unassigned
Ceilometer 2015.1.0 "kilo"

Bug Description

UPDATE: this is because hbase builds unique id from message_signature which is not guaranteed to be present/unique.

i found the bug!

this is the same as bug 1328550 but not it affects hbase tests. certain hbase tests will fail when

[publisher]
telemetry_secret=

is set in /etc/ceilometer/ceilometer.conf

======================================================================
FAIL: ceilometer.tests.storage.test_storage_scenarios.RawSampleTest.test_get_samples_by_resource(hbase)
tags: worker-1
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

Traceback (most recent call last):
  File "ceilometer/tests/base.py", line 97, in skip_if_not_implemented
    return func(*args, **kwargs)
  File "ceilometer/tests/storage/test_storage_scenarios.py", line 557, in test_get_samples_by_resource
    meter = results[1]
IndexError: list index out of range
======================================================================
FAIL: ceilometer.tests.storage.test_storage_scenarios.RawSampleTest.test_get_samples_by_start_time(hbase)
tags: worker-1
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

Traceback (most recent call last):
  File "ceilometer/tests/base.py", line 97, in skip_if_not_implemented
    return func(*args, **kwargs)
  File "ceilometer/tests/storage/test_storage_scenarios.py", line 590, in test_get_samples_by_start_time
    self.assertEqual(1, len(results))
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 1 != 0
======================================================================
FAIL: ceilometer.tests.storage.test_storage_scenarios.RawSampleTest.test_get_samples_by_user_limit_bigger(hbase)
tags: worker-3
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

Traceback (most recent call last):
  File "ceilometer/tests/base.py", line 97, in skip_if_not_implemented
    return func(*args, **kwargs)
  File "ceilometer/tests/storage/test_storage_scenarios.py", line 541, in test_get_samples_by_user_limit_bigger
    self.assertEqual(3, len(results))
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 3 != 1
======================================================================
FAIL: ceilometer.tests.storage.test_storage_scenarios.RawSampleTest.test_get_samples_by_end_time(hbase)
tags: worker-2
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

Traceback (most recent call last):
  File "ceilometer/tests/base.py", line 97, in skip_if_not_implemented
    return func(*args, **kwargs)
  File "ceilometer/tests/storage/test_storage_scenarios.py", line 610, in test_get_samples_by_end_time
    self.assertEqual(1, len(results))
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 1 != 0
======================================================================
FAIL: ceilometer.tests.storage.test_storage_scenarios.RawSampleTest.test_get_samples_by_user(hbase)
tags: worker-2
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

Traceback (most recent call last):
  File "ceilometer/tests/base.py", line 97, in skip_if_not_implemented
    return func(*args, **kwargs)
  File "ceilometer/tests/storage/test_storage_scenarios.py", line 526, in test_get_samples_by_user
    self.assertEqual(3, len(results))
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/opt/stack/ceilometer/.tox/py-mysql/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 3 != 1

See original description
Revision history for this message
ZhiQiang Fan (aji-zqfan) wrote :

hbase use (meter, timestamp, signature) as the unique key, now we enable set signature to '', meanwhile our test case use same timestamp for some samples, then test fails

I think hbase needs to choose another way to identify different samples, for example, use message id (which definitely exists)

mock the ceilometer.conf can avoid this, but may be prepare a specific env can be OK too
always use default value may lead to some ignored case such as this one

Changed in ceilometer:
assignee: nobody → ZhiQiang Fan (aji-zqfan)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/174767

Changed in ceilometer:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)

Reviewed: https://review.openstack.org/174767
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=15a92ec1ef2ab3e6d1d187ad909a0d5800ee4a2e
Submitter: Jenkins
Branch: master

commit 15a92ec1ef2ab3e6d1d187ad909a0d5800ee4a2e
Author: ZhiQiang Fan <email address hidden>
Date: Fri Apr 17 17:06:13 2015 +0800

    use message id to generate hbase unique key

    We currently can disable computing signature for metering message,
    but HBase still uses signature as a dimension to generate unique row
    key, this will lead to different samples treated as same one.

    Message id should be used in stead of message signature, because
    id will always exist and also unique.

    Change-Id: I47cdce59c9934573076cf609ce1a0c37aea75c44
    Closes-Bug: #1445227

Changed in ceilometer:
status: In Progress → Fix Committed
gordon chung (chungg)
summary: - tests are using live /etc/ceilometer/ceilometer.conf
+ hbase should not use message_signature as unique id
description: updated
tags: added: kilo-backport-potential
Eoghan Glynn (eglynn)
Changed in ceilometer:
milestone: none → liberty-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (stable/kilo)

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/175807

Thierry Carrez (ttx)
tags: added: kilo-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (stable/kilo)

Reviewed: https://review.openstack.org/175807
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=f5b994829bbd19794eba062ea79599f3b78a9286
Submitter: Jenkins
Branch: stable/kilo

commit f5b994829bbd19794eba062ea79599f3b78a9286
Author: ZhiQiang Fan <email address hidden>
Date: Fri Apr 17 17:06:13 2015 +0800

    use message id to generate hbase unique key

    We currently can disable computing signature for metering message,
    but HBase still uses signature as a dimension to generate unique row
    key, this will lead to different samples treated as same one.

    Message id should be used in stead of message signature, because
    id will always exist and also unique.

    Change-Id: I47cdce59c9934573076cf609ce1a0c37aea75c44
    Closes-Bug: #1445227
    (cherry picked from commit 15a92ec1ef2ab3e6d1d187ad909a0d5800ee4a2e)

Thierry Carrez (ttx)
tags: removed: kilo-backport-potential kilo-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/179290

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)
Download full text (3.5 KiB)

Reviewed: https://review.openstack.org/179290
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=b192f69f166317bbf048886f3a78d68fc89e64d1
Submitter: Jenkins
Branch: master

commit b0447ed8e7bee371bf7095c86e47d717abe89edc
Author: Chris Dent <email address hidden>
Date: Thu Apr 23 14:01:35 2015 +0000

    Have eventlet monkeypatch the time module

    Without this, mongod retry logic in the various services started as
    commands fails to behave as expected and does not reconnect as soon as
    the mongod service has returned to availability.

    In addition to the mongod sleep there are two other time.sleep calls
    that may be reached by this change. Review and discussion with others
    indicates that their behavior will continue to be correct with the
    monkeypatch in place.

    Cherry-pick from https://review.openstack.org/#/c/176751/

    Change-Id: I4eca290acc3b06658951f070935ebb39936e13d3
    Closes-Bug: #1447599

commit 2a0ecfa6e8246352376f463b3237017b529ca464
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Apr 23 02:12:27 2015 +0000

    Updated from global requirements

    Change-Id: Ia56c489de41738e318d00f04698a68a149bf3ac5

commit 2004c4cb563a8adb38b2498711ff8a24c2897e49
Author: Lan Qi song <email address hidden>
Date: Wed Apr 22 15:05:35 2015 +0800

    Fix valueerror when ceilometer-api start

    If change local language to other language(like ja), ceilometer-api
    will failed to start.

    This patch will remove i18 support from ceilometer/storage/__init__.py

    Change-Id: I8dc7ff0921d69e64a41e588207c97548513c99ed
    Closes-Bug: #1446983
    (cherry picked from commit f2ab05ac3dc95981ce183277a52e8c395a4cdbeb)

commit f5b994829bbd19794eba062ea79599f3b78a9286
Author: ZhiQiang Fan <email address hidden>
Date: Fri Apr 17 17:06:13 2015 +0800

    use message id to generate hbase unique key

    We currently can disable computing signature for metering message,
    but HBase still uses signature as a dimension to generate unique row
    key, this will lead to different samples treated as same one.

    Message id should be used in stead of message signature, because
    id will always exist and also unique.

    Change-Id: I47cdce59c9934573076cf609ce1a0c37aea75c44
    Closes-Bug: #1445227
    (cherry picked from commit 15a92ec1ef2ab3e6d1d187ad909a0d5800ee4a2e)

commit 446cd1440beda53e4216dc5da362fd74ded4f1df
Author: Mehdi Abaakouk <email address hidden>
Date: Wed Apr 15 14:31:24 2015 +0200

    gnocchi: fix typo in the aggregation endpoint

    Closes-bug: #1446201

    Change-Id: I92a727b518f77ad7149fc43a564adea5e8e97240
    (cherry picked from commit d1130e810d40ecd553a325fa5c32626be944625c)

commit dc6f4bfecbde54f663e96f485a948944223ef231
Author: Andreas Jaeger <email address hidden>
Date: Mon Apr 20 11:13:57 2015 +0200

    Release Import of Translations from Transifex

    Manual import of Translations from Transifex. This change also removes
    all po files that are less than 66 per cent translated since such
    partially translated files will not help users.

    This change needs to be don...

Read more...

Thierry Carrez (ttx)
Changed in ceilometer:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ceilometer:
milestone: liberty-1 → 5.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.