Landscape Client

swift reporting stopped working, scout.scout() changed result format

Bug #1563565 reported by Andreas Hasenack on 2016-03-29

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Landscape Client	Fix Released	High	Bogdana Vereha	Landscape Client 16.03
	landscape-client (Ubuntu)	Fix Released	High	Andreas Hasenack

Bug Description

This upstream commit (https://github.com/openstack/swift/commit/dd2f1be3b124d3901ebbc176a7adc462b6449667) changed the result of scout.scout():

from: tuple of (recon url used, response body, and status)
to: tuple of (recon url used, response body, status, time start and time end)

This breaks landscape-client's swiftusage.py:

def _perform_recon_call(self, host):
...
_, disk_usage, code = scout.scout(host) <----- backtraces

Here is a sample result from scout.scout():
('http://10.96.12.152:6000/recon/diskusage', [{u'device': u'sdb', u'avail': 437459259392, u'mounted': True, u'used': 62404411392, u'size': 499863670784}], 200, 1459286522.711885, 1459286522.716989)

I haven't investigated which version of python-swiftclient got this change, but right now it's in this one at least:
*** 2.5.0-0ubuntu1~cloud0 0
500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/liberty/main amd64 Packages

Tags:

Related branches

lp://staging/~bogdana/landscape-client/swift-reporting

Merged into lp://staging/~landscape/landscape-client/trunk at revision 840

Данило Шеган (community): Approve on 2016-04-05

Adam Collard (community): Approve on 2016-04-05

🤖 Landscape Builder (landscape-builder) on 2016-03-29

tags:

removed: kanban

Revision history for this message

Andreas Hasenack (ahasenack) wrote on 2016-03-29:

Something else that apparently changed. The text keys and values from the scout() result are now unicode, and that breaks the client elsewhere:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1201, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 797, in runUntilCurrent
    f(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 382, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/landscape/monitor/swiftusage.py", line 145, in _handle_usage
    timestamp, usage[key], persist_key)
  File "/usr/lib/python2.7/dist-packages/landscape/accumulate.py", line 81, in __call__
    previous_timestamp, accumulated_value = self._persist.get(key, (0, 0))
  File "/usr/lib/python2.7/dist-packages/landscape/lib/persist.py", line 385, in get
    return self.parent.get(self.root + path, default, soft, hard, weak)
exceptions.TypeError: can only concatenate tuple (not "unicode") to tuple

We only check for str in persist's get():
    def get(self, path, default=None, soft=False, hard=False, weak=False):
        if type(path) is str:
            path = path_string_to_tuple(path)
        return self.parent.get(self.root + path, default, soft, hard, weak)

Revision history for this message

Andreas Hasenack (ahasenack) wrote on 2016-03-29:

I applied this as a quick test, seems to be working, the swift graph is back:

=== modified file 'landscape/monitor/swiftusage.py'
--- landscape/monitor/swiftusage.py 2014-05-20 13:12:48 +0000
+++ landscape/monitor/swiftusage.py 2016-03-29 22:45:38 +0000
@@ -118,7 +118,9 @@

         scout = Scout("diskusage")
         # Perform the actual call
- _, disk_usage, code = scout.scout(host)
+ scout_result = scout.scout(host)
+ disk_usage = scout_result[1]
+ code = scout_result[2]
         if code == 200:
             return disk_usage

@@ -130,7 +132,7 @@
if not usage["mounted"]:
continue

- device = usage["device"]
+ device = usage["device"].encode("utf-8")
devices.add(device)

step_values = []

Adam Collard (adam-collard) on 2016-03-30

Changed in landscape-client:
status:	New → Confirmed
status:	Confirmed → Triaged

Adam Collard (adam-collard) on 2016-03-30

Changed in landscape-client:
status:	Triaged → In Progress
assignee:	nobody → Adam Collard (adam-collard)
milestone:	none → 16.03

Adam Collard (adam-collard) on 2016-03-30

Changed in landscape-client:
assignee:	Adam Collard (adam-collard) → Bogdana Vereha (bogdana)

🤖 Landscape Builder (landscape-builder) on 2016-04-04

Changed in landscape-client:
status:	In Progress → Fix Committed

Revision history for this message

Andreas Hasenack (ahasenack) wrote on 2016-04-04:

It's missing the utf-8 fix:
- device = usage["device"]
+ device = usage["device"].encode("utf-8")

Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1201, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 797, in runUntilCurrent
    f(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 382, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/landscape/monitor/swiftusage.py", line 144, in _handle_usage
    timestamp, usage[key], persist_key)
  File "/usr/lib/python2.7/dist-packages/landscape/accumulate.py", line 81, in __call__
    previous_timestamp, accumulated_value = self._persist.get(key, (0, 0))
  File "/usr/lib/python2.7/dist-packages/landscape/lib/persist.py", line 385, in get
    return self.parent.get(self.root + path, default, soft, hard, weak)
exceptions.TypeError: can only concatenate tuple (not "unicode") to tuple

Changed in landscape-client:
status:	Fix Committed → Confirmed

🤖 Landscape Builder (landscape-builder) on 2016-04-05

Changed in landscape-client:
status:	Confirmed → Fix Committed

Andreas Hasenack (ahasenack) on 2016-04-05

Changed in landscape-client (Ubuntu):
status:	New → In Progress
importance:	Undecided → High
assignee:	nobody → Andreas Hasenack (ahasenack)

Revision history for this message

Launchpad Janitor (janitor) wrote on 2016-04-05:

This bug was fixed in the package landscape-client - 16.03-0ubuntu2

---------------
landscape-client (16.03-0ubuntu2) xenial; urgency=medium

* Cope with an api change in python-swift that broke swift storage
reporting in Autopilot. (LP: #1563565)

-- Andreas Hasenack <email address hidden> Tue, 05 Apr 2016 09:15:40 -0300

Changed in landscape-client (Ubuntu):
status:	In Progress → Fix Released

Andreas Hasenack (ahasenack) on 2016-04-13

Changed in landscape-client:
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.