Landscape client never completes tasks

Bug #1813442 reported by Henry Baldursson
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Landscape Client
Fix Committed
Juanmi Taboada

Bug Description

It appears to run scripts I run, but package updates don't work

Here's example of errors I see in logs:

==> package-changer.log <==
2019-01-27 02:57:52,028 ERROR [MainThread] None
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 724: invalid start byte

==> package-reporter.log <==
2019-01-27 02:57:53,229 ERROR [MainThread] None
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/twisted/internet/", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python3/dist-packages/landscape/client/package/", line 92, in <lambda>
    result.addCallback(lambda x: self.request_unknown_hashes())
  File "/usr/lib/python3/dist-packages/landscape/client/package/", line 485, in request_unknown_hashes
  File "/usr/lib/python3/dist-packages/landscape/lib/apt/package/", line 265, in ensure_channels_reloaded
  File "/usr/lib/python3/dist-packages/landscape/lib/apt/package/", line 253, in reload_channels
    version, with_info=False).get_hash()
  File "/usr/lib/python3/dist-packages/landscape/lib/apt/package/", line 402, in get_package_skeleton
    return build_skeleton_apt(pkg, with_info=with_info, with_unicode=True)
  File "/usr/lib/python3/dist-packages/landscape/lib/apt/package/", line 131, in build_skeleton_apt
    version.record, "Provides", DEB_PROVIDES))
  File "/usr/lib/python3/dist-packages/apt/", line 690, in record
    return Record(self._records.record)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 724: invalid start byte

Revision history for this message
Henry Baldursson (henrythor) wrote :

Hello? Is this thing on? My organization pays for this service. I assumed I might actually get an answer on account of that. Is Landscape abandoned?

Revision history for this message
Uli Köhler (ulikoehler) wrote :

Hi, we've encountered this issue as well. Our issue was with a misplaced U+FFFD � REPLACEMENT CHARACTER in the veeam packages file.

I've published a blogpost on how to patch the issue in the Python code, and we've reported the issue to veeam, but I'd like to see an offical fix anyway.

Changed in landscape-client:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Mitch Burton (mitchburton)
Revision history for this message
Juanmi Taboada (juanmitaboada) wrote :

I made a test to reproduce the bug:

    test_build_skeleton_broken_description ... [ERROR]

Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/", line 59, in testPartExecutor
  File "/usr/lib/python3.10/unittest/", line 591, in run
  File "/usr/lib/python3.10/unittest/", line 549, in _callTestMethod
  File ".../landscape-client/landscape/lib/apt/package/tests/", line 340, in test_build_skeleton_broken_description
    skeleton = build_skeleton_apt(brokendescription_package)
  File ".../landscape-client/landscape/lib/apt/package/", line 136, in build_skeleton_apt
    parse_record_field(version.record, "Provides", DEB_PROVIDES),
  File "/usr/lib/python3/dist-packages/apt/", line 671, in record
    return Record(self._records.record)
builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 427: invalid start byte

Ran 1 tests in 0.223s

FAILED (errors=1)

This is NOT a BUG in Landscape itself. I believe it is a bug in "dpkg". It allowed me to build a new deb package using the wrong encoding inside the CONTROL file.

Checking the documentation, it says clearly: "All control files must be encoded in UTF-8"

The problem reported here refers to a package built using the wrong encoding for the CONTROL file, so I don't think we should fix anything in Landscape for this bug.

I made a test just in case the Landscape team decides to manage this situation differently.

I created a PR with this test:

Revision history for this message
Mitch Burton (mitchburton) wrote :

I agree with Juanmi's description as this package name being a violation of the encoding standards for debian packages - however we should probably still handle this case in a manner that allows for package upgrade to succeed. Sometimes we are beholden to the strange behaviours of others.

Revision history for this message
Juanmi Taboada (juanmitaboada) wrote (last edit ):

I think that from an error, another unpredicted error can be derived.

I would first notify landscape-server to let the administrator decide about this scenario, take responsibility for it and give the admin some choices:
- Bypass the package
- Try to install at any cost (we would take the description and, in general, many of the fields with a default value if we can not read them from the package)
- Mark the package not to upgrade
- Uninstall this package to avoid this situation in the future
- Some other?

As a first version, we can bypass the broken package, make landscape-client keep going, and notify in the logs.
This would be a temporal solution not to prevent the system from upgrading until a final scenario is decided.

Revision history for this message
Mitch Burton (mitchburton) wrote :

I'm fine with bypass-style temporary solution (though those have a habit of becoming rather permanent).

Revision history for this message
Juanmi Taboada (juanmitaboada) wrote :

I got deeper into this problem and found that it was happening because we were using "version.record" which is the source of the failure.

Since we don't want to mess up with the UnicodeDecodeError still happening in the apt package, I decided to switch the way the dependencies information is readen from the package, and I removed "version.record" from the code so it doesn't try to parse the broken data anymore.

Also, I discovered in the original source code that "PreDepends" was empty in all my tests. Now it has information as I think it should work. I would like somebody to check my solution in search of some collateral problems with the new "PreDepends".

The PR that resolves this BUG:

Thank you,

Changed in landscape-client:
assignee: Mitch Burton (mitchburton) → Juanmi Taboada (juanmitaboada)
status: Confirmed → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.