Ubuntu
zlib package

[Ubuntu 20.10] zlib: DFLTCC compression level switching issues

Bug #1893170 reported by bugproxy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Skipper Bug Screeners
zlib (Ubuntu)
Fix Released
Undecided
Michael Hudson-Doyle
Focal
Fix Released
Undecided
Michael Hudson-Doyle
Groovy
Fix Released
Undecided
Michael Hudson-Doyle

Bug Description

SRU Justification:
==================

[Impact]

 * This SRU fixes a combination of multiple issues:

 * zlib DFLTCC compression level switching can corrupt data because hardware and software compression states become desynchronized. (LP 1893170)

 * Since zlib is not working on all s390x system configurations support for switching between software and hardware compression is required especially for Java. (LP 1882494)

 * zlib on s390x may produce incomplete raw streams (but not gzip/zlib). (LP 1889059)

[Test Case]

 * Since especially DFLTCC requires a s390x generation z15 or LinuxONE III the tests need to be done by IBM.

 * IBM has a set of tests available, that exercise public zlib APIs by calling them in different sequences with different buffer sizes and flush modes.

* Partially these come from the IBM z/OS team, who developed their own zlib support for the hardware accelerator, and partially they are developed by the IBM Linux team based on issues encountered during development as well as fuzzing.

* IBM also uses the zlib-ng test-suite as well as squash and stress-ng.

 * Compress data using zlib/DFLTCC with different compression level and verify state and if data got corrupted or not. (LP 1893170)

 * On a system with patched zlib package, switch between hardware and software compression and use zlib via the java.util.zip package (http://java.sun.com/developer/technicalArticles/Programming/compression/). (LP 1882494)

* On a z15 system with hardware acceleration compression (DFLTCC) enabled, create a raw (negative windowBits value) stream with zlib. Then check if EOBS is missing or truncated. (LP 1889059)

[Regression Potential]

 * There is a certain risk for regressions with the modifications that are introduced by the four LP bugs.

 * In case the package fails entirely, it will have (in worst case) an impact on all zlib, gzip and DFLTCC compression/decompression functions, that are very wide spread (gzip, tar cfz, everything with zlib) in Linux and would virtually make the system unusable.

 * If potential issues are limited to hardware assisted compression (which is more likely, since these patches are mostly about hw assisted compression), then issues would be limited to systems that provide this feature (latest s390x generation only).

* A switch back to software got introduced (by setting DFLTCC=0 environment variable) that will help to mitigate a potential negative impact of the hw assisted compression.

 * Only the latest s390x generation (z15 and LinuxONE III) supports hardware assisted DFLTCC and is potentially affected.

 * A patched test package was made available and got successfully tested by IBM. All four bugs were finally considered as solved with zlib (1:1.2.11.dfsg-2ubuntu2~ppa2) available here: https://launchpad.net/~mwhudson/+archive/ubuntu/devirt/+packages?field.name_filter=zlib

__________

Description: zlib: DFLTCC compression level switching issues
Symptom: Switching compression levels corrupts data
Problem: Hardware and software compression states become desynchronized.
Solution: Improve compression state synchronization. Since zlib project
               does not accept patches at the moment, the fix has been
               integrated into the DFLTCC pull request:
               https://github.com/madler/zlib/pull/410
               The commitid is 992a7afc3edfa511dff0650d1c545b11bf64e655.
Reproduction: Not possible with popular command line tools. The issues were
               discovered using example_call_fuzzer from:
               https://github.com/iii-i/zlib-ng/tree/fuzz/test/fuzz/

This needs also be applied against 20.04 !

See original description
Revision history for this message
bugproxy (bugproxy) wrote : patch

Default Comment by Bridge

tags: added: architecture-s39064 bugnameltc-187859 severity-high targetmilestone-inin2010
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → zlib (Ubuntu)
Michael Hudson-Doyle (mwhudson)
Changed in zlib (Ubuntu Groovy):
assignee: Skipper Bug Screeners (skipper-screen-team) → Michael Hudson-Doyle (mwhudson)
Changed in zlib (Ubuntu Focal):
assignee: nobody → Michael Hudson-Doyle (mwhudson)
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
importance: Undecided → High
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
Francis Ginther (fginther)
tags: added: id-5f47e13db3cb5759fc5c5d2f
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Hi I've updated the patch to the latest version and uploaded the package to my PPA as version 1.2.11.dfsg-2ubuntu2~ppa2 which should appear here soon: https://launchpad.net/~mwhudson/+archive/ubuntu/devirt/+packages?field.name_filter=zlib. Can you test this on your hardware and let me know how it goes?

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → In Progress
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2020-09-07 07:12 EDT-------
zlib1g_1.2.11.dfsg-2ubuntu2~ppa2_s390x.deb passes all my tests.

Revision history for this message
Frank Heimes (fheimes) wrote :

Thanks for testing, Ilya.
With that we now have a patches zlib (1:1.2.11.dfsg-2ubuntu2~ppa2) package that was verified to fix not only this LP 1893170, but also LP 1884514, LP 1882494 and LP 1889059.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Great thanks for testing. I've uploaded to groovy -- th ext step is to SRU to focal I guess -- can you provide an impact statement / test case / regression potential as described at https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template ?

Frank Heimes (fheimes)
description: updated
Frank Heimes (fheimes)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package zlib - 1:1.2.11.dfsg-2ubuntu2

---------------
zlib (1:1.2.11.dfsg-2ubuntu2) groovy; urgency=medium

  * Update d/patches/410.patch to current state. LP: #1882494, #1889059, #1893170

 -- Michael Hudson-Doyle <email address hidden> Thu, 20 Aug 2020 11:52:59 +1200

Changed in zlib (Ubuntu Groovy):
status: New → Fix Released
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted zlib into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/zlib/1:1.2.11.dfsg-2ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in zlib (Ubuntu Focal):
status: New → Fix Committed
tags: added: verification-needed verification-needed-focal
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (zlib/1:1.2.11.dfsg-2ubuntu1.1)

All autopkgtests for the newly accepted zlib (1:1.2.11.dfsg-2ubuntu1.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

python3.8/3.8.2-1ubuntu1.2 (ppc64el)
cyrus-imapd/3.0.13-5 (s390x, amd64, ppc64el, arm64, armhf)
aspectc++/unknown (ppc64el)
apt/2.0.2ubuntu0.1 (armhf)
dovecot/1:2.3.7.2-1ubuntu3.2 (armhf)
hilive/2.0a-3build2 (ppc64el)
ariba/unknown (ppc64el)
cmake/3.16.3-1ubuntu1 (arm64)
bacula/unknown (ppc64el)
makedumpfile/1:1.6.7-1ubuntu2.1 (ppc64el)
apache2/2.4.41-4ubuntu3.1 (armhf)
pyxplot/unknown (ppc64el)
boost1.67/1.67.0-17ubuntu8 (amd64)
git-annex/unknown (ppc64el)
lighttpd/unknown (arm64)
gemma/0.98.1+dfsg-1 (armhf)
forked-daapd/26.4+dfsg1-1build3 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#zlib

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2020-09-28 07:33 EDT-------
http://launchpadlibrarian.net/498978117/zlib1g_1.2.11.dfsg-2ubuntu1.1_s390x.deb passes all my tests.

Revision history for this message
Frank Heimes (fheimes) wrote :

Thx for the verification - updating tags ...

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package zlib - 1:1.2.11.dfsg-2ubuntu1.1

---------------
zlib (1:1.2.11.dfsg-2ubuntu1.1) focal; urgency=medium

  * Update d/patches/410.patch to current state to fix issues with hardware
    accelerated zlib on new s390x machines. LP: #1893170

 -- Michael Hudson-Doyle <email address hidden> Thu, 20 Aug 2020 11:52:59 +1200

Changed in zlib (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for zlib has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Matthieu Clemenceau (mclemenceau)
tags: added: fr-590
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2020-10-20 08:50 EDT-------
IBM Bugzilla closed, Fix Released with all requested distros

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.