Ubuntu
sosreport package

[sru] bug in do_file_private_sub for the juju plugin

  1. Jammy (22.04)
  2. Bug #2024547
Bug #2024547 reported by nikhil kshirsagar
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sosreport (Ubuntu)
Status tracked in Mantic
Focal
Fix Committed
Medium
nikhil kshirsagar
Jammy
Fix Committed
Medium
nikhil kshirsagar
Kinetic
Won't Fix
Medium
nikhil kshirsagar
Lunar
Fix Committed
Medium
nikhil kshirsagar
Mantic
Fix Released
Medium
nikhil kshirsagar

Bug Description

[IMPACT]

As described in https://github.com/sosreport/sos/issues/3261 , the flags=re.IGNORECASE argument cannot be passed with a compiled pattern. This affects sos upstream 4.5.4 and 4.5.3.

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sos/report/__init__.py", line 1459, in postproc
    plug.postproc()
  File "/usr/lib/python3/dist-packages/sos/report/plugins/juju.py", line 69, in postproc
    self.do_file_private_sub(agents_path)
  File "/usr/lib/python3/dist-packages/sos/report/plugins/__init__.py", line 1257, in do_file_private_sub
    self.do_file_sub(path, _certmatch, replace)
  File "/usr/lib/python3/dist-packages/sos/report/plugins/__init__.py", line 1287, in do_file_sub
    result, replacements = re.subn(regexp, subst, content,
  File "/usr/lib/python3.8/re.py", line 221, in subn
    return _compile(pattern, flags).subn(repl, string, count)
  File "/usr/lib/python3.8/re.py", line 299, in _compile
    raise ValueError(
ValueError: cannot process flags argument with a compiled pattern

The issue was introduced by this commit - https://github.com/sosreport/sos/commit/bb43ac3b0dddcecf9bf34e489490296e15a2addc

It is fixed by https://github.com/sosreport/sos/pull/3263

The patch is taken for the 4.5.4 sos release in https://bugs.launchpad.net/ubuntu/+source/sosreport/+bug/2022915, as d/p/0002-regex-flags.patch, pasted below for reference,

Description: Fix do_file_private_sub() bug related to regex flags
Origin: upstream
Bug: https://github.com/sosreport/sos/issues/3261
Applied-Upstream: https://github.com/sosreport/sos/pull/3263
---
This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
Index: sos-4.5.4/sos/report/plugins/__init__.py
===================================================================
--- sos-4.5.4.orig/sos/report/plugins/__init__.py
+++ sos-4.5.4/sos/report/plugins/__init__.py
@@ -1274,7 +1274,12 @@ class Plugin():
         """
         try:
             path = self._get_dest_for_srcpath(srcpath)
- pattern = regexp.pattern if hasattr(regexp, "pattern") else regexp
+ if hasattr(regexp, "pattern"):
+ pattern = regexp.pattern
+ flags = regexp.flags | re.IGNORECASE
+ else:
+ pattern = regexp
+ flags = re.IGNORECASE
  self._log_debug("substituting scrpath '%s'" % srcpath)
  self._log_debug("substituting '%s' for '%s' in '%s'"
                             % (subst, pattern, path))
@@ -1284,8 +1289,8 @@ class Plugin():
  content = readable.read()
  if not isinstance(content, str):
      content = content.decode('utf8', 'ignore')
- result, replacements = re.subn(regexp, subst, content,
- flags=re.IGNORECASE)
+ result, replacements = re.subn(pattern, subst, content,
+ flags=flags)
  if replacements:
      self.archive.add_string(result, srcpath)
  else:

[TEST PLAN]

The 4.5.4 ppa's built with this additional fix are tested as per the process described https://wiki.ubuntu.com/SosreportUpdates and verfied to have fixed the issue.

[WHERE PROBLEMS COULD OCCUR]

As this bug is in a method invoked by multiple plugins, verification also would need to performed in environments that ensures those plugins are enabled, particularly on other distributions. In our testing, we have found no issues, but there might be some disabled plugins that use that method with assumptions that are made invalid by the patch. However we have found no evidence so far of such occurences.

See original description
Revision history for this message
nikhil kshirsagar (nkshirsagar) wrote :

Fixed upstream, and also built on top of the upstraem 4.5.4 sos release for ubuntu 4.5.4 release - https://bugs.launchpad.net/ubuntu/+source/sosreport/+bug/2022915

description: updated
description: updated
Changed in sosreport (Ubuntu):
status: New → Fix Committed
assignee: nobody → nikhil kshirsagar (nkshirsagar)
assignee: nikhil kshirsagar (nkshirsagar) → nobody
Dariusz Gadomski (dgadomski)
Changed in sosreport (Ubuntu Mantic):
status: Fix Committed → Fix Released
importance: Undecided → Medium
Changed in sosreport (Ubuntu Lunar):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Kinetic):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Jammy):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Focal):
importance: Undecided → Medium
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Jammy):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Kinetic):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Lunar):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Mantic):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
nikhil kshirsagar (nkshirsagar)
summary: - bug in do_file_private_sub for the juju plugin
+ [sru] bug in do_file_private_sub for the juju plugin
nikhil kshirsagar (nkshirsagar)
description: updated
nikhil kshirsagar (nkshirsagar)
description: updated
Revision history for this message
David Negreira (dnegreira) wrote :

I have verified that the issue does not happen on 4.5.4 PPA release [0] and the issue is fixed.

Before the fix we would see an extra file from with the juju error on sosreport under sos_logs/ folder, as well as an ERROR on sos_logs/sos.log

[0] https://launchpad.net/~nkshirsagar/+archive/ubuntu/focal-sos-4.5.4-sru
### Output ###
ls */sos_logs/
sosreport-juju-43d0aa-f-sos-t-0-2023-06-24-oikuwgu/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-1-2023-06-24-gitiaxq/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-10-2023-06-24-vksjbrs/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-11-2023-06-24-miqwnah/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-12-2023-06-24-ildlhxd/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-13-2023-06-24-cxrvvvz/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-14-2023-06-24-nqxieks/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-15-2023-06-24-bihuyia/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-16-2023-06-24-yhsnwir/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-17-2023-06-24-cvothqw/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-2-2023-06-24-waeusgt/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-3-2023-06-24-phxflms/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-4-2023-06-24-kfclnxe/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-5-2023-06-24-hmkekbt/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-6-2023-06-24-cwcjkqk/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-7-2023-06-24-jnrzyux/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-8-2023-06-24-khhfxui/sos_logs/:
sos.log ui.log

sosreport-juju-43d0aa-f-sos-t-9-2023-06-24-vqepwsx/sos_logs/:
sos.log ui.log
---
ubuntu@dnegreira-bastion:~/sosreportesting/proposedfiles/focal$ grep -r ERR */sos_logs/ | wc -l
0

####

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello nikhil, or anyone else affected,

Accepted sosreport into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.5.4ubuntu0.23.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Lunar):
status: New → Fix Committed
tags: added: verification-needed verification-needed-lunar
Changed in sosreport (Ubuntu Kinetic):
status: New → Fix Committed
tags: added: verification-needed-kinetic
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello nikhil, or anyone else affected,

Accepted sosreport into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.5.4ubuntu0.22.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Jammy):
status: New → Fix Committed
tags: added: verification-needed-jammy
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello nikhil, or anyone else affected,

Accepted sosreport into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.5.4ubuntu0.22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Focal):
status: New → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello nikhil, or anyone else affected,

Accepted sosreport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.5.4ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (sosreport/4.5.4ubuntu0.20.04.1)

All autopkgtests for the newly accepted sosreport (4.5.4ubuntu0.20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.5.4ubuntu0.20.04.1 (amd64, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Proposed package removed from archive

The version of sosreport in the proposed pocket of Kinetic that was purported to fix this bug report has been removed because one or more bugs that were to be fixed by the upload have failed verification and been in this state for more than 10 days.

tags: removed: verification-needed-kinetic
Changed in sosreport (Ubuntu Kinetic):
status: Fix Committed → Confirmed
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

This is being superseeded by 4.5.6, which does not mention this bug in its changes file. I guess that's ok, since the bug was only present in 4.5.3 and 4.5.4, and neither was released to ubuntu.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

4.5.6 was released to L, J, F.

Changed in sosreport (Ubuntu Kinetic):
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.