[sru] sosreport does not obfuscate a mac address when --mask is used

Bug #1962733 reported by nikhil kshirsagar
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sosreport
Fix Released
Unknown
sosreport (Ubuntu)
Fix Released
Medium
Unassigned
Bionic
Fix Released
Medium
Unassigned
Focal
Fix Released
Medium
Unassigned
Impish
Won't Fix
Medium
Unassigned

Bug Description

[Impact]

sos 4.3 has a regression in mac address obfuscation. The file etc/netplan/50-cloud-init.yaml ends up with an unobfuscated mac address inspite of using --mask.

[TEST PLAN]

Documentation for Special Cases:
https://wiki.ubuntu.com/SosreportUpdates

[WHERE PROBLEMS COULD OCCUR]

Since we are changing the regex parser code in sos/cleaner/parsers/mac_parser.py we would need to ensure no other regex behavior is changed. The unit tests in autopkgtest will suffice to determine that.

-------------
[Other Info]

Upstream issue is https://github.com/sosreport/sos/issues/2873
Upstream MR is https://github.com/sosreport/sos/pull/2875

Reproducer details:
sos 4.2 shows correct behavior. testing shows..

/etc/netplan/50-cloud-init.yaml contains

network:
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: '52:54:00:12:34:56'
            set-name: ens3
    version: 2

4.2 sos contains the file but with the obfuscated mac address. correct behavior.

# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: '53:4f:53:cf:3a:9e'
            set-name: ens3
    version: 2

----------------------

4.3 testing shows the bug,

the /etc/netplan/50-cloud-init.yaml contains

# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: '52:54:00:12:34:56'
            set-name: ens3
    version: 2

---

generated sosreport (run with --mask) contains

# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: '52:54:00:12:34:56'
            set-name: ens3
    version: 2

So the mac address isnt being obfuscated.

Whats interesting is.. sosreport-host0-2022-03-02-kluxwcz-private_map stored in /tmp does indeed contain that entry and the obfuscated entry too..

{
    "hostname_map": {
        "autopkgtest": "host0"
    },
    "ip_map": {
        "10.0.2.0/24": "100.0.0.0/24",
        "10.0.2.15/24": "100.0.0.1/24",
        "10.0.2.255": "100.0.0.255",
        "10.0.2.3": "100.0.0.2/24",
        "91.189.89.199": "96.79.58.28",
        "5.4.0.102": "73.90.65.47",
        "5.4.0.100": "69.14.29.26",
        "5.4.0.26": "58.82.14.73",
        "224.0.0.1": "34.30.38.86",
        "91.189.89.198": "59.29.50.62",
        "3.192.30.10": "14.98.98.25",
        "5.4.0.99": "74.19.96.35",
        "10.0.2.0/28": "101.0.0.1/28",
        "10.0.2.0/30": "102.0.0.1/30",
        "192.168.200.1": "20.50.36.98",
        "192.168.200.4": "28.58.14.33",
        "192.168.200.9": "97.65.46.86",
        "192.168.201.0/24": "103.0.0.1/24",
        "192.168.201.0/25": "104.0.0.1/25",
        "224.0.0.251": "67.40.56.81",
        "239.255.255.250": "26.23.39.82",
        "123.45.67.89": "20.91.68.26",
        "192.168.0.133": "79.76.79.11"
    },
    "mac_map": {
        "52:54:00:12:34:56": "53:4f:53:79:76:69", <---
        "33:33:00:00:00:16": "53:4f:53:73:7a:0e",
        "33:33:00:00:00:02": "53:4f:53:5e:53:b8",
        "33:33:ff:12:34:56": "53:4f:53:eb:10:e9",
        "33:33:00:00:00:01": "53:4f:53:dc:de:c4",
        "01:00:5e:00:00:01": "53:4f:53:4d:05:93",
        "01:80:c2:00:00:00": "53:4f:53:b6:04:55",
        "01:80:c2:00:00:03": "53:4f:53:34:cb:ef",
        "01:80:c2:00:00:0e": "53:4f:53:f1:77:bc",
        "52:55:0a:00:02:02": "53:4f:53:08:59:72",
        "52:55:0a:00:02:03": "53:4f:53:97:23:da",
        "52:56:00:00:00:02": "53:4f:53:80:ba:7d"
    },
    "keyword_map": {},
    "username_map": {}
}

But somehow the file itself is collected without changing the ip address from original ("52:54:00:12:34:56") to the obfuscated one ("53:4f:53:79:76:69")

----

Tested with upstream sos:

root@autopkgtest:/home/nikhil/sos# git remote -v
origin https://github.com/sosreport/sos/ (fetch)
origin https://github.com/sosreport/sos/ (push)
root@autopkgtest:/home/nikhil/sos# ls
AUTHORS bin docs LICENSE man MANIFEST.in plugins_overview.py po README.md requirements.txt setup.py snap sos sos.conf sos.spec tests tmpfilesd-sos.conf
root@autopkgtest:/home/nikhil/sos#

root@autopkgtest:/home/nikhil/sos/bin# cat /etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: '52:54:00:12:34:56'
            set-name: ens3
    version: 2

Note it contains the mac addr 52:54:00:12:34:56

Now we will run the sos report, (upstream version)

oot@autopkgtest:/home/nikhil/sos/bin# ls
sos sos-collector sosreport
root@autopkgtest:/home/nikhil/sos/bin# pwd
/home/nikhil/sos/bin
root@autopkgtest:/home/nikhil/sos/bin# ./sos report --mask

sosreport (version 4.3)

This command will collect system configuration and diagnostic
information from this Ubuntu system.

For more information on Canonical visit:

        Community Website : https://www.ubuntu.com/
        Commercial Support : https://www.canonical.com

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.

Press ENTER to continue, or CTRL-C to quit.

Optionally, please enter the case id that you are generating this report for []:

 Setting up archive ...
 Setting up plugins ...
[plugin:firewall_tables] skipped command 'nft list ruleset': required kmods missing: nf_tables, nfnetlink. Use '--allow-system-changes' to enable collection.
[plugin:firewall_tables] skipped command 'iptables -vnxL': required kmods missing: nf_tables, iptable_filter.
[plugin:firewall_tables] skipped command 'ip6tables -vnxL': required kmods missing: ip6table_filter, nf_tables.
[plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec. Use '--allow-system-changes' to enable collection.
[plugin:networking] skipped command 'ss -peaonmi': required kmods missing: unix_diag, udp_diag, netlink_diag, af_packet_diag, inet_diag, xsk_diag, tcp_diag. Use '--allow-system-changes' to enable collection.
[plugin:ufw] skipped command 'ufw status numbered': required kmods missing: bpfilter, iptable_filter.
[plugin:ufw] skipped command 'ufw app list': required kmods missing: bpfilter, iptable_filter.
[plugin:wireless] skipped command 'iw list': required kmods missing: cfg80211.
[plugin:wireless] skipped command 'iw dev': required kmods missing: cfg80211.
[plugin:wireless] skipped command 'iwconfig': required kmods missing: cfg80211.
[plugin:wireless] skipped command 'iwlist scanning': required kmods missing: cfg80211.
 Running plugins. Please wait ...

  Finishing plugins [Running: systemd]
  Finished running plugins
Found 1 total reports to obfuscate, processing up to 4 concurrently

sosreport-autopkgtest-2022-03-02-abhwscl : Beginning obfuscation...
sosreport-autopkgtest-2022-03-02-abhwscl : Obfuscation completed [removed 16 unprocessable files]

Successfully obfuscated 1 report(s)

Creating compressed archive...

A mapping of obfuscated elements is available at
 /tmp/sosreport-host0-2022-03-02-abhwscl-private_map

Your sosreport has been generated and saved in:
 /tmp/sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz

 Size 2.27MiB
 Owner root
 sha256 e9d19933cfed512a59790edf65f70a0139f8da162f406153c298bb093bfbd939

Please send this file to your support representative.

Lets open the file and see if mac address in that file is left unobfuscated,

root@autopkgtest:/tmp# cat sosreport-host0-2022-03-02-abhwscl/etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: '52:54:00:12:34:56'
            set-name: ens3
    version: 2
root@autopkgtest:/tmp#

Note,

root@autopkgtest:/tmp# ls -lrt
total 9448
drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-systemd-resolved.service-7kMEUf
drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-systemd-timesyncd.service-FqCM6e
drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-systemd-logind.service-xFJpBh
drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-ModemManager.service-x5UZXh
-rwxr-xr-x 1 root root 691 Mar 2 15:48 eofcat
-rwxr-xr-x 1 root root 285 Mar 2 15:48 autopkgtest-reboot
-rwxr-xr-x 1 root root 269 Mar 2 15:48 autopkgtest-reboot-prepare
drwxrwxrwt 5 root root 4096 Mar 2 15:48 autopkgtest.RixDKr
drwx------ 10 root root 4096 Mar 2 15:48 sosreport-autopkgtest-2022-03-02-zwngejm
drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-fwupd.service-Zasqxf
drwx------ 3 root root 4096 Mar 2 15:48 systemd-private-e709306472c1435993a0b8d1f15e9dd3-upower.service-qb69Wg
-rw------- 1 root root 1645 Mar 2 15:49 sosreport-host0-2022-03-02-bwcteqj-private_map
-rw------- 1 root root 2389116 Mar 2 15:50 sosreport-host0-2022-03-02-bwcteqj-obfuscated.tar.xz
drwxr-xr-x 12 root root 4096 Mar 2 15:50 sosreport_test
drwx------ 12 root root 4096 Mar 2 15:55 sosreport-autopkgtest-2022-03-02-nwzytde
-rw------- 1 root root 2409380 Mar 2 15:55 sosreport-autopkgtest-2022-03-02-nwzytde.tar.xz
-rw-r--r-- 1 root root 65 Mar 2 15:55 sosreport-autopkgtest-2022-03-02-nwzytde.tar.xz.sha256
-rw------- 1 root root 2411848 Mar 2 15:58 sosreport-autopkgtest-2022-03-02-hkqkbak.tar.xz
-rw-r--r-- 1 root root 65 Mar 2 15:58 sosreport-autopkgtest-2022-03-02-hkqkbak.tar.xz.sha256
drwx------ 12 root root 4096 Mar 2 15:58 sosreport-host0-2022-03-02-abhwscl
-rw------- 1 root root 1645 Mar 2 15:59 sosreport-host0-2022-03-02-abhwscl-private_map <---
-rw------- 1 root root 2378324 Mar 2 15:59 sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz
-rw------- 1 root root 65 Mar 2 15:59 sosreport-host0-2022-03-02-abhwscl-obfuscated.tar.xz.sha256

root@autopkgtest:/tmp# cat sosreport-host0-2022-03-02-abhwscl-private_map
{
    "hostname_map": {
        "autopkgtest": "host0"
    },
    "ip_map": {
        "10.0.2.0/24": "100.0.0.0/24",
        "10.0.2.15/24": "100.0.0.1/24",
        "10.0.2.255": "100.0.0.255",
        "10.0.2.3": "100.0.0.2/24",
        "91.189.89.198": "33.43.50.21",
        "5.4.0.102": "80.74.90.96",
        "5.4.0.100": "69.87.15.65",
        "5.4.0.26": "13.16.68.51",
        "224.0.0.1": "92.20.91.63",
        "91.189.94.4": "42.38.68.46",
        "3.192.30.10": "93.87.22.28",
        "5.4.0.99": "37.44.72.50",
        "10.0.2.0/28": "101.0.0.1/28",
        "10.0.2.0/30": "102.0.0.1/30",
        "192.168.200.1": "37.72.13.85",
        "192.168.200.4": "19.35.86.99",
        "192.168.200.9": "39.80.73.13",
        "192.168.201.0/24": "103.0.0.1/24",
        "192.168.201.0/25": "104.0.0.1/25",
        "224.0.0.251": "19.45.84.66",
        "239.255.255.250": "93.52.70.42",
        "123.45.67.89": "92.20.45.84",
        "192.168.0.133": "29.81.60.51"
    },
    "mac_map": {
        "52:54:00:12:34:56": "53:4f:53:45:22:61", <----- never made it into the file collected
        "33:33:00:00:00:16": "53:4f:53:63:ca:e1",
        "33:33:00:00:00:02": "53:4f:53:46:bc:12",
        "33:33:ff:12:34:56": "53:4f:53:9a:2e:fa",
        "33:33:00:00:00:01": "53:4f:53:e6:29:f1",
        "01:00:5e:00:00:01": "53:4f:53:67:6f:e7",
        "01:80:c2:00:00:00": "53:4f:53:20:c4:71",
        "01:80:c2:00:00:03": "53:4f:53:fb:50:f9",
        "01:80:c2:00:00:0e": "53:4f:53:2d:57:8a",
        "52:55:0a:00:02:03": "53:4f:53:50:2b:72",
        "52:55:0a:00:02:02": "53:4f:53:64:ec:34",
        "52:56:00:00:00:02": "53:4f:53:3e:92:21"
    },
    "keyword_map": {},
    "username_map": {}

-------------

Changed in sosreport (Ubuntu Bionic):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Focal):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Impish):
importance: Undecided → Medium
Changed in sosreport (Ubuntu):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Bionic):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Focal):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport (Ubuntu Impish):
assignee: nobody → nikhil kshirsagar (nkshirsagar)
Changed in sosreport:
status: Unknown → New
summary: - sosreport does not obfuscate a mac address even with --mask is used
+ [sru] sosreport does not obfuscate a mac address even with --mask is
+ used
summary: - [sru] sosreport does not obfuscate a mac address even with --mask is
- used
+ [sru] sosreport does not obfuscate a mac address when --mask is used
Changed in sosreport (Ubuntu Impish):
milestone: none → impish-updates
Changed in sosreport (Ubuntu Focal):
milestone: none → focal-updates
Changed in sosreport (Ubuntu Bionic):
milestone: none → bionic-updates
Changed in sosreport (Ubuntu):
milestone: none → jammy-updates
description: updated
Changed in sosreport:
status: New → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

Ubuntu 21.10 (Impish Indri) has reached end of life, so this bug will not be fixed for that specific release.

Changed in sosreport (Ubuntu Impish):
status: New → Won't Fix
Revision history for this message
Arif Ali (arif-ali) wrote :

This is resolved in 4.4, and was released in bionic and focal, so solved this issue

Changed in sosreport (Ubuntu Focal):
status: New → Fix Released
Changed in sosreport (Ubuntu Bionic):
status: New → Fix Released
Changed in sosreport (Ubuntu):
status: New → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

We were scanning old bugs with assigned people.
Mostly to better manage expectations on bugs that have not yet been solved, but this one is good already \o/
Either way, removing the assignee as there is not much more coming e.g. on the "Impish Won't Fix" task

Changed in sosreport (Ubuntu):
assignee: nikhil kshirsagar (nkshirsagar) → nobody
Changed in sosreport (Ubuntu Bionic):
assignee: nikhil kshirsagar (nkshirsagar) → nobody
Changed in sosreport (Ubuntu Focal):
assignee: nikhil kshirsagar (nkshirsagar) → nobody
Changed in sosreport (Ubuntu Impish):
assignee: nikhil kshirsagar (nkshirsagar) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.