oscap with com.ubuntu.xenial.cve.oval.xml wrongly reports many unpatched (and unknown) non-installed packages on Ubuntu Xenial 16.04.1 LTS

Status changed to 'Confirmed' because the bug affects multiple users.

It's still the same issue, I downloaded OVAL definitions last week and also today.

I got thousands of unknown and unpatched packages, most are not even installed.

Tested with Redhat OVAL (on a Redhat system) and it works perfectly fine (I just wanted to verify that I used correct syntax).

Norbert, Johan, have either of you done enough investigation to know if the issue is the openscap that is in the archives or is the issue in the generated oval XML?

Thanks

Hi Seth,

I have not verified the XML or tried to compile from source. I tried to reach out to ubuntu-hardened email list before without any response.

I'm just trying to get some help where I can to have this issue resolved.

OVAL/CVE scanning within the Ubuntu community must be very very low, or everyone are using other tools.

// Johan

I have compiled from source and both the maintained package and direct from GitHub behaves the same.

Build steps:
 sudo apt-get install build-essential cmake libqt4-dev libxslt1-dev libcurl4-openssl-dev libz-dev autoconf libtool libpcre3-dev asciidoctor git checkinstall libgcrypt-dev pkg-config

 cd /tmp
 git clone https://github.com/OpenSCAP/openscap
 cd openscap
 git checkout maint-1.2
 ./autogen.sh
 ./configure --prefix /usr --disable-python --disable-util-oscap-docker
 make -j4
 sudo checkinstall make install # specify version 1.2.14

Cred: https://answers.launchpad.net/ubuntu/+source/openscap/+question/242354

I made test with Xenial amd64 again on system with all updates:

   cd /tmp
   wget https://people.canonical.com/~ubuntu-security/oval/com.ubuntu.xenial.cve.oval.xml
   sudo apt-get install libopenscap8
   oscap oval eval --results /tmp/results-xenial.xml --report /tmp/report-xenial.html /tmp/com.ubuntu.xenial.cve.oval.xml
   #...
   #Evaluation done.
   firefox /tmp/report-xenial.html

OVAL shema is 5.11.1, it reports 354 unpatched problems.

I made test with openscap 1.2.14 from Github on Xenial amd64.

   sudo apt-get purge libopenscap8
   cd /tmp
   sudo apt-get install build-essential cmake libqt4-dev libxslt1-dev libcurl4-openssl-dev libz-dev autoconf libtool libpcre3-dev asciidoctor git checkinstall libgcrypt-dev
   git clone https://github.com/OpenSCAP/openscap
   cd openscap
   git checkout maint-1.2
   ./autogen.sh
   ./configure --prefix /usr --disable-python --disable-util-oscap-docker
   make -j4
   sudo checkinstall make install # specified version 1.2.14
   oscap oval eval --results /tmp/results-xenial.xml --report /tmp/report-xenial.html /tmp/com.ubuntu.xenial.cve.oval.xml
   #...
   #Evaluation done.
   firefox /tmp/report-xenial.html

OVAL shema is 5.11.1, it reports 0 unpatched problems. It's great.
So we really need SRU for openscap package (see bug 1658529).
openscap-workbench is needed to be packaged too (see bug 1658492).

Norbert, the expected result is not to have zero unpatched CVEs. That's unrealistic and I'm afraid that openscap from github is giving you the wrong results. For instance, as of this writing, CVE-2015-5180 against glibc is unfixed so it should not be reporting zero unpatched CVEs.

Was this bug fixed with the oval_lib.py change, as mentioned in https://lists.ubuntu.com/archives/ubuntu-hardened/2017-July/000940.html, being committed?

With ocsap from github CVE-2015-5180 is marked Unknown.
The full statistics is:
  Non-Compliant/Vulnerable/Unpatched = 0,
  Compliant/Non-Vulnerable/Patched = 1988,
  Error = 0,
  Unknown = 6389,
  Other = 1.

With oscap from official repository:
  Non-Compliant/Vulnerable/Unpatched = 354 (11 high, 229 medium, 102 low, 12 negligible),
  Compliant/Non-Vulnerable/Patched = 6829,
  Error = 0,
  Unknown = 1194,
  Other = 1.
Here some CVEs contain references to Android, Qualcomm, aarch64, PuTTY (and WinSCP), but I do not understand this (I'm using amd64 laptop).
I have Wireshark installed, oscap reports CVEs in it. I removed it. Unpatched decreased to 264, Patched increased to 6919. So wireshark has 90 unpatched CVEs. But openscap from github does not change values.

I'll use oscap from official package libopenscap8.

As answer to your comment 10 I can say that CVEs 2012-2150, 2017-8386, 2014-8111 (https://lists.ubuntu.com/archives/ubuntu-hardened/2017-July/000940.html) are marked fixed in my results. I read this conversation and understood it.

So It seems that you are right.
Thank you, Tyler!
This bug is fixed.

On 07/10/2017 12:44 PM, Norbert wrote
> So It seems that you are right.
> Thank you, Tyler!
> This bug is fixed.

Great to hear! Jesus Linares deserves the credit for the bug fix.

Fixed in ubuntu-cve-tracker by http://bazaar.launchpad.net/~ubuntu-security/ubuntu-cve-tracker/master/revision/12856

Marking the openscap task as in Invalid since it was a problem in the OVAL content itself.