[MIR] sane-airscan

"Ubuntu Desktop Bugs" (~desktop-bugs) is now also subscribed to sane-airscan.

"Desktop Packages" (~desktop-package) is now also subscribed to sane-airscan.

[Summary]
- What is going to happen to sane-escl? Can we remove it from sans-backends, not shipping if or separate it to demote it to universe?
- Can we remerge the version against latest upstream? (We lag one version behind compared to debian testing).
- Related to above: can we resync with debian using an alternative build-dep? (debian-build-dep | ubuntu-build-dep)?
- Will this package be brought as a recommends of another package or seeded directly?
- Once all those questions are answered, happy to forward it to the security team for a security review.

[Duplication]
Nothing to add over the top request if sane-escl can be removed from main.

[Dependencies]
OK:
- nothing outside main

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking

[Security]
OK:
- no CVEs, but really fresh new package.
- does not use webkit2,2
- does not use lib*v9 directly
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not open a port directly (but will communicate through opened port via sane and zeroconf subcription)
- does not run a daemon as root

Problems:
- does parse a lot of data in different formats (xml, via http…): needing a security review

[Common blockers]
OK:
- does not FTBFS currently
- does have a test suite that runs at build time (good size)
  - test suite fails will fail the build upon error.
- no translation on CLI tool (but this is only a debugging discover command, common to not have them here). Messages returned to Sane are translated though.
- not a python package, no extra constraints to consider int hat regard
- C package that uses dh standard tool
- Team subscription is OK

[Packaging red flags]
OK:
- Ubuntu does not carry a delta
- symbols tracking not applicable for this kind of code.
- d/watch is present and looks ok
- Upstream update history is good
- Debian/Ubuntu update history is good, but short
- upstream do regular releasse (upstream tarball are created via CI)
- promoting this does not seem to cause issues for MOTUs that so far maintained the package
- no lintian issue
- d/rules is clean and minimal

[Upstream red flags]
OK:
- no Errors/warnings during the build
- no use of sudo, gksu, pkexec, or LD_LIBRARY_PATH
- no use of user nobody
- no use of setuid
- no important open bugs (crashers, etc) in Debian or Ubuntu
- no dependency on webkit, qtwebkit, seed or libgoa-*
- no embedded source copies
- no upstream bug opened at this date (51 bugs opened closed, so good upstream hygiene)

- What is going to happen to sane-escl? Can we remove it from sans-backends, not shipping if or separate it to demote it to universe?

There are no special ./configure options to suppress a backend in sane-backends, one could build without libcurl dependency, as sane-escl needs it, but there is the risk that in later sane-backends versions libcurl is needed by something else. Or libcurl can be present on the build server due to another dependency.

So perhaps the best is either

  o to install sane-escl into a separate package which is not seeded or
    pulled in by another package's dependency
  o simply to comment out "escl" in /etc/sane.d/dll.conf

- Can we remerge the version against latest upstream? (We lag one version behind compared to debian testing).

Yes, I have skipped the 0.99.13 as it did not work with my scanner. I will update the package to one of the later versions.

- Related to above: can we resync with debian using an alternative build-dep? (debian-build-dep | ubuntu-build-dep)?

I will ask the Debian guys

- Will this package be brought as a recommends of another package or seeded directly?

Could be recommended from sane-backends.

> So perhaps the best is either
> o to install sane-escl into a separate package which is not seeded or
> pulled in by another package's dependency
> o simply to comment out "escl" in /etc/sane.d/dll.conf

I would go with option 1, do you mind organizing that?

The rest looks good to me, let’s subcribe the security team now, thanks!

I will prepare the sane-backends package appropriately:

1. Spin out sane-escl into its own binary package

2. Recommend sane-airscan

One remark:

sane-escl is actually not developed any more due to sane-airscan, so I am looking into whether there is an easy and maintainable way to (3) remove it altogether? WDYT?

See also

https://openprinting.github.io/OpenPrinting-News-April-2020/#driverless-scanning

Syncable sane-airscan 0.99.15-1 is uploaded to Debian.

sane-airscan 0.99.15-1 has arrived in Debian, I have synced it now to Ubuntu.

I have prepared a sane-backends package to recommend sane-airscan and remove sane-escl:

----------
sane-backends (1.0.30-1~experimental2ubuntu2) groovy; urgency=medium

  * Suppress building the "escl" backend, we will use the more
    versatile "airscan" (in separate package) instead
  * Recommend the sane-airscan package (for AirPrint/eSCL/WSD scanners
    and in the future also for IPP Scan).

 -- Till Kamppeter <email address hidden> Tue, 25 Aug 2020 22:57:00 +0200
----------

When should I upload it? Has it to be before Feature Freeze? Or only when this MIR got completely accepted?

Hi Till,

The escl backend is not abandoned!
It is being maintained and will continue to be maintained as long as sane-airscan does not offer the full functionality of the escl backend.
Support for the escl backend is complete, ADF, Brightness, Threshold, Sharpen and Contrast.
Some devices require PDF to work.
Unix socket support is present.

sane-backends is in version 1.0.31

Thierry, thanks for the update. That is good news and so we know that the eSCL protocol is well investigated by you and Alex and so we can improve the support for it more and more. So I hope also that Alex will take you newest additions as base to complete the eSCL part of sane-airscan, to make it a universal backend making all AirScan scanners fully working. AFAIR sane-airscan does not have all of Brightness, Threshold, Sharpen, and Contrast yet.
For the devices which require PDF to work, do they work with current sane-escl and/or sane-airscan? For Scanner Applications we will be able to convert forth and back between PDF and raster formats as I have made library functions out of all CUPS filters.

Currently only sane-escl supports PDF. My PDF solution is not perfect (I use Poppler).
I resumed development when I realized that sane-airscan would not merge with Sane.
As a reminder, Sane is a free project, you just have to propose a pull-request to integrate the project.

sane-airscan can also work around bug 1897962, but independent of this we should also backport the fix in the "escl" backend.

@Till: Interesting info. Do you think it would be possible to backport sane-airscan? -backports is more useful for new packages than it is for already existing ones, since the low priority doesn't matter if there is nothing in -release or -updates.

My comment was intended for bug #1897962. Please disregard it in this MIR context.

@gunnarhj,

My own builds for sane-airscan exist for Ubuntu starting from the version 16.04, so I think that technically "backporting" is a trivial task.

Regarding PDF, I purposely trade it off in a favor of lower memory footprint. PDF decompression requires allocation of а full-size buffer for the decompressed image (~1/2Gb for Color A4 at 1200 DPI!), which I'm trying to avoid.

I will reconsider my opinion regarding PDF support, if there are really some devices in existence that cannot scan without it. But so far I'm not convinced that such devices exist.

Status changed to 'Confirmed' because the bug affects multiple users.

For what its' worth -- r.e. groovy release..

libsane1 (1.0.31-2) seems to include libsane-escl listed normally in dll.conf (i.e. likely a reasonable version of escl, better than focal known-buggy version).

sane-airscan 0.99.15-1 is in gusty universe, so will require an SRU in order to be changed to 0.99.18 etc. Not sure how much difference that makes [maybe maintainer can comment!].

Not clear to me that installing sane-airscan package will 'just work' to take-over from escl when installed, by default (or if manual modding dll.conf needed), however.

Maybe, the way this works, is such that users will see scanner listed twice, both driver options offered, or maybe airscan should take over escl if installed....

The two backends are not in conflict.
sane will list an additional backend.

Before :
PIXMA TS-3100 SERIES
ESCL TS-3100 SERIES

After :
PIXMA TS-3100 SERIES
ESCL TS-3100 SERIES
AIRSCAN TS-3100 SERIES

Without certitiude on the order.

Fwiw, this is likely to be such a common usability problem, LinuxMint decided to provide more useful sane-airscan version directly in their repositories for mint20.1 based open focal20.04.
https://github.com/linuxmint/linuxmint/issues/327

Since, Focal 20.04LTS does not have 'sane-airscan' package I think ubuntu should look at providing/backporting this more easily for focal users -- that is, not requiring a convoluted hidden process requiring manual search for bug-reports/PPAs, with no clue to the user this may be needed, and so-on.

Any "Driver-manager" etc could assist with the sane-airscan adding / sane-backends+escl-backport options.

I reviewed sane-airscan 0.99.24-1 as checked into hirsute. This shouldn't be
considered a full audit but rather a quick gauge of maintainability.

sane-airscan is a package that provides a sane backend which supports
driverless network-connected scanners.

- CVE History:
  - No CVE history, this is a relatively new package that was only first
    released in December 2019 and only exists in Ubuntu since groovy.
- Build-Depends
  - gnutls, libavahi, libxml2 and libjpeg-turbo are all depedencies
- No pre/post inst/rm scripts
- No init scripts
- No systemd units
- No dbus services
- No setuid binaries
- 1 binary in PATH
  - -rwxr-xr-x root/root 175312 2021-02-08 01:49 ./usr/bin/airscan-discover
- No sudo fragments
- No polkit files
- No udev rules
- unit tests / autopkgtests
  - unit tests are compiled in the build log but do not appear to be run
    during the build - these appear to just do simple testing of image
    decoding, URI parsing and HTTP multipart decoding. There are also tests
    for the zeroconf handling and these appear to be more extensive. These
    should be executed during the build itself.
  - there are no autopkgtests
- No cron jobs
- Build logs are quite clean

- No processes spawned
- Memory management
  - sane-airscan is all in C so there is a fair amount of dynamic memory
    management - this all appears to be done quite carefully, Coverity only
    found a couple issues (detailed below) but in general this is quite
    good quality code.
- File IO
  - Loads inifile from /etc/sane.d or from path specified in environment
    variable CONFIG_PATH_ENV - this appears to be quite defensive and safe
- Extensive logging but this all appears safe
- Environment variable usage:
  - CONFIG_PATH_ENV - as noted above for the path to the configuration directory
  - HOME - use of this is safely, checking length etc before use - this is
    used to be able to expand ~ in configuration settings to $HOME etc
  - CONFIG_ENV_AIRSCAN_DEBUG - this is used to be able to turn on debug
    logging
- No use of privileged functions
- Use of cryptography / random number sources
  - Uses /dev/urandom to generate random value for retransmit timer - this
    is not security sensitive but /dev/urandom is a good choice since this
    is non-blocking
  - Uses gnutls for SHA256 hash and TLS for HTTPS - but doesn't perform any
    certificate verification of the remote host - but this is expected
    since scanners on a local network are not going to have FQDN nor
    appropriately issued certificates (ie they are likely to just be
    self-signed)
- No use of temp files
- Use of networking is generally defensive but relies on dependent
  libraries to do parsing of image formats etc so this risk is delegated
- No use of WebKit
- No use of PolicyKit

- No significant cppcheck results
- A couple significant Coverity results which should be relatively easy to
  fix - I have reported these upstream at
  https://github.com/alexpevzner/sane-airscan/issues/131 but the two most
  important ones are:

-----
airscan-http.c:1275
  Type: Uninitialized pointer read (UNINIT)

airscan-http.c:1261:
  1. var_decl: Declaring variable "parser" without initializer.
air...

Alexander P,

for sane-aiscane getting promoted into the Main part of Ubuntu, its Debian/Ubuntu package needs unit tests run during package build (usually triggered by one of "make test" or "make check").

Currently "make test" builds a battery of test programs but does not start any of them.

We need that either there are unit tests triggered by "make test" or "make check" or a so-called autopkgtest which the build server runs after installing sane-airscan on a well-defined virtual machine.

My thought here is to make some or all of your test programs being run as unit tests, meaning that "make test" does not only build them but also runs them, one after the other and exits with non-zero status if one fails.

For that we need tests whose results are

- Independent of the presence or absence of supported scanner devices

- Independent of the presence or absence of the internet and of any content in the internet

This is required as the tests will run on isolated virtual machines on the build servers for Debian and Ubuntu.

Could you tell which tests of yours are fulfilling these conditions (so that we can explicitly call them during package build), or better, make "make test" run all of your tests which fulfill these conditions (skipping the ones which need devices or something in the internet).

Alexander m,

thanks for the review. There are tests in the upstream code, they are only not auto-started by "make test". I will find out with Alexander P which ones are suitable to make them run during package build.

Hi Alex, Till,

I've fixed Coverity warnings (as much as I could) and added "make check" to run automated tests. Please confirm that everybody are happy with whatever now we have, and I will issue the next, 0.99.25 release.

Alexander, am I right that you did the "make check" fixes today and the Coverity fixes on March 17?

For me this looks OK, so you could release this.

Alex Murray, are the fixes committed upstream here

https://github.com/alexpevzner/sane-airscan/commits/master

from March 17 and April 11 OK?

Yes, Till.

If you plan to test something, it is better to is do before release. As soon as you confirm that everything is OK, I'll issue the new release. If no further testing is planned, I can issue new release immediately.

I'm trying to avoid unneeded releases with nearly no noticeable changes between them, as every new release affects many people.

Alexander, I think I do not need to do any testing, probably it is much easier for Alex Murray to check whether the Coverity problems are fixed.

From my side you could release.

Alex Murray,

do I need to fix something else before we can release?

No, thanks again for your work in addressing these (sorry I didn't realise you were blocked on me responding)

Till, Alex,

0.99.25 released. Thank you for feedback!

Thanks to you all.

I have uploaded the new release now as sane-airscan_0.99.25-0ubuntu1 to hirsute.

@Till - If I might ask to know where to expect it. What will be the packaging (adding a dependency) or seed change that will make this show up as component mismatch?

Till,

yet another question: will ipp-usb reach the Ubuntu Main?

@Alexander P, ipp-usb is already in Main.

@Christian, I could add a Recommends to ipp-usb, to sane-backends, or we could add it to the ubuntu-desktop seed. I think sane-backends is the best place for the Recommends: as for scanning one would install sane-backends and the Recommends: would then complete the scanner driver set with this important driver.

WDYT?

By the way, the upload is not yet visible in LP, as Universe seems to need manual approval because of post-Beta, too.

> @Christian, I could add a Recommends to ipp-usb, to sane-backends, or we
> could add it to the ubuntu-desktop seed. I think sane-backends is the
> best place for the Recommends: as for scanning one would install sane-
> backends and the Recommends: would then complete the scanner driver set
> with this important driver.
>
> WDYT?

There is no right/wrong to this, mostly only opinion.
I'd prefer a recommends over a seed change, but that is opinion.

If sane-backends is the right place is up to you, but what you
described sounded plausible to me.
I checked and sane-backends is only the source package.

saine-airscan provides
/usr/lib/x86_64-linux-gnu/sane/libsane-airscan.so.1

Which plugs into pkg:libsane1 - so from src:sane-backends a recommends
most likely would be added to binary package libsane1 I guess?

sane-airscan_0.99.25-0ubuntu1 got accepted into hirsute-proposed now.

@Christian, while waiting for your answer I already decided for adding the Recommends to libsane1. I am currently preparing the package and will upload it soon.

@Christian, sane-backends 1.0.32-0ubuntu2 with libsane1 recommending sane-airscan is uploaded.

@Christian, sane-backends_1.0.32-0ubuntu2 got approved into hirsute-proposed.

I see it now, thanks Till!

sane-airscan: sane-airscan
MIR: #1891682 (Confirmed)
[Reverse-Recommends: libsane1 (MAIN)]

Thereby moving this along to be ready for an Archive admin to promote.

$ ./change-override -c main -S sane-airscan
Override component to main
sane-airscan 0.99.25-0ubuntu1 in hirsute: universe/misc -> main
sane-airscan 0.99.25-0ubuntu1 in hirsute amd64: universe/graphics/optional/100% -> main
sane-airscan 0.99.25-0ubuntu1 in hirsute arm64: universe/graphics/optional/100% -> main
sane-airscan 0.99.25-0ubuntu1 in hirsute armhf: universe/graphics/optional/100% -> main
sane-airscan 0.99.25-0ubuntu1 in hirsute ppc64el: universe/graphics/optional/100% -> main
sane-airscan 0.99.25-0ubuntu1 in hirsute riscv64: universe/graphics/optional/100% -> main
sane-airscan 0.99.25-0ubuntu1 in hirsute s390x: universe/graphics/optional/100% -> main
Override [y|N]? y
7 publications overridden.

@Alexander P, thank you very much for your fixes and the quick release! sane-airscan made it into Ubuntu Main now, 2 days before Final Freeze and less than 2 weeks before release of 21.04.

@Alexander M, @Christian, @Didier, thank you very much for the quick finalization of this MIR!

Till,

nice to hear, thank you for the good news!

@Alexander P, this especially means that sane-airscan is included with the default Ubuntu Desktop installation from 21.04 Hirsute on, meaning that the scanners of thousands multi-function devices will just work with Ubuntu.