status_update not sent any more, causes eternal hang on conffile prompts in aptdaemon

I believe aptdaemon/progress.py _on_status_update() is responsible for intercepting conffile prompts. However, this does not get called *at all* any more, I checked this by adding a raise RuntimeError('caught!') at the top, and also some log.warning() (which never appears). So it seems recent apt breaks the status-fd handling?

I bisected this down to http://anonscm.debian.org/gitweb/?p=apt/apt.git;a=commitdiff;h=11ef54810d

Tentatively moving this to apt as this doesn't seem to be an intended change.

Thanks Martin! There is currently no apt-progress-fd testcase for conffile (we have one for error though). I will work on one now.

Here is the fix:

diff --git a/apt-pkg/deb/dpkgpm.cc b/apt-pkg/deb/dpkgpm.cc
index b4bfd14..e77dd54 100644
--- a/apt-pkg/deb/dpkgpm.cc
+++ b/apt-pkg/deb/dpkgpm.cc
@@ -608,7 +608,7 @@ void pkgDPkgPM::ProcessDpkgStatusLine(char *line)
          WriteApportReport(list[1].c_str(), list[3].c_str());
          return;
       }
- else if(action == "conffile")
+ else if(action == "conffile-prompt")
       {
          d->progress->ConffilePrompt(list[1], PackagesDone, PackagesTotal,
                                      list[3]);

I'm working on a proper testcase now to ensure this does not break again. Sorry for the trouble and thanks a bunch
for your help!

Many Thanks Michael, you are a star!

Michael, btw, if you upload this please note that apt currently fails its autopkgtest. Primarily it produces a lot of stderr and fails to build during the test, but when I do that manually in a VM I also get one test case failure.

But this autopkgtest doesn't help much in the first place: it only tests the build tree, but it must test the actually installed apt (i. e. /usr/bin/). Can you run the tests against the installed package?

This bug was fixed in the package apt - 0.9.14.1ubuntu1

---------------
apt (0.9.14.1ubuntu1) trusty; urgency=low

  * Merge from debian/sid
  * disable autopkgtest for now until it can test the installed
    packages and not only the build-tree (thanks to Martin Pitt)

apt (0.9.14.1) unstable; urgency=medium

  * fix apt-get source -t dist regression (closes: #731853)
    and add testcase
  * clarify error message when apt-get source=ver fails
    (thans to David Kalnischkies)
  * Fix conffile prompt regression (LP: #1260297)
    and add testcase
  * improve error message for apt-get source pkg:arch{=ver,/release}
 -- Michael Vogt <email address hidden> Thu, 12 Dec 2013 21:19:49 +0100

Hey Michael, many thanks for the fix!

On Thu, Dec 12, 2013 at 01:49:46PM -0000, Martin Pitt wrote:
> Michael, btw, if you upload this please note that apt currently fails
> its autopkgtest. Primarily it produces a lot of stderr and fails to
> build during the test, but when I do that manually in a VM I also get
> one test case failure.
>
> But this autopkgtest doesn't help much in the first place: it only tests
> the build tree, but it must test the actually installed apt (i. e.
> /usr/bin/). Can you run the tests against the installed package?

Indeed, sorry for that. I disabled autopkgtest for the last upload
again and prepared a branch that redirects stderr to a file (output is
mostly noise) and test the installed files instead of the build file.

One question - whats the best practise here, should the test also
check against the build tree? Or is it better to run the tests against
the build tree during the package build? I guess the later but would
be nice if you could confirm :)

Cheers,
 Michael

Indeed, run unit tests against the source tree during package build (failing the build if they fail), and integration tests in autopkgtest against the system-installed package. The latter don't necessarily need to be the same upstream tests; coverage of fine details is already handled during package build, it's sufficient to have some kind of smoke test that "apt-get update", "apt-cache search", "apt-get install" etc. work in general: I. e. binaries are installed, dependencies are correct, expected directories in /var exist, and so on.