Ubuntu
sshuttle package

sshuttle merged to 1.1.0-1ubuntu1 fails autopkgtests - needs merged for kinetic

Bug #1962286 reported by Frank Heimes
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sshuttle (Ubuntu)
New
High
Frank Heimes
Ubuntu ubuntu-22.05

Bug Description

An attempt to merge sshuttle prior to jammy FF failed, because of autopkgtest failures.

The delta between Debian and Ubuntu is d/t/* only.
A v1.1.0-1ubuntu1 is available here: https://launchpad.net/~fheimes/+archive/ubuntu/sshuttle

The failures can be found here:
https://autopkgtest.ubuntu.com/results/autopkgtest-jammy-ddstreet-sponsor/jammy/amd64/s/sshuttle/20220224_214719_d5d00@/log.gz

Especially:

--------%<----------------%<----------------%<----------------%<--------

remote-trusty: lxc start remote-trusty
remote-trusty: Waiting for remote-trusty to finish starting
this ssh connection should timeout:
running: lxc exec remote-trusty -- ssh -o ConnectionAttempts=2 -v 10.252.0.1 -- cat /proc/sys/kernel/hostname
ssh: connect to host 10.252.0.1 port 22: No route to host
ssh failed (as expected)
running: lxc exec remote-trusty -- sshuttle --python python3 -r 10.254.0.2 10.252.0.0/24
waiting for sshuttle to start....sshuttle failed :-(
skipped 'This is an expected failure, ignoring test failure'
ERROR

======================================================================
ERROR: setUpClass (__main__.SshuttleTest_xenial)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 544, in setUpClass
    remote = Remote(f'remote-{cls.release}', cls.release)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 287, in __init__
    self._wait_for_networking()
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 319, in _wait_for_networking
    raise Exception(f'Timed out waiting for remote {self.name} networking')
Exception: Timed out waiting for remote remote-xenial networking

======================================================================
FAIL: test_local_to_remote_nopy (__main__.SshuttleTest_trusty)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 661, in test_local_to_remote_nopy
    self._test_to_remote(self, self.remote, None)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 683, in _test_to_remote
    self.sshuttle_start(dst, python)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 608, in sshuttle_start
    self.fail('sshuttle process failed to start')
VerboseAssertionError: sshuttle process failed to start
Test detail: testbed-trusty sshuttle 1.1.0-1ubuntu1 to remote-trusty

======================================================================
FAIL: test_local_to_remote_py2 (__main__.SshuttleTest_trusty)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 664, in test_local_to_remote_py2
    self._test_to_remote(self, self.remote, 'python2')
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 683, in _test_to_remote
    self.sshuttle_start(dst, python)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 608, in sshuttle_start
    self.fail('sshuttle process failed to start')
VerboseAssertionError: sshuttle process failed to start
Test detail: testbed-trusty sshuttle 1.1.0-1ubuntu1 to remote-trusty python2

======================================================================
FAIL: test_remote_to_reverse_remote_nopy (__main__.SshuttleTest_trusty)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 670, in test_remote_to_reverse_remote_nopy
    self._test_to_remote(self.remote, self.reverse_remote, None)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 683, in _test_to_remote
    self.sshuttle_start(dst, python)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 608, in sshuttle_start
    self.fail('sshuttle process failed to start')
VerboseAssertionError: sshuttle process failed to start
Test detail: remote-trusty sshuttle 0.54-2 to reverse-remote-jammy

======================================================================
FAIL: test_remote_to_reverse_remote_py2 (__main__.SshuttleTest_trusty)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 673, in test_remote_to_reverse_remote_py2
    self._test_to_remote(self.remote, self.reverse_remote, 'python2')
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 683, in _test_to_remote
    self.sshuttle_start(dst, python)
  File "/tmp/autopkgtest.IbY0Uu/build.hRt/src/debian/tests/cross-release", line 608, in sshuttle_start
    self.fail('sshuttle process failed to start')
VerboseAssertionError: sshuttle process failed to start
Test detail: remote-trusty sshuttle 0.54-2 to reverse-remote-jammy python2

----------------------------------------------------------------------
Ran 30 tests in 2749.995s

FAILED (failures=4, errors=1, skipped=2)

--------%<----------------%<----------------%<----------------%<--------

It seems that this is likely due to jammy's move to cgroup2, which now prevents older trusty and xenial containers from starting.

Options are:
1) fix trusty and xenial to be able to boot as a container under jammy
2) disable the tests against trusty and xenial (which would be unfortunate, since its idea is to ensure Ubuntu cross-release compatibility of sshuttle)

This needs further investigations and discussions, hence opening this FFe.

Revision history for this message
Dan Streetman (ddstreet) wrote :

I looked a bit more into the trusty failure and it seems like that isn't a cgroups issue, since it doesn't use systemd and instead uses upstart, which doesn't seem to care about what cgroup is mounted. On trusty, the problem appears to be that it's simply too old to run with the python from jammy.

By default, trusty sshuttle attempts to use 'python' on the remote system, which no longer exists in jammy.

If --python is manually specified using jammy's python3, it fails:

root@test-t:~# sshuttle --python python3 -r ubuntu@10.46.117.1 1.1.1.1/24
ubuntu@10.46.117.1's password:
  File "<string>", line 1
    import sys; skip_imports=1; verbosity=0; exec compile(sys.stdin.read(764), "assembler.py", "exec")
                                                  ^
SyntaxError: invalid syntax
client: fatal: server died with error code 1

If --python is manually specified using jammy's python2 (which isn't installed by default), that still fails:

root@test-t:~# sshuttle --python python2 -r ubuntu@10.46.117.1 1.1.1.1/24
ubuntu@10.46.117.1's password:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 26, in <module>
  File "server.py", line 170, in main
  File "server.py", line 71, in list_routes
  File "server.py", line 51, in _list_routes
  File "ssubprocess.py", line 606, in __init__
  File "ssubprocess.py", line 1117, in _execute_child
OSError: [Errno 2] No such file or directory
client: fatal: server died with error code 1

So it may not be possible to use sshuttle from trusty->jammy at all, unfortunately, and the jammy autopkgtests might need to just stop running the trusty->jammy tests completely.

Note that the jammy->trusty connection does appear to work.

Revision history for this message
Dan Streetman (ddstreet) wrote :

For the xenial container under jammy problem, I opened bug 1962332

Frank Heimes (fheimes)
summary: - [FFE] sshuttle merged to 1.1.0-1ubuntu1 fails autopkgtests
+ sshuttle merged to 1.1.0-1ubuntu1 fails autopkgtests
Bryce Harrington (bryce)
tags: added: server-todo
removed: needs-merge
summary: - sshuttle merged to 1.1.0-1ubuntu1 fails autopkgtests
+ sshuttle merged to 1.1.0-1ubuntu1 fails autopkgtests - needs merged for
+ kinetic
tags: added: kinetic needs-merge
removed: jammy server-todo
Changed in sshuttle (Ubuntu):
milestone: none → ubuntu-22.05
importance: Undecided → High
Christian Ehrhardt  (paelzer)
Changed in sshuttle (Ubuntu):
assignee: nobody → Frank Heimes (fheimes)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Since it is unlikely that Trusty will get a fix for a universe package while in ESM I'd think we should ignore that.
A similar fate would apply to Xenial - and the other issue can independently be solved by bug 1962332 that was split out of this.

But overall I think the issues with these old releases should not hold back the new version of sshuttle (which has caused neither of these issues) much longer.

Therefore I'd propose to:
- stop testing esm releases in the cross tests (or if you want make them ok to fail)
- lxd test dependency won't work in >=jammy (no deb package anymore) but it is default installed. Drop the dependency in d/t/control

@Frank:
Once that (and whatever other changes you have in mind) are in a kinetic build in your PPA this can be re-tested in that PPA. Let me (or Dan) know so that we can help you with testing.

Revision history for this message
Frank Heimes (fheimes) wrote (last edit ):

I now modified the test,
removed the esm distros
and had to handle LXD in a different way.

With that the latest version (for kinetic) in:
https://launchpad.net/~fheimes/+archive/ubuntu/sshuttle
works for me (by excluding T and X),
and hopefully unblocks the merge.

autopkgtest run was fine:
"
Ran 30 tests in 3229.385s
results - - - - - - - - - -
cross-release PASS
"

Revision history for this message
Dan Streetman (ddstreet) wrote :

> - stop testing esm releases in the cross tests (or if you want make them ok to fail)

sorry, nak; this is entirely short-sighted and defeats the purpose of the testing.

The test needs to include all supported releases, and ESM *is* supported, even if it's not supported by product engineering ;-)

There is a mechanism inside the test to ignore failures for specific, known bugs that (the is_expected_failure* functions); that's the appropriate place to put handling for known failures like this.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.