Launchpad itself

Bug #2030672
Comment #3

Comment 3 for bug 2030672

Revision history for this message

dann frazier (dannf) wrote on 2023-08-07:

I'm not familiar with the spec, but the documentation[*] says:
"If a stage is a list of job names, then those jobs are executed in parallel."

If they were executed in parallel and each allowed to run to completion, then I think this issue would be resolved. However, that is not what I'm observing:

# Here's a .launchpad.yaml with a pipeline comprising 2 jobs, with the first designed to fail:
ubuntu@cortez-vm-0:~$ cat .launchpad.yaml
pipeline:
- ["job1", "job2"]

jobs:
  job1:
    architectures: amd64
    series: jammy
    run: |-
      echo "Hello job1"
      /bin/false
  job2:
    architectures: amd64
    series: jammy
    run: |-
      echo "Hello job2"
# Here you can see that we only get output from the first job:
ubuntu@cortez-vm-0:~$ lpci
Starting lpci, version 0.2.3
Logging execution to '/home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log'
Running ['bash', '--noprofile', '--norc', '-ec', 'echo "Hello job1"\n/bin/false']
:: Hello job1
Job 'job1' for jammy/amd64 failed with exit status 1.
Full execution log: '/home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log'
# The log includes no evidence of an attempt to run job2:
ubuntu@cortez-vm-0:~$ cat /home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log
2023-08-07 22:17:38.577 Starting lpci, version 0.2.3
2023-08-07 22:17:38.738 Executing on host: lxc --project default profile show local:default
2023-08-07 22:17:38.858 Launching environment for jammy/amd64
2023-08-07 22:17:38.859 Using hostname 'craft-buildd-instance'
2023-08-07 22:17:38.860 Executing on host: lxc remote list --format=yaml
2023-08-07 22:17:38.964 Remote 'craft-com.ubuntu.cloud-buildd' already exists.
2023-08-07 22:17:38.964 Using hostname 'lpci-ubuntu-7340034-jammy-amd64'
2023-08-07 22:17:38.965 Executing on host: lxc project list local: --format=yaml
2023-08-07 22:17:39.076 Executing on host: lxc --project default profile show local:default
2023-08-07 22:17:39.196 Executing on host: lxc --project lpci profile edit local:default
2023-08-07 22:17:39.352 Executing on host: lxc project list local: --format=yaml
2023-08-07 22:17:39.472 Set LXD instance name to 'lpci-ubuntu-7340034-jammy-amd64'
2023-08-07 22:17:39.472 Checking for instance 'lpci-ubuntu-7340034-jammy-amd64' in project 'lpci' in remote 'local'
2023-08-07 22:17:39.472 Executing on host: lxc --project lpci list local: --format=yaml
2023-08-07 22:17:39.624 Executing on host: lxc --project lpci config get local:lpci-ubuntu-7340034-jammy-amd64 raw.idmap
2023-08-07 22:17:39.741 Executing on host: lxc --project lpci list local: --format=yaml
2023-08-07 22:17:39.891 Instance exists and is not running. Starting instance.
2023-08-07 22:17:39.891 Executing on host: lxc --project lpci start local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:40.402 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin test -f /etc/craft-instance.conf
2023-08-07 22:17:40.588 Executing on host: lxc --project lpci file pull local:lpci-ubuntu-7340034-jammy-amd64/etc/craft-instance.conf /home/ubuntu/tmpv2onip1u.tmp-craft/tmpnefv4ser
2023-08-07 22:17:40.784 Instance has already been setup.
2023-08-07 22:17:40.784 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin cat /etc/os-release
2023-08-07 22:17:40.967 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin test -f /etc/craft-instance.conf
2023-08-07 22:17:41.107 Executing on host: lxc --project lpci file pull local:lpci-ubuntu-7340034-jammy-amd64/etc/craft-instance.conf /home/ubuntu/tmpmyhx9n4a.tmp-craft/tmpdoavwwau
2023-08-07 22:17:41.227 Instance is compatible with compatibility tag 'lpci-buildd-base-v1.0'
2023-08-07 22:17:41.227 Waiting for environment to be ready...
2023-08-07 22:17:41.227 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin systemctl is-system-running
2023-08-07 22:17:41.403 Waiting for networking to be ready...
2023-08-07 22:17:41.403 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin getent hosts snapcraft.io
2023-08-07 22:17:41.643 Holding refreshes for snaps.
2023-08-07 22:17:41.643 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap set system refresh.hold=2023-08-08T22:17:41.643015Z
2023-08-07 22:17:41.955 Waiting for pending snap refreshes to complete.
2023-08-07 22:17:41.955 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap watch '--last=auto-refresh?'
2023-08-07 22:17:42.129 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap unset system proxy.http
2023-08-07 22:17:42.443 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap unset system proxy.https
2023-08-07 22:17:42.735 No snaps to install.
2023-08-07 22:17:42.735 Executing on host: lxc --project lpci config device show local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:42.848 Executing on host: lxc --project lpci config device add local:lpci-ubuntu-7340034-jammy-amd64 disk-/root/tmp-project disk source=/home/ubuntu path=/root/tmp-project
2023-08-07 22:17:42.994 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- rm -rf /build/lpci/project
2023-08-07 22:17:43.162 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- mkdir -p /build/lpci
2023-08-07 22:17:43.322 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- cp -a /root/tmp-project /build/lpci/project
2023-08-07 22:17:43.478 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- rm -f /usr/local/sbin/policy-rc.d
2023-08-07 22:17:43.654 Executing on host: lxc --project lpci config device show local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:43.771 Executing on host: lxc --project lpci config device remove local:lpci-ubuntu-7340034-jammy-amd64 disk-/root/tmp-project
2023-08-07 22:17:43.924 Running command for the job...
2023-08-07 22:17:43.925 Running ['bash', '--noprofile', '--norc', '-ec', 'echo "Hello job1"\n/bin/false']
2023-08-07 22:17:43.926 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 --cwd /build/lpci/project -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin bash --noprofile --norc -ec 'echo "Hello job1"
/bin/false'
2023-08-07 22:17:44.096 :: Hello job1
2023-08-07 22:17:44.101 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- rm -rf /build/lpci/project
2023-08-07 22:17:44.263 Executing on host: lxc --project lpci config device show local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:44.385 Executing on host: lxc --project lpci stop local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:45.824 Job 'job1' for jammy/amd64 failed with exit status 1.
2023-08-07 22:17:45.824 Full execution log: '/home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log'

[*] https://lpci.readthedocs.io/en/latest/configuration.html

I'm not familiar with the spec, but the documentation[*] says:
 "If a stage is a list of job names, then those jobs are executed in parallel."
 
If they were executed in parallel and each allowed to run to completion, then I think this issue would be resolved. However, that is not what I'm observing:

# Here's a .launchpad.yaml with a pipeline comprising 2 jobs, with the first designed to fail:
ubuntu@cortez-vm-0:~$ cat .launchpad.yaml
pipeline:
- ["job1", "job2"]

jobs:
  job1:
    architectures: amd64
    series: jammy
    run: |-
      echo "Hello job1"
      /bin/false
  job2:
    architectures: amd64
    series: jammy
    run: |-
      echo "Hello job2"
# Here you can see that we only get output from the first job:
ubuntu@cortez-vm-0:~$ lpci
Starting lpci, version 0.2.3                                                                                            
Logging execution to '/home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log'                               
Running ['bash', '--noprofile', '--norc', '-ec', 'echo "Hello job1"\n/bin/false']                                       
:: Hello job1                                                                                                           
Job 'job1' for jammy/amd64 failed with exit status 1.                                                                   
Full execution log: '/home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log'                 
# The log includes no evidence of an attempt to run job2:        
ubuntu@cortez-vm-0:~$ cat /home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log
2023-08-07 22:17:38.577 Starting lpci, version 0.2.3
2023-08-07 22:17:38.738 Executing on host: lxc --project default profile show local:default
2023-08-07 22:17:38.858 Launching environment for jammy/amd64
2023-08-07 22:17:38.859 Using hostname 'craft-buildd-instance'
2023-08-07 22:17:38.860 Executing on host: lxc remote list --format=yaml
2023-08-07 22:17:38.964 Remote 'craft-com.ubuntu.cloud-buildd' already exists.
2023-08-07 22:17:38.964 Using hostname 'lpci-ubuntu-7340034-jammy-amd64'
2023-08-07 22:17:38.965 Executing on host: lxc project list local: --format=yaml
2023-08-07 22:17:39.076 Executing on host: lxc --project default profile show local:default
2023-08-07 22:17:39.196 Executing on host: lxc --project lpci profile edit local:default
2023-08-07 22:17:39.352 Executing on host: lxc project list local: --format=yaml
2023-08-07 22:17:39.472 Set LXD instance name to 'lpci-ubuntu-7340034-jammy-amd64'
2023-08-07 22:17:39.472 Checking for instance 'lpci-ubuntu-7340034-jammy-amd64' in project 'lpci' in remote 'local'
2023-08-07 22:17:39.472 Executing on host: lxc --project lpci list local: --format=yaml
2023-08-07 22:17:39.624 Executing on host: lxc --project lpci config get local:lpci-ubuntu-7340034-jammy-amd64 raw.idmap
2023-08-07 22:17:39.741 Executing on host: lxc --project lpci list local: --format=yaml
2023-08-07 22:17:39.891 Instance exists and is not running. Starting instance.
2023-08-07 22:17:39.891 Executing on host: lxc --project lpci start local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:40.402 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin test -f /etc/craft-instance.conf
2023-08-07 22:17:40.588 Executing on host: lxc --project lpci file pull local:lpci-ubuntu-7340034-jammy-amd64/etc/craft-instance.conf /home/ubuntu/tmpv2onip1u.tmp-craft/tmpnefv4ser
2023-08-07 22:17:40.784 Instance has already been setup.
2023-08-07 22:17:40.784 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin cat /etc/os-release
2023-08-07 22:17:40.967 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin test -f /etc/craft-instance.conf
2023-08-07 22:17:41.107 Executing on host: lxc --project lpci file pull local:lpci-ubuntu-7340034-jammy-amd64/etc/craft-instance.conf /home/ubuntu/tmpmyhx9n4a.tmp-craft/tmpdoavwwau
2023-08-07 22:17:41.227 Instance is compatible with compatibility tag 'lpci-buildd-base-v1.0'
2023-08-07 22:17:41.227 Waiting for environment to be ready...
2023-08-07 22:17:41.227 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin systemctl is-system-running
2023-08-07 22:17:41.403 Waiting for networking to be ready...
2023-08-07 22:17:41.403 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin getent hosts snapcraft.io
2023-08-07 22:17:41.643 Holding refreshes for snaps.
2023-08-07 22:17:41.643 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap set system refresh.hold=2023-08-08T22:17:41.643015Z
2023-08-07 22:17:41.955 Waiting for pending snap refreshes to complete.
2023-08-07 22:17:41.955 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap watch '--last=auto-refresh?'
2023-08-07 22:17:42.129 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap unset system proxy.http
2023-08-07 22:17:42.443 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin snap unset system proxy.https
2023-08-07 22:17:42.735 No snaps to install.
2023-08-07 22:17:42.735 Executing on host: lxc --project lpci config device show local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:42.848 Executing on host: lxc --project lpci config device add local:lpci-ubuntu-7340034-jammy-amd64 disk-/root/tmp-project disk source=/home/ubuntu path=/root/tmp-project
2023-08-07 22:17:42.994 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- rm -rf /build/lpci/project
2023-08-07 22:17:43.162 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- mkdir -p /build/lpci
2023-08-07 22:17:43.322 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- cp -a /root/tmp-project /build/lpci/project
2023-08-07 22:17:43.478 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- rm -f /usr/local/sbin/policy-rc.d
2023-08-07 22:17:43.654 Executing on host: lxc --project lpci config device show local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:43.771 Executing on host: lxc --project lpci config device remove local:lpci-ubuntu-7340034-jammy-amd64 disk-/root/tmp-project
2023-08-07 22:17:43.924 Running command for the job...
2023-08-07 22:17:43.925 Running ['bash', '--noprofile', '--norc', '-ec', 'echo "Hello job1"\n/bin/false']
2023-08-07 22:17:43.926 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 --cwd /build/lpci/project -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin bash --noprofile --norc -ec 'echo "Hello job1"
/bin/false'
2023-08-07 22:17:44.096 :: Hello job1
2023-08-07 22:17:44.101 Executing in container: lxc --project lpci exec local:lpci-ubuntu-7340034-jammy-amd64 -- rm -rf /build/lpci/project
2023-08-07 22:17:44.263 Executing on host: lxc --project lpci config device show local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:44.385 Executing on host: lxc --project lpci stop local:lpci-ubuntu-7340034-jammy-amd64
2023-08-07 22:17:45.824 Job 'job1' for jammy/amd64 failed with exit status 1.
2023-08-07 22:17:45.824 Full execution log: '/home/ubuntu/.local/state/lpci/log/lpci-20230807-221738.576685.log'

[*] https://lpci.readthedocs.io/en/latest/configuration.html