Kernel CI fails if hwpack isn't available

Bug #1175132 reported by Fathi Boudra
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Linaro CI
Confirmed
Medium
Fathi Boudra

Bug Description

Kernel CI on Origen fails since the hwpacks aren't available anymore (per pruning policy).

As we want to ensure Origen is still working on mainline and that Linaro doesn't support Origen anymore, we should fix find_latest.py script to grab latest available hwpack from releases (fallback).

Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: none → 2013.05
assignee: nobody → Fathi Boudra (fboudra)
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

This issue re-produced on panda build=30

https://ci.linaro.org/jenkins/job/linux-next/hwpack=panda,label=precise_hwpack_cloud/30/console

01:31:55 ++ python lci-build-tools/get_latest_slo_hwpack
01:31:55 Traceback (most recent call last):
01:31:55 File "lci-build-tools/get_latest_slo_hwpack", line 16, in <module>
01:31:55 hwpack_url = find_latest_hwpack(hwpack_type)
01:31:55 File "/mnt/ci_build/workspace/linux-next/hwpack/panda/label/precise_hwpack_cloud/lci-build-tools/find_latest.py", line 96, in find_latest_hwpack
01:31:55 filename = find_latest(hwpack_url)
01:31:55 File "/mnt/ci_build/workspace/linux-next/hwpack/panda/label/precise_hwpack_cloud/lci-build-tools/find_latest.py", line 72, in find_latest
01:31:55 lastdate = dates.pop()
01:31:55 IndexError: pop from empty list

Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :
Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.05 → 2013.06
Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.06 → 2013.07
Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.07 → 2013.08
Fathi Boudra (fboudra)
summary: - Kernel CI on Origen fails if hwpack isn't available
+ Kernel CI fails if hwpack isn't available
Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

Kernel CI failed summart report:
Failed to build linux-next and linux-arm-soc-for-next: snowball hw-pack.
kernel image (zImage) build successful but failed to create hw-pack.

Failed logs:
- https://ci.linaro.org/jenkins/job/linux-next/hwpack=snowball,label=kernel_cloud/113/consoleFull
- https://ci.linaro.org/jenkins/job/linux-arm-soc-for-next/hwpack=snowball,label=kernel_cloud/115/consoleFull

Error log:
02:18:33 dpkg-deb: building package
`linux-image-3.11.0-rc2-linaro-u8500' in
`../linux-image-3.11.0-rc2-linaro-u8500_3.11.0-rc2-linaro-u8500-1_armhf.deb'.
02:18:38 ++ date +%s
02:18:38 + END=1375064317
02:18:38 + EXECUTION_TIME_IN_SEC=666
02:18:38 + '[' 1 -eq 1 ']'
02:18:38 ++ python lci-build-tools/get_latest_slo_hwpack
02:18:39 Traceback (most recent call last):
02:18:39 File "lci-build-tools/get_latest_slo_hwpack", line 16, in <module>
02:18:39 hwpack_url = find_latest_hwpack(hwpack_type)
02:18:39 File
"/mnt/ci_build/workspace/linux-next/hwpack/snowball/label/kernel_cloud/lci-build-tools/find_latest.py",
line 96, in find_latest_hwpack
02:18:39 filename = find_latest(hwpack_url)
02:18:39 File
"/mnt/ci_build/workspace/linux-next/hwpack/snowball/label/kernel_cloud/lci-build-tools/find_latest.py",
line 72, in find_latest
02:18:39 lastdate = dates.pop()
02:18:39 IndexError: pop from empty list
02:18:39 + latest_kernel=
02:18:39 + cleanup_exit
02:18:39 + cd /mnt/ci_build/workspace/linux-next/hwpack/snowball/label/kernel_cloud
02:18:39 + rm -rf lci-build-tools
02:18:39 + rm -rf builddir-3.11.0-rc2-linaro-u8500
02:18:40 Build step 'Execute shell' marked build as failure
02:18:40 Finished: FAILURE

Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.08 → 2013.09
Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

Snowball hwpack build failure summary report:
-------------------------------------------------------------------
https://ci.linaro.org/jenkins/job/linux-next/hwpack=snowball,label=kernel_cloud/165/consoleFull
https://ci.linaro.org/jenkins/job/linux-next/hwpack=snowball,label=kernel_cloud/166/consoleFull

Build error log:
---------------------
02:31:14 ++ python lci-build-tools/get_latest_slo_hwpack
02:31:14 Traceback (most recent call last):
02:31:14 File "lci-build-tools/get_latest_slo_hwpack", line 16, in <module>
02:31:14 hwpack_url = find_latest_hwpack(hwpack_type)
02:31:14 File "/mnt/ci_build/workspace/linux-next/hwpack/snowball/label/kernel_cloud/lci-build-tools/find_latest.py", line 96, in find_latest_hwpack
02:31:14 filename = find_latest(hwpack_url)
02:31:14 File "/mnt/ci_build/workspace/linux-next/hwpack/snowball/label/kernel_cloud/lci-build-tools/find_latest.py", line 72, in find_latest
02:31:14 lastdate = dates.pop()
02:31:14 IndexError: pop from empty list

Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

STE Snowball Kernel CI build failed on all following 3 branches since 19 September 2013:

linux-arm-soc-for-next: https://ci.linaro.org/jenkins/job/linux-arm-soc-for-next/hwpack=snowball,label=kernel_cloud/167/console
linux-mainline: https://ci.linaro.org/jenkins/job/linux-mainline/hwpack=snowball,label=kernel_cloud/165/console
linux-next: https://ci.linaro.org/jenkins/job/linux-next/hwpack=snowball,label=kernel_cloud/165/console

The error looks same:

++ python lci-build-tools/get_latest_slo_hwpack
Traceback (most recent call last):
  File "lci-build-tools/get_latest_slo_hwpack", line 16, in <module>
    hwpack_url = find_latest_hwpack(hwpack_type)
  File "/mnt/ci_build/workspace/linux-mainline/hwpack/snowball/label/kernel_cloud/lci-build-tools/find_latest.py", line 96, in find_latest_hwpack
    filename = find_latest(hwpack_url)
  File "/mnt/ci_build/workspace/linux-mainline/hwpack/snowball/label/kernel_cloud/lci-build-tools/find_latest.py", line 72, in find_latest
    lastdate = dates.pop()
IndexError: pop from empty list

So looks like the hardware pack is missing again. I also found another 2 errors in build log:

[1] ERROR: ld.so: object 'libfakeroot-sysv.so' from LD_PRELOAD cannot be preloaded: ignored.
[2] stat: cannot stat `arch/arm/boot/*.dtb': No such file or directory

I noticed that at the end of build log, it says:
Sending email to: <email address hidden>, <email address hidden>, <email address hidden>
Therefore I assume you have received notification email.

Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.09 → 2013.10
Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

The project linux-next » snowball,kernel_cloud Build # 180 - Still Failing.
Console output at https://ci.linaro.org/jenkins/job/linux-next/./hwpack=snowball,label=kernel_cloud/180/

Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

The project linux-next » snowball,kernel_cloud Build # 190 - Still Failing.
Console output at https://ci.linaro.org/jenkins/job/linux-next/./hwpack=snowball,label=kernel_cloud/190/

Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

linux-next Build # 198 - Still Failing.

console output at https://ci.linaro.org/jenkins/job/linux-next/198/

Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

 linux-next » snowball,kernel_cloud Build # 224 - Still Failing.
console output at https://ci.linaro.org/jenkins/job/linux-next/./hwpack=snowball,label=kernel_cloud/224/

Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.10 → 2013.11
Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

linux-next » snowball,kernel_cloud Build # 240 - Still Failing.
Console output at https://ci.linaro.org/jenkins/job/linux-next/./hwpack=snowball,label=kernel_cloud/240/

Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.11 → 2013.12
Revision history for this message
Naresh Kamboju (naresh-kamboju) wrote :

linux-next » snowball,kernel_cloud Build # 256 - Still Failing
console output at https://ci.linaro.org/jenkins/job/linux-next/./hwpack=snowball,label=kernel_cloud/256

Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2013.12 → 2014.01
Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2014.01 → 2014.03
Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2014.03 → 2014.04
Fathi Boudra (fboudra)
Changed in linaro-ci:
milestone: 2014.04 → 2014.05
Revision history for this message
Chase Qi (chase-qi) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.