[Power 9, Power NV][WSP/Boston]Ubuntu 16.04.03 : perf test is hung at Breakpoint overflow signal handler

Bug #1711075 reported by bugproxy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Incomplete
High
Canonical Kernel Team
linux (Ubuntu)
In Progress
High
Joseph Salisbury

Bug Description

== Comment: # - Shriya R. Kulkarni <> - 2017-06-14 01:11:02 ==
Problem Description :
=============
perf test is hung at Breakpoint overflow signal handler.

Machine details :
==========
System : WSP , P9 baremetal.
OS : Ubuntu 16.04.03
uname -a : Linux ltc-wspoon3 4.10.0-23-generic #25~16.04.1-Ubuntu SMP Fri Jun 9 10:43:34 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Testing:
====
root@ltc-wspoon3:~# perf test
 1: vmlinux symtab matches kallsyms : Skip
 2: Detect openat syscall event : Ok
 3: Detect openat syscall event on all cpus : Ok
 4: Read samples using the mmap interface : Ok
 5: Parse event definition strings : Ok
 6: PERF_RECORD_* events & perf_sample fields : Ok
 7: Parse perf pmu format : Ok
 8: DSO data read : Ok
 9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler :^C

root@ltc-wspoon3:~# perf test -v 17
17: Breakpoint overflow signal handler :
--- start ---
test child forked, pid 3802
failed opening event 0
failed opening event 0

Attach : strace output.

Steps to reproduce :
============
1. Install perf.
2. Run perf test.
3. When system hangs , exit it by : ^c.

== Comment: # - Ravikumar B. Bangoria <> - 2017-07-13 04:44:29 ==
Shriya,

Can you please check after applying:
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=d89ba5353f301971dd7d2f9fdf25c4432728f38e

== Comment: # - Shriya R. Kulkarni <> - 2017-07-19 06:28:13 ==
Verified and Tested on Ubuntu 16.04.03:
=========================
uname -a : 4.10.0-27-generic
System : Boston

Testing without patch using distro perf:
========================
root@ltc-boston27:~# perf test
 1: vmlinux symtab matches kallsyms : Skip
 2: Detect openat syscall event : Ok
 3: Detect openat syscall event on all cpus : Ok
 4: Read samples using the mmap interface : Ok
 5: Parse event definition strings : Ok
 6: PERF_RECORD_* events & perf_sample fields : Ok
 7: Parse perf pmu format : Ok
 8: DSO data read : Ok
 9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler :^C
root@ltc-boston27:~#

Testing with Patch using distro perf :
======================
root@ltc-boston27:~/linux-hwe-4.10.0/tools/perf# ./perf test
 1: vmlinux symtab matches kallsyms : Ok
 2: Detect openat syscall event : Ok
 3: Detect openat syscall event on all cpus : Ok
 4: Read samples using the mmap interface : Ok
 5: Parse event definition strings : Ok
 6: PERF_RECORD_* events & perf_sample fields : Ok
 7: Parse perf pmu format : Ok
 8: DSO data read : Ok
 9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Ok
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler : FAILED!
18: Breakpoint overflow sampling : FAILED!
19: Number of exit events of a simple workload : Ok
20: Software clock events period values : Ok
21: Object code reading : FAILED!
22: Sample parsing : Ok
23: Use a dummy software event to keep tracking: Ok
24: Parse with no sample_id_all bit set : Ok
25: Filter hist entries : Ok
26: Lookup mmap thread : Ok
27: Share thread mg : Ok
28: Sort output of hist entries : Ok
29: Cumulate child hist entries : Ok
30: Track with sched_switch : Ok
31: Filter fds with revents mask in a fdarray : Ok
32: Add fd to a fdarray, making it autogrow : Ok
33: kmod_path__parse : Ok
34: Thread map : Ok
35: LLVM search and compile :
35.1: Basic BPF llvm compile : Skip
35.2: kbuild searching : Skip
35.3: Compile source for BPF prologue generation: Skip
35.4: Compile source for BPF relocation : Skip
36: Session topology : Ok
37: BPF filter :
37.1: Basic BPF filtering : Skip
37.2: BPF relocation checker : Skip
38: Synthesize thread map : Ok
39: Remove thread map : Ok
40: Synthesize cpu map : Ok
41: Synthesize stat config : Ok
42: Synthesize stat : Ok
43: Synthesize stat round : Ok
44: Synthesize attr update : Ok
45: Event times : Ok
46: Read backward ring buffer : Ok
47: Print cpu map : Ok
48: Probe SDT events : Ok
49: is_printable_array : Ok
50: Print bitmap : Ok
51: perf hooks : Ok
52: builtin clang support : Skip (not compiled in)
53: Test dwarf unwind : Ok

Revision history for this message
bugproxy (bugproxy) wrote : perf_test_17

Default Comment by Bridge

tags: added: architecture-ppc64le bugnameltc-155718 severity-high targetmilestone-inin16043
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → kernel-package (Ubuntu)
Changed in ubuntu-power-systems:
importance: Undecided → High
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Changed in kernel-package (Ubuntu):
importance: Undecided → High
status: New → In Progress
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Joseph Salisbury (jsalisbury)
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

I built a Zesty test kernel with a pick of commit d89ba5353f301. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1711075

Can you test this kernel and see if it resolves this bug?

Thanks in advance!

Manoj Iyer (manjo)
tags: added: triage-g
Changed in ubuntu-power-systems:
status: New → Incomplete
Revision history for this message
Manoj Iyer (manjo) wrote :

IBM, could you pls test the kernel provided by Joe and report back here?

affects: kernel-package (Ubuntu) → linux (Ubuntu)
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla
Download full text (3.5 KiB)

------- Comment From <email address hidden> 2017-09-27 03:26 EDT-------
Verified on the kernel : http://kernel.ubuntu.com/~jsalisbury/lp1711075/

System : Witherspoon + DD1.0
uname -a : Linux ltc-wspoon3 4.10.0-32-generic #36~lp1711075 SMP Wed Aug 16 14:09:56 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

root@ltc-wspoon3:/usr/lib/linux-tools/4.10.0-36-generic# ./perf test
1: vmlinux symtab matches kallsyms : Skip
2: Detect openat syscall event : Ok
3: Detect openat syscall event on all cpus : Ok
4: Read samples using the mmap interface : Ok
5: Parse event definition strings : Ok
6: PERF_RECORD_* events & perf_sample fields : Ok
7: Parse perf pmu format : Ok
8: DSO data read : Ok
9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler : FAILED!
18: Breakpoint overflow sampling : FAILED!
19: Number of exit events of a simple workload : Ok
20: Software clock events period values : Ok
21: Object code reading : FAILED!
22: Sample parsing : Ok
23: Use a dummy software event to keep tracking: Ok
24: Parse with no sample_id_all bit set : Ok
25: Filter hist entries : Ok
26: Lookup mmap thread : Ok
27: Share thread mg : Ok
28: Sort output of hist entries : Ok
29: Cumulate child hist entries : Ok
30: Track with sched_switch : Ok
31: Filter fds with revents mask in a fdarray : Ok
32: Add fd to a fdarray, making it autogrow : Ok
33: kmod_path__parse : Ok
34: Thread map : Ok
35: LLVM search and compile :
35.1: Basic BPF llvm compile : Skip
35.2: kbuild searching : Skip
35.3: Compile source for BPF prologue generation: Skip
35.4: Compile source for BPF relocation : Skip
36: Session topology : Ok
37: BPF filter :
37.1: Basic BPF filtering : Skip
37.2: BPF prologue generation : Skip
37.3: BPF relocation checker : Skip
38: Synthesize thread map : Ok
39: Remove thread map : Ok
40: Synthesize cpu map : Ok
41: Synthesize stat config : Ok
42: Synthesize stat : Ok
43: Synthesize stat round : Ok
44: Synthesize attr update : Ok
45: Event times : Ok
46: Read backward ring buffer : Ok
47: Print cpu map : Ok
...

Read more...

Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Commit d89ba5353f301 was SRU'd for bug 1706033, so I'll mark this bug as a duplicate of that bug.

Revision history for this message
bugproxy (bugproxy) wrote :
Download full text (3.5 KiB)

------- Comment From <email address hidden> 2017-12-13 23:26 EDT-------
Verified with latest kernel on Ubuntu 16.04.03 and it works as expected
=============================================

root@ltc-wspoon12:/usr/lib/linux-hwe-tools-4.10.0-42# ./perf test
1: vmlinux symtab matches kallsyms : Skip
2: Detect openat syscall event : Ok
3: Detect openat syscall event on all cpus : Ok
4: Read samples using the mmap interface : Ok
5: Parse event definition strings : Ok
6: PERF_RECORD_* events & perf_sample fields : Ok
7: Parse perf pmu format : Ok
8: DSO data read : Ok
9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler : FAILED!
18: Breakpoint overflow sampling : FAILED!
19: Number of exit events of a simple workload : Ok
20: Software clock events period values : Ok
21: Object code reading : FAILED!
22: Sample parsing : Ok
23: Use a dummy software event to keep tracking: Ok
24: Parse with no sample_id_all bit set : Ok
25: Filter hist entries : Ok
26: Lookup mmap thread : Ok
27: Share thread mg : Ok
28: Sort output of hist entries : Ok
29: Cumulate child hist entries : Ok
30: Track with sched_switch : Ok
31: Filter fds with revents mask in a fdarray : Ok
32: Add fd to a fdarray, making it autogrow : Ok
33: kmod_path__parse : Ok
34: Thread map : Ok
35: LLVM search and compile :
35.1: Basic BPF llvm compile : Skip
35.2: kbuild searching : Skip
35.3: Compile source for BPF prologue generation: Skip
35.4: Compile source for BPF relocation : Skip
36: Session topology : Ok
37: BPF filter :
37.1: Basic BPF filtering : Skip
37.2: BPF prologue generation : Skip
37.3: BPF relocation checker : Skip
38: Synthesize thread map : Ok
39: Remove thread map : Ok
40: Synthesize cpu map : Ok
41: Synthesize stat config : Ok
42: Synthesize stat : Ok
43: Synthesize stat round : Ok
44: Synthesize attr update : Ok
45: Event times : Ok
46: Read backward ring buffer : Ok
47: Print cpu map : Ok
48: Probe SDT events : Skip
49: is_printable_array : Ok
50: Print ...

Read more...

Brad Figg (brad-figg)
tags: added: cscc
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.