can't open trace-cmd output file

>So you can't use trace-cmd output with idlestat
Right. it only takes idlestat output format.(1)

The ASCII trace data and cpu information needs to be extracted from trace-cmd output file.
To support this trace-cmd header parsing is required as per "trace-cmd.dat - trace-cmd file format".

(1)
idlestat output format:
------------------------------------
version = 1
cpus=4 << here it reads the number of cpus
clusterA:
        core0
                cpu0
        core1
                cpu1
        core2
                cpu2
        core3
                cpu3

      <idle>-0 [002] d..2 14853.939301: cpu_idle: state=1 cpu_id=2
          <idle>-0 [000] .N.2 14853.939331: cpu_idle: state=4294967295 cpu_id=0
          <idle>-0 [000] d..2 14853.939514: cpu_idle: state=1 cpu_id=0

/* generate the ASCII report from trace.dat */
trace-cmd report trace.dat > tracecmdreport

/* idlestat can accept the raw report format */
idlestat -o tracecmdreport

I am able to get the idlestat output from trace-cmd report with some correction in the parsing grammar.
topology info is not required.
>> Still some issue with the cluster data which leads to segfault. Will be done by tomorrow.

It's still not working with idlestat commit (commit def498a73e7fb47efded9b41d2e91c88110ce0a9, 2014-06-01).

Like you said, it's related to the expected format:

diff --git a/idlestat.c b/idlestat.c
index 9228618a5e70..33871fff7775 100644
--- a/idlestat.c
+++ b/idlestat.c
@@ -789,7 +789,7 @@ static struct cpuidle_datas *idlestat_load(const char *path)

        do {
                if (strstr(buffer, "cpu_idle")) {
- assert(sscanf(buffer, TRACE_FORMAT, &time, &state,
+ assert(sscanf(buffer, TRACE_CMD_FORMAT, &time, &state,
                                      &cpu) == 3);

                        if (start) {
@@ -802,7 +802,7 @@ static struct cpuidle_datas *idlestat_load(const char *path)
                        count++;
                        continue;
                } else if (strstr(buffer, "cpu_frequency")) {
- assert(sscanf(buffer, TRACE_FORMAT, &time, &freq,
+ assert(sscanf(buffer, TRACE_CMD_FORMAT, &time, &freq,
                                      &cpu) == 3);
                        assert(datas->pstates[cpu].pstate != NULL);
                        cpu_change_pstate(datas, cpu, freq, time);

If I use TRACE_CMD_FORMAT instead of TRACE_FORMAT, I can evaluate ASCII trace files from trace-cmd with idlestat which is one of the reqs.

So instead of:

$ idlestat -o trace.txt
idlestat: idlestat.c:793: idlestat_load: Assertion `sscanf(buffer, "%*[^]]] %*s %lf:%*[^=]=%u%*[^=]=%d", &time, &state, &cpu) == 3' failed.

I get:

$ idlestat -o trace.txt
Log is 12.270789 secs long with 5593 events
cpu0@state hits total(us) avg(us) min(us) max(us)
     POLL 455 216970.00 476.86 3.00 9721.00
     C1-SNB 99 445857.00 4503.61 360.00 10820.00
cpu0 wakeups name count
cpu1@state hits total(us) avg(us) min(us) max(us)
     POLL 432 133254.00 308.46 14.00 11139.00
     C1-SNB 262 493110.00 1882.10 384.00 18621.00
cpu1 wakeups name count
cpu2@state hits total(us) avg(us) min(us) max(us)
     POLL 1114 117450.00 105.43 7.00 28868.00
     C1-SNB 32 409670.00 12802.19 276.00 126962.00
cpu2 wakeups name count
cpu3@state hits total(us) avg(us) min(us) max(us)
     C1-SNB 29 578758.00 19957.17 761.00 200294.00
cpu3 wakeups name count
cpu4@state hits total(us) avg(us) min(us) max(us)
     POLL 37 97635.00 2638.78 7.00 96211.00
     C1-SNB 325 428380.00 1318.09 246.00 117725.00
cpu4 wakeups name count
cluster@state hits total(us) avg(us) min(us) max(us)
        C1-SNB 30 48256.00 1608.53 0.00 9926.00

even wo/ the topology information at the beginning of the ASCII file. Not sure about the cluster data though. Wakeups are missing too and the name of the C states is also not correct.

So I don't know why idlestat needs this extra substring e.g '.n..' in
<idle>-0 [001] .n.. 5883.994620: cpu_idle: state=4294967295 cpu_id=1 ???

We do need this postprocessing feature from idlestat to work w/ trace-cmd derived output.
Why does idlestat needs a different event string? Maybe you ca...

A temp patch is on my work tree which takes a command line option -R
indicating it as a tracereport file . I will rebase and update the main git.

https://git.linaro.org/people/sandeep.tripathy/idlestat.git/blobdiff/f59a60e20a050473bf00cfa9d28cb1d591c2c913..33a4407178c08c01023da33bca6b05d1e5929a25:/idlestat.c

What you noticed is exactly what is lacking in this.

While postprocessing following are required.
 1- topology information in case of multi cluster . ( for single cluster
system the topology information is not required)
    I am yet to see that part of code, how to handle multi cluster etc.
 2- other information like c-state name / p-state / irq info etc.

which means trace-cmd has to gather these details from the target system
and append to its header.
which means header parsing logic for the trace.dat gets affected in many
tools.

On 2 June 2014 23:49, Dietmar Eggemann <email address hidden> wrote:

> It's still not working with idlestat commit (commit
> def498a73e7fb47efded9b41d2e91c88110ce0a9, 2014-06-01).
>
> Like you said, it's related to the expected format:
>
> diff --git a/idlestat.c b/idlestat.c
> index 9228618a5e70..33871fff7775 100644
> --- a/idlestat.c
> +++ b/idlestat.c
> @@ -789,7 +789,7 @@ static struct cpuidle_datas *idlestat_load(const char
> *path)
>
> do {
> if (strstr(buffer, "cpu_idle")) {
> - assert(sscanf(buffer, TRACE_FORMAT, &time, &state,
> + assert(sscanf(buffer, TRACE_CMD_FORMAT, &time,
> &state,
> &cpu) == 3);
>
> if (start) {
> @@ -802,7 +802,7 @@ static struct cpuidle_datas *idlestat_load(const char
> *path)
> count++;
> continue;
> } else if (strstr(buffer, "cpu_frequency")) {
> - assert(sscanf(buffer, TRACE_FORMAT, &time, &freq,
> + assert(sscanf(buffer, TRACE_CMD_FORMAT, &time,
> &freq,
> &cpu) == 3);
> assert(datas->pstates[cpu].pstate != NULL);
> cpu_change_pstate(datas, cpu, freq, time);
>
> If I use TRACE_CMD_FORMAT instead of TRACE_FORMAT, I can evaluate ASCII
> trace files from trace-cmd with idlestat which is one of the reqs.
>
> So instead of:
>
> $ idlestat -o trace.txt
> idlestat: idlestat.c:793: idlestat_load: Assertion `sscanf(buffer,
> "%*[^]]] %*s %lf:%*[^=]=%u%*[^=]=%d", &time, &state, &cpu) == 3' failed.
>
> I get:
>
> $ idlestat -o trace.txt
> Log is 12.270789 secs long with 5593 events
> cpu0@state hits total(us) avg(us) min(us) max(us)
> POLL 455 216970.00 476.86 3.00 9721.00
> C1-SNB 99 445857.00 4503.61 360.00 10820.00
> cpu0 wakeups name count
> cpu1@state hits total(us) avg(us) min(us) max(us)
> POLL 432 133254.00 308.46 14.00 11139.00
> C1-SNB 262 493110.00 1882.10 384.00 18621.00
> cpu1 wakeups name count
> cpu2@state hits total(us) ...

I don't think you can change trace-cmd for this.

(1) What about you provide idlestat with topology and the other information when we ask it to process an trace.txt file from
     trace-cmd? Then idlestat could be used in a cross-development env for reporting mode (a.k.a post-porcessing) too.

(2) What is the useful information when you pass a trace.txt file from trace-cmd (i.e. w/ the -R option) wo/ providing topology
      and the other information? AFAICS, there is no wakeup and no per-cluster information. Can I at least trust the per-cpu
     residency information? Why does the tool show 32 hits for state C1 for cluster?

     Example from my ARM TC2 platorm (using a patched idlestat 0.3):

     # idlestat --import -f trace_tc.txt
     Log is 11.554232 secs long with 2650 events
     cpu0@state hits total(us) avg(us) min(us) max(us)
          WFI 230 106730.00 464.04 4.00 9920.00
          C1 99 436119.00 4405.24 367.00 10936.00
          900 1 755050.00 755050.00 755050.00 755050.00
          1200 249 9427025.00 37859.54 7.00 9351644.00
     cpu0 wakeups name count
     cpu1@state hits total(us) avg(us) min(us) max(us)
          WFI 375 117911.00 314.43 31.00 32392.00
          C1 98 375363.00 3830.23 374.00 20799.00
          900 1 755048.00 755048.00 755048.00 755048.00
          1200 42 9433195.00 224599.88 26.00 9407861.00
     cpu1 wakeups name count
     ...
     cluster@state hits total(us) avg(us) min(us) max(us)
             C1 32 31402.00 981.31 0.00 7098.00