Brand New IndexError when recovering from a halted test run

Zygmunt wanted a new bug, so here ya go :)

I have the following installed on a test system...

To reproduce this every time, I do the following:
$ canonical-certification-server

If prompted to recover I say no (doesn't happen on the first run)

while c-c-s is doing the job definition thing, halt with ctrl-c

restart c-c-s and when prompted to resume, say Yes. You should get something that looks like this:
ubuntu@bailey:/etc/apt/sources.list.d$ canonical-certification-server
Do you want to resume the previous session? [y, n] y
==============================[ Running All Jobs ]==============================
==================================[ Results ]===================================
2013.com.canonical.certification::__benchmarks__: pass
2013.com.canonical.certification::__cpu__: pass
2013.com.canonical.certification::__info__: pass
2013.com.canonical.certification::__miscellanea__: pass
2013.com.canonical.certification::benchmarks/disk/hdparm-cache-read: pass
2013.com.canonical.certification::benchmarks/disk/hdparm-read: pass
2013.com.canonical.certification::device: pass
2013.com.canonical.certification::info/hdparm: pass
2013.com.canonical.certification::optical_drive: not-supported
2013.com.canonical.certification::package: pass
Executable 'checkbox' invoked with Namespace(c3_url='https://certification.canonical.com/submissions/submit/', check_config=False, command=<checkbox_ng.commands.certification.CertificationCommand object at 0x7f9ea8324710>, debug_console=False, debug_interrupt=False, exclude_pattern_list=[], include_pattern_list=[], log_level=None, not_interactive=False, pdb=False, providers=None, secure_id='a00D000000T8H3mIAF', trace=[], whitelist=[]) has crashed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 513, in dispatch_and_catch_exceptions
    return self.dispatch_command(ns)
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 509, in dispatch_command
    return ns.command.invoked(ns)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/certification.py", line 132, in invoked
    self.settings, ns).run()
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/cli.py", line 596, in run
    self._run_jobs(ns, manager)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/cli.py", line 648, in _run_jobs
    self.save_results(manager)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/certification.py", line 51, in save_results
    super().save_results(session)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/cli.py", line 704, in save_results
    exporter.dump(data_subset, stream)
  File "/usr/lib/python3/dist-packages/plainbox/impl/exporter/xml.py", line 182, in dump
    root = self.get_root_element(data)
  File "/usr/lib/python3/dist-packages/plainbox/impl/exporter/xml.py", line 205, in get_root_element
    self._add_hardware(root, data)
  File "/usr/lib/python3/dist-packages/plainbox/impl/exporter/xml.py", line 263, in _add_hardware
    for i in range(int(data["resource_map"]["{}cpuinfo".format(self.NS)][0]["count"])):
IndexError: list index out of range
Traceback (most recent call last):
  File "/usr/bin/canonical-certification-server", line 9, in <module>
    load_entry_point('checkbox-ng==0.1', 'console_scripts', 'canonical-certification-server')()
  File "/usr/lib/python3/dist-packages/checkbox_ng/main.py", line 192, in cert_server
    CertificationNGTool().main(['certification-server'] + args))
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 291, in main
    return self.dispatch_and_catch_exceptions(ns)
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 513, in dispatch_and_catch_exceptions
    return self.dispatch_command(ns)
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 509, in dispatch_command
    return ns.command.invoked(ns)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/certification.py", line 132, in invoked
    self.settings, ns).run()
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/cli.py", line 596, in run
    self._run_jobs(ns, manager)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/cli.py", line 648, in _run_jobs
    self.save_results(manager)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/certification.py", line 51, in save_results
    super().save_results(session)
  File "/usr/lib/python3/dist-packages/checkbox_ng/commands/cli.py", line 704, in save_results
    exporter.dump(data_subset, stream)
  File "/usr/lib/python3/dist-packages/plainbox/impl/exporter/xml.py", line 182, in dump
    root = self.get_root_element(data)
  File "/usr/lib/python3/dist-packages/plainbox/impl/exporter/xml.py", line 205, in get_root_element
    self._add_hardware(root, data)
  File "/usr/lib/python3/dist-packages/plainbox/impl/exporter/xml.py", line 263, in _add_hardware
    for i in range(int(data["resource_map"]["{}cpuinfo".format(self.NS)][0]["count"])):
IndexError: list index out of range

interestingly, on a subsequent run, I hit ctrl-c at the welcome screen. THEN I restarted, and said Yes to resume. THIS TIME, c-c-s just exited:
ubuntu@bailey:/etc/apt/sources.list.d$ canonical-certification-server
Do you want to resume the previous session? [y, n] y
==============================[ Running All Jobs ]==============================
==================================[ Results ]===================================

Saving submission file to /home/ubuntu/.local/share/plainbox/submission.xml
View results (HTML): file:///home/ubuntu/.local/share/plainbox/results.html
Submitting results to https://certification.canonical.com/submissions/submit/ for secure_id a00D000000T8H3mIAF
Successfully sent, submission status at https://certification.canonical.com/submissions/status/8183

on the third attempt, I allowed it to get to the test selection screen AFTER job definitions were done. Again, I hit ctrl-c on this screen and then 'y' when asked to resume the next time I ran c-c-s. When I did this, I got the basically the same trace as above.

Finally the forth attempt, I let it actually start running tests.
this time it succeeded.

ubuntu@bailey:/etc/apt/sources.list.d$ canonical-certification-server
Do you want to resume the previous session? [y, n] y
We have previously tried to execute 2013.com.canonical.certification::cpu/frequency_governors
What do you want to do with that job? [skip, fail, run] r
What do you want to do with that job? [skip, fail, run] run
==============================[ Running All Jobs ]==============================
------------------------------------[ None ]------------------------------------
This test checks that CPU frequency governors are obeyed when set.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Running... (output in /home/ubuntu/.cache/plainbox/sessions/pbox-akusbjt1.session/2013.com.canonical.certification__cpu_frequency_governors.*)

so as long as you're recovering after tests have begun, you're ok. It's recovering elsewhere that you run into issues.