networking-odl

Need pre/post test hooks for tempest

Bug #1672620 reported by Vishal Thapar
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
networking-odl
Triaged
Wishlist
rajiv
tempest
Invalid
Undecided
Unassigned

Bug Description

Tempest tests are run as part of ODL CSIT. For all ODL CSIT test cases we capture some debug information at end of every test which makes it easier to troubleshoot any failures. However, we can't do the same for tempest tests. Only information we can capture is at end of entire suite. This makes it very difficult to troubleshoot sporadic failures. We can't even confirm if it was a failure to do cleanup correctly.

We need pre/post test hooks in tempest where we can run custom commands/scripts to capture debug information before/after every individual test.

Reference: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-gate-stateful-carbon/378/archives/robot-plugin/log.html.gz#s1-s2

Isaku Yamahata (yamahata)
tags: added: rfe
rajiv (rajiv-kumar)
Changed in networking-odl:
assignee: nobody → rajiv (rajiv-kumar)
Revision history for this message
rajiv (rajiv-kumar) wrote :

More clarity on use case

The problem we have today is that if 1 of these 244 fails, I'd like to capture some information(flows on OVS, data on ODL etc.) at that point. but we don't get control back till entire suite including the teardown/cleanup is done. if we can get a hook to capture information in case of failure, before teardown is done, that is also good enough.

Revision history for this message
rajiv (rajiv-kumar) wrote :

I had discusion with tempest guy, he suggested one of the approach is to run each tempest test case individsually and run pre and post hook scripts. It can have a huge maintaince cost, as more test cases get's added or some of them deleted etc, we have to keep track of all of them. For other approaches stay tuned for the updates on this bug :)

Isaku Yamahata (yamahata)
Changed in networking-odl:
importance: Undecided → Wishlist
Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

yea from Tempest side, it is very costly to provide pre/post hook. To catch the info i feel debug logs should be there. is there any other information you want to capture than that can be presented in logs?

Depends on actual requirement we can check the best way.

Ghanshyam Mann (ghanshyammann)
Changed in tempest:
status: New → Incomplete
Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

Got to know we have pause cleanup feature
- second last bullet in New Features

https://docs.openstack.org/releasenotes/tempest/unreleased.html#id6

You need to make this flag true and analyze the data. will it help?

 - https://github.com/openstack/tempest/blob/6c562fbc01263eca890c4f83454a897d83f5bad3/tempest/config.py#L1083

Revision history for this message
Andrea Frittoli (andrea-frittoli) wrote :

If you are writing your own tests you can use setUp and tearDown to collect more information.

If this is about existing tests I would like to understand what kind of log you are missing.
It may be something interesting to implement in for OpenStack CI as well.

We try to keep our tests debuggable as much as possible since we run them a lot in the gate and it would be a nightmare otherwise to debug failures.

Sama Madhu Sudhan Reddy (smsreddy)
Changed in tempest:
assignee: nobody → Sama Madhu Sudhan Reddy (smsreddy)
Mike Kolesnik (mkolesni)
Changed in networking-odl:
status: New → Triaged
Revision history for this message
Doug Schveninger (ds6901) wrote :

If tempest logging is at debug level the restclient will log all api req- id an you can gather them and look at the server logs to see what is happening. We have a subunit tool that gather them and creates an html report the link in to our kibana instance. Just a suggestion to help to debug when tempest test fails.

https://github.com/openstack/tempest/blob/master/tempest/lib/common/rest_client.py

Revision history for this message
Martin Kopec (mkopec) wrote :

removing assignee as there has been no activity for more than a year

Changed in tempest:
assignee: Sama Madhu Sudhan Reddy (smsreddy) → nobody
Revision history for this message
Martin Kopec (mkopec) wrote :

This has been in Incomplete state for several years, it's time to mark this as Invalid.

Changed in tempest:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.