tempest

Need policy-awared test

Bug #1817811 reported by Yang Youseok
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tempest
New
Undecided
Unassigned

Bug Description

Hi, tempest team!

We are using rally + tempest quite a while and one problem that we have to deal with is to maintain skip-list which was failed due to our custom policy.

For example, our Neutron policy does not allow network creation by normal user. (Only admin can control about it). There were few more other things that we customized and every test cases that related them were failed.

Our approach is to maintain skip list for the failed cases which results from our customized policy, but I felt it's getting harder to maintain those lists.

I think if tempest can aware of policy which is different from default configuration and skip the related list, it's much better to run tests.

I wonder how other operators handle of this problem since I think changing a policy is quite common.

Thanks!

Revision history for this message
Doug Schveninger (ds6901) wrote :

see this bug and comment if you fell that this will help this bug. https://bugs.launchpad.net/tempest/+bug/1738076

Revision history for this message
Doug Schveninger (ds6901) wrote :

We handle this by creating a qa domain and running our tests with preprovision creds that we create the project with uuid for the test run. We give all creds admin roles for the test even if they are for the admin,primary or alt cred groups. When the test are done we delete the project and roles assignments. Let me know if this helps and if you want to talk more about this IRC bigdogstl.

Revision history for this message
Paras Babbar (pbabbar) wrote :

But how you are dealing with non admin tempest test cases that needs to be executed as non admin users? are they passing as well??

Revision history for this message
Doug Schveninger (ds6901) wrote :

A little more detail.

We had over 150 test cases failed due under-permission to our elevated RBAC policy we are using. We use preprovision creds so in the accounts.yaml we tell tempest that we have admin and other (Member in Devstack), for primary and alt but Keystone knows them all as admin only. With this over-permission direction we have about 80 test cases that are failing on test cases that are seeing items that should not in teh functional test cases .

We would like to submit a spec to change the cred provide in tempest to support 3 hash maps, admin, primary and other, over the current 2, admin and other.

I will try to make the Office hours this week to see what other interest exists in this area. https://wiki.openstack.org/wiki/Meetings/QATeamMeeting

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.