Fuel for OpenStack

[10.0.main.ubuntu.bvt_2][1038] Tests failed during Galera cluster recovery

Bug #1652043 reported by Ivan Udovichenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Confirmed
High
Sergii Golovatiuk
Fuel for OpenStack 10.x-updates

Bug Description

Job failed [1][2][3] with error:
"- Launch instance, create snapshot, launch instance from snapshot (failure) Image can not be booted. Please refer to OpenStack logs for more details."

Plenty of errors due to Galera cluster unavailability from OpenStack services [4].

Partial log from node-1.test.domain.local/ocf-mysql-wss.log [5].

[1] https://product-ci.infra.mirantis.net/job/10.0.main.ubuntu.bvt_2/1038/
[2] http://paste.openstack.org/show/593136/
[3] http://paste.openstack.org/show/593134/
[4] http://paste.openstack.org/show/593137/
[5] http://paste.openstack.org/show/593139/

Ivan Udovichenko (iudovichenko)
Changed in fuel:
milestone: none → 10.x-updates
Revision history for this message
Ivan Udovichenko (iudovichenko) wrote :

The issue reproduced during https://product-ci.infra.mirantis.net/job/10.0.main.ubuntu.bvt_2/1040/console run.

Keystone lost connection to MySQL server and threw an error: https://paste.mirantis.net/show/2897/

Galera cluster was recovering: https://paste.mirantis.net/show/2898/

And that's why "Task[primary-keystone/1]" failed: https://paste.mirantis.net/show/2899/

Sergii Golovatiuk (sgolovatiuk)
Changed in fuel:
status: New → Incomplete
importance: Undecided → High
assignee: nobody → Sergii Golovatiuk (sgolovatiuk)
Revision history for this message
Sergii Golovatiuk (sgolovatiuk) wrote :

The log doesn't contain neither mysqld.log nor atop log.

Changed in fuel:
status: Incomplete → Confirmed
Revision history for this message
Sergii Golovatiuk (sgolovatiuk) wrote :

It looks very similar to 1652008. I am marking as dublicate of 1652008. From what I can understand from logs, snapshot was created but mysql didn't recover during normal amount of time. However, our tests didn't check that, so they started ostf immediately after snapshot recovery, so services were not able to connect to DB.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.