| 2016-05-13 10:08:20 |
Bogdan Dobrelya |
description |
Fuel integration testing lacks recovery/self-heal coverage for clusters, which is to check network partitions tolerance and post-recovery behavior.
The idea is to run integration tests of that type either against a several components, like Corosync/Pacemaker/DB/MQ, or a whole deployment done. And with leveraging either existing frameworks like Jepsen's Nemesis or written ground up in Fuel QA codebase as well.
I provided example approach with Jepsen [0]. Details may be read there as well.
This work allowed me to discover ~10 high bugs related to DB cluster deployments and several bugs to MQ. You can find the list of DB bugs in this backport patch (just gives a nice view) [1].
[0] https://goo.gl/VHyIIE
[1] https://review.openstack.org/#/c/315989/ |
Fuel integration testing lacks one important sub-type of destructive testing to provide recovery/self-heal coverage for clusters, which is to check network partitions tolerance and post-recovery behavior.
The idea is to run integration tests of that type either against a several components, like Corosync/Pacemaker/DB/MQ, or a whole deployment done. And with leveraging either existing frameworks like Jepsen's Nemesis or written ground up in Fuel QA codebase as well.
I provided example approach with Jepsen [0]. Details may be read there as well.
This work allowed me to discover ~10 high bugs related to DB cluster deployments and several bugs to MQ. You can find the list of DB bugs in this backport patch (just gives a nice view) [1].
[0] https://goo.gl/VHyIIE
[1] https://review.openstack.org/#/c/315989/ |
|
