sporadic rally failures and ssl errors with pike, even though ssl isn't enabled

Bug #1790635 reported by Jason Hobbs
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Keystone Charm
Triaged
Medium
Unassigned

Bug Description

A deployment of pike using the current *next charms is completing, but failing sporadically during rally testing. For example:

http://paste.ubuntu.com/p/YMxjD3r7ws/

This is unusual; we haven't seen this failure mode before. In keystone logs we see:

(keystoneclient.common.cms): 2018-09-03 02:34:31,131 ERROR Signing error: Unable to load certificate - ensure you have configured PKI with "keystone-manage pki_setup"
(keystone.common.wsgi): 2018-09-03 02:34:31,132 ERROR Command 'openssl' returned non-zero exit status 3

bundle:
http://paste.ubuntu.com/p/KmwD9N3zm8/

Revision history for this message
Jason Hobbs (jason-hobbs) wrote :
summary: - ERROR Signing error: Unable to load certificate - ensure you have
- configured PKI with "keystone-manage pki_setup"
+ sporadic rally failures and ssl errors with pike
description: updated
summary: - sporadic rally failures and ssl errors with pike
+ sporadic rally failures and ssl errors with pike, even though ssl isn't
+ enabled
Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :

The apache error logs on the keystone units are likely important:

[Mon Sep 03 00:27:26.613789 2018] [mpm_event:notice] [pid 60759:tid 140693993224064] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 Python/2.7.12 configured -- resuming normal operations
[Mon Sep 03 00:27:26.613816 2018] [core:notice] [pid 60759:tid 140693993224064] AH00094: Command line: '/usr/sbin/apache2'
[Mon Sep 03 00:34:39.198802 2018] [mpm_event:notice] [pid 60759:tid 140693993224064] AH00491: caught SIGTERM, shutting down
[Mon Sep 03 00:34:40.424561 2018] [wsgi:warn] [pid 133226:tid 139642871338880] mod_wsgi: Compiled for Python/2.7.11.
[Mon Sep 03 00:34:40.424618 2018] [wsgi:warn] [pid 133226:tid 139642871338880] mod_wsgi: Runtime using Python/2.7.12.
[Mon Sep 03 00:34:40.429273 2018] [mpm_event:notice] [pid 133226:tid 139642871338880] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 Python/2.7.12 configured -- resuming normal operations
[Mon Sep 03 00:34:40.429294 2018] [core:notice] [pid 133226:tid 139642871338880] AH00094: Command line: '/usr/sbin/apache2'
[Mon Sep 03 00:38:55.677239 2018] [mpm_event:notice] [pid 133226:tid 139642871338880] AH00491: caught SIGTERM, shutting down
[Mon Sep 03 00:38:56.828834 2018] [wsgi:warn] [pid 189383:tid 140551433283456] mod_wsgi: Compiled for Python/2.7.11.
[Mon Sep 03 00:38:56.828898 2018] [wsgi:warn] [pid 189383:tid 140551433283456] mod_wsgi: Runtime using Python/2.7.12.

Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :

The BadStatusLine error in the pastebin is because of a broken socket, and if something is causing apache to constantly restart then this would be expected.

Revision history for this message
Jason Hobbs (jason-hobbs) wrote :

rally output

Changed in charm-keystone:
status: New → Triaged
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.