Openstack HA : Provisioning is failing in mitaka , not able to connect to mysql db

Bug #1601956 reported by venu kolli
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.1
Fix Committed
Critical
amudhar
Trunk
Fix Committed
Critical
amudhar

Bug Description

Build 2740 , Mitaka .

Please find the setup details @ 10.84.24.32/33/34`0

find complete log attached.

2016-07-11 11:27:41:486887: [root@10.84.24.32] out: + echo 'SELECT 1;'
2016-07-11 11:27:41:486912: [root@10.84.24.32] out: + mysql -u root --password=0fca3cc4cd3cf1eeb17d
2016-07-11 11:27:41:486936: [root@10.84.24.32] out: + echo 'Verified connectivity to MySQL.'
2016-07-11 11:27:41:486959: [root@10.84.24.32] out: Verified connectivity to MySQL.
2016-07-11 11:27:41:486981: [root@10.84.24.32] out: + '[' init = init ']'
2016-07-11 11:27:41:487005: [root@10.84.24.32] out: ++ echo 'SELECT COUNT(*) FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME='\''heat'\'';'
2016-07-11 11:27:41:487028: [root@10.84.24.32] out: ++ mysql -u root --password=0fca3cc4cd3cf1eeb17d
2016-07-11 11:27:41:487052: [root@10.84.24.32] out: ++ tail -n+2
2016-07-11 11:27:41:487075: [root@10.84.24.32] out: + dbs=0
2016-07-11 11:27:41:488543: [root@10.84.24.32] out: + '[' 0 '!=' 0 ']'
2016-07-11 11:27:41:488614: [root@10.84.24.32] out: ++ echo 'SELECT COUNT(*) FROM mysql.user WHERE User = '\''heat'\'';'
2016-07-11 11:27:41:488668: [root@10.84.24.32] out: ++ mysql -u root --password=0fca3cc4cd3cf1eeb17d
2016-07-11 11:27:41:488719: [root@10.84.24.32] out: ++ tail -n+2
2016-07-11 11:27:41:488780: [root@10.84.24.32] out: + users=0
2016-07-11 11:27:41:492189: [root@10.84.24.32] out: + '[' 0 '!=' 0 ']'
2016-07-11 11:27:41:492243: [root@10.84.24.32] out: + '[' init = init ']'
2016-07-11 11:27:41:492272: [root@10.84.24.32] out: + echo 'Creating '\''heat'\'' database.'
2016-07-11 11:27:41:492295: [root@10.84.24.32] out: Creating 'heat' database.
2016-07-11 11:27:41:492317: [root@10.84.24.32] out: + '[' heat = nova_api ']'
2016-07-11 11:27:41:492338: [root@10.84.24.32] out: + mysql -u root --password=0fca3cc4cd3cf1eeb17d
2016-07-11 11:27:41:492358: [root@10.84.24.32] out: + cat
2016-07-11 11:27:41:492381: [root@10.84.24.32] out: + '[' init = init ']'
2016-07-11 11:27:41:495744: [root@10.84.24.32] out: + '[' c0ntrail123 '!=' heat ']'
2016-07-11 11:27:41:495798: [root@10.84.24.32] out: + echo 'Updating '\''heat'\'' database password in /etc/heat/heat.conf'
2016-07-11 11:27:41:495826: [root@10.84.24.32] out: Updating 'heat' database password in /etc/heat/heat.conf
2016-07-11 11:27:41:495852: [root@10.84.24.32] out: + '[' heat = nova_api ']'
2016-07-11 11:27:41:495875: [root@10.84.24.32] out: + sed -i -e 's|^#* *\(\(sql_\)*connection *= *mysql://heat\):.*@|\1:c0ntrail123@|' /etc/heat/heat.conf
2016-07-11 11:27:41:495897: [root@10.84.24.32] out: + echo 'Initializing the heat database, please wait...'
2016-07-11 11:27:41:499235: [root@10.84.24.32] out: Initializing the heat database, please wait...
2016-07-11 11:27:41:499287: [root@10.84.24.32] out: + '[' heat = nova ']'
2016-07-11 11:27:41:499317: [root@10.84.24.32] out: + '[' heat = cinder ']'
2016-07-11 11:27:41:499348: [root@10.84.24.32] out: + '[' heat = nova_api ']'
2016-07-11 11:27:41:499368: [root@10.84.24.32] out: + db_sync=db_sync
2016-07-11 11:27:41:499389: [root@10.84.24.32] out: + '[' 0 -eq 1 ']'
2016-07-11 11:27:41:499411: [root@10.84.24.32] out: + '[' 1 -eq 1 ']'
2016-07-11 11:27:41:499432: [root@10.84.24.32] out: + '[' heat = nova_api ']'
2016-07-11 11:27:41:499452: [root@10.84.24.32] out: + heat-manage db_sync
2016-07-11 11:27:41:499472: [root@10.84.24.32] out: 2016-07-11 11:27:43.954 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -1 attempts left.
2016-07-11 11:27:43:968345: [root@10.84.24.32] out: 2016-07-11 11:27:51.952 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -2 attempts left.
2016-07-11 11:27:51:987285: [root@10.84.24.32] out: 2016-07-11 11:28:04.952 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -3 attempts left.
2016-07-11 11:28:04:976125: [root@10.84.24.32] out: 2016-07-11 11:28:17.977 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -4 attempts left.
2016-07-11 11:28:18:007141: [root@10.84.24.32] out: 2016-07-11 11:28:24.980 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -5 attempts left.
2016-07-11 11:28:24:987355: [root@10.84.24.32] out: 2016-07-11 11:28:31.980 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -6 attempts left.
2016-07-11 11:28:32:003591: [root@10.84.24.32] out: 2016-07-11 11:28:39.982 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -7 attempts left.
2016-07-11 11:28:39:988578: [root@10.84.24.32] out: 2016-07-11 11:28:47.981 15783 WARNING oslo_db.sqlalchemy.engines [-] SQL connection failed. -8 attempts left.

Tags: blocker ha sanity
Revision history for this message
venu kolli (vkolli) wrote :
Revision history for this message
amudhar (amudha) wrote :

Fixing through bug #1582932 (tracking bug for mitaka support).

https://review.opencontrail.org/#/c/21894/ is the review for setting api_database in nova.conf

Revision history for this message
Ritam Gangopadhyay (ritam) wrote :

Can you please checkin a fix for R3.1 branch as well, this review is for mainline https://review.opencontrail.org/#/c/21894/

information type: Proprietary → Public
tags: added: blocker
Revision history for this message
amudhar (amudha) wrote :

Fix committed into R3.1 through https://review.opencontrail.org/22032 (#1582932).

Jeba Paulaiyan (jebap)
tags: added: ha sanity
Revision history for this message
venu kolli (vkolli) wrote :

Issue is still present with the way haproxy is started

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.