Fuel for OpenStack

[systest] HA flat add compute - step wait for ha proxy keystone backend failed

Bug #1352964 reported by Tatyanka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Bogdan Dobrelya
Fuel for OpenStack 5.1
5.0.x
Fix Released
High
Sergii Golovatiuk
Fuel for OpenStack 5.0.2

Bug Description

Latest one:
Reproduced on Centos HA flat add compute
with 504 ERROR from keystone on auth
http://paste.openstack.org/show/93826/

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "424"
  build_id: "2014-08-12_02-01-17"
  astute_sha: "b52910642d6de941444901b0f20e95ebbcb2b2e9"
  fuellib_sha: "54d834afccf1f8a97e4a82d7d081e7d1a1e068a1"
  ostf_sha: "d2a894d228c1f3c22595a77f04b1e00d09d8e463"
  nailgun_sha: "c7e00e5a00499d9f8dec608541dad1d745a8dd2e"
  fuelmain_sha: "9d4463400b4924159c978af43855e48bcf2a84b2"

http://jenkins-product.srt.mirantis.net:8080/view/0_master_swarm/job/master_fuelmain.system_test.centos.thread_4/131/consoleFull

P.S. Snapshot was updated with latest result. Failed node - node-1
Also /etc/init.d status says that keystone is running but
. openrc and manual exution of nova list command showw the next result
http://paste.openstack.org/show/93827/
keystone tenenat-list command result
http://paste.openstack.org/show/93828/

Previous fail:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "389"
  build_id: "2014-08-03_02-01-14"
  astute_sha: "ce86172e77661026c91fdf1ff8066d7df1f7d89d"
  fuellib_sha: "4e3fdd75f8dabde8e5d07067545d8043a70a176b"
  ostf_sha: "a3fa823ea0e4e03beb637ae07a91adea82c33182"
  nailgun_sha: "bd0127be0061029f9f910547db5e633c82244942"
  fuelmain_sha: "e99879292cf6e96b8991300d947df76b69134bb1"
Test ha flat add compute fail on ubuntu with next error in puppet on controller node

http://paste.openstack.org/show/90540/

See original description
Tags: puppet
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
summary: [systest] simple flat add compute - step wait for ha proxy keystone
- client failed
+ backend failed
Revision history for this message
Nastya Urlapova (aurlapova) wrote :

Looks like invalid, ha_flat_add_compute passed tree time on centos|ubuntu

summary: - [systest] simple flat add compute - step wait for ha proxy keystone
- backend failed
+ [systest] HA flat add compute - step wait for ha proxy keystone backend
+ failed
description: updated
Changed in fuel:
status: New → Invalid
Revision history for this message
Dmitry Borodaenko (angdraug) wrote :

ISO #389 didn't include https://review.openstack.org/111716

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

Dmitry, Nastya I do not agree that it is invalid/duplicate. We can see that it is reproduced on CI but we fail not with 401 error we failed with 504 error (and I beleive it is not client error and we should to be sure that everithing fine with server) - so please, take a look at the updated description and snapshots

Changed in fuel:
status: Invalid → Confirmed
Tatyanka (tatyana-leontovich)
description: updated
Bogdan Dobrelya (bogdando)
Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Bogdan Dobrelya (bogdando)
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

2014-08-03T18:25:24.042159+01:00 info: (/Stage[main]/Osnailyfacter::Cluster_ha/Exec[wait-for-haproxy-keystone-backend]) Starting to evaluate the resource
...
2014-08-03T18:38:03.047263+01:00 notice: (/Stage[main]/Openstack::Ha::Keystone/Openstack::Ha::Haproxy_service[keystone-1]/Haproxy::Listen[keystone-1]/Haproxy::Service[keystone-1]/Concat[/etc/haproxy/conf.d/020-keystone-1.cfg]/Exec[concat_/etc/haproxy/conf.d/020-keystone-1.cfg]) Triggered 'refresh' from 4 events

rc: haproxy backend service triggered after its evaluation.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/113870

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/113870
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=bd89083f75b37c1119b8eb7e6e12103d4c68b6aa
Submitter: Jenkins
Branch: master

commit bd89083f75b37c1119b8eb7e6e12103d4c68b6aa
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Aug 13 15:52:18 2014 +0300

    Fix haproxy backend checks dependencies

    Openstack::Ha::Haproxy_service['keystone*']
    should always be evaluated *before* any
    Exec['wait-for-keystone*'] backend checks.

    Closes-bug: #1352964

    Change-Id: I579f33d673925bea27ceab48f3380ed1dda40170
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/5.0)

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/114508

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/5.0)

Reviewed: https://review.openstack.org/114508
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=70f0f0f1ff0f6547b1d70e1206d9cbacf3d704d3
Submitter: Jenkins
Branch: stable/5.0

commit 70f0f0f1ff0f6547b1d70e1206d9cbacf3d704d3
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Aug 13 15:52:18 2014 +0300

    Fix haproxy backend checks dependencies

    Openstack::Ha::Haproxy_service['keystone*']
    should always be evaluated *before* any
    Exec['wait-for-keystone*'] backend checks.

    Closes-bug: 1352964

    Change-Id: I579f33d673925bea27ceab48f3380ed1dda40170
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Tatyanka (tatyana-leontovich)
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.