kolla

if /run/keepalived.pid file exists, keepalive wont start

Bug #1485079 reported by Steven Dake
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Duane DeCapite
kolla liberty-3 "liberty-3"

Bug Description

See summary. If the containers fail for some reason or cleanup-containers -fv is fun, keepalive will no longer start. One easy solution is to remove the pid file for keepalived. We don't really need it. ATM we specify it as a config option.

Revision history for this message
Steven Dake (sdake) wrote :

This wouldn't be such a problem but the location where the pid file is stored is bindmounted to /run:/run.

Changed in kolla:
status: New → Confirmed
importance: Undecided → Critical
milestone: none → liberty-3
Revision history for this message
Steven Dake (sdake) wrote :

My preferred solution to this problem is to remove /run:/run as bindmounts, and store /run as a data container shared between haproxy and keepalived.

Sam Yaple (s8m)
Changed in kolla:
assignee: nobody → Sam Yaple (s8m)
Revision history for this message
Steven Dake (sdake) wrote :

note even sharing /run in comment #2, pid file still needs removal.

Steven Dake (sdake)
Changed in kolla:
assignee: Sam Yaple (s8m) → Duane DeCapite (ddecapit)
Revision history for this message
Steven Dake (sdake) wrote :

Apparently this only happens after an unclean shutdown of the container, (such as a crash or container-cleanup -fv). Unclear if we can remove the --pid flag to keepalived and still have a working system in the cases of HA. Recommend testing if the file exists, and if it does, delete it at container start time.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/215380

Changed in kolla:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/215380
Committed: https://git.openstack.org/cgit/stackforge/kolla/commit/?id=129f1d06a141c75643336399181ea157fb0dd82a
Submitter: Jenkins
Branch: master

commit 129f1d06a141c75643336399181ea157fb0dd82a
Author: Duane DeCapite <email address hidden>
Date: Thu Aug 20 19:51:57 2015 -0400

    Workaround for existing keepalived.pid file

    Fixes issue where keepalived will not restart after container failure.
    Removes keepalived.pid file if one exists on container startup.

    Change-Id: I6ec3297a5e49a4d5255323aa7f75dfdc10d0c5ec
    Closes-Bug: #1485079

Changed in kolla:
status: In Progress → Fix Committed
Sam Yaple (s8m)
Changed in kolla:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.