[centos] mcollective autostart is not ensured on nodes

Bug #1443291 reported by Bogdan Dobrelya
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Critical
Alex Schultz
5.0.x
Invalid
Critical
Unassigned
5.1.x
Invalid
Critical
Unassigned
6.0.x
Won't Fix
Medium
Unassigned
6.1.x
Fix Released
Critical
Alex Schultz

Bug Description

# chkconfig | grep mcollective
mcollective 0:off 1:off 2:off 3:off 4:off 5:off 6:off

This results in nodes became unmanaged for Fuel orchestration after reboot.
Reproduced at 6.1 dev ISO #291

Changed in fuel:
importance: Undecided → Critical
milestone: none → 6.1
assignee: nobody → Fuel Library Team (fuel-library)
status: New → Confirmed
tags: added: low-hanging-fruit
Revision history for this message
Alex Schultz (alex-schultz) wrote :

So I looked into this bug on 6.1 and while chkconfig reports mcollective as being off, if you reboot and mcollectived is being started. Are there additional steps required to reboot and result in mcollective in not running?

As far as I can tell, it is being started via rc.local which calls fix-configs-on-startup.

Revision history for this message
Alex Schultz (alex-schultz) wrote :

In looking further, the bootstrap centos starts mcollective via the fix-configs-on-startup but if you provision an actual centos environment it does not start mcollective. I'm also not sure if the ubuntu version starts mcollective either, going to continue to dig deeper.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

There is no such entry in /etc/rc.local at my Centos Fuel 6.1 lab:

# cat /etc/rc.local
flock -w 0 -o /var/lock/agent.lock -c "/opt/nailgun/bin/agent >> /var/log/nailgun-agent.log 2>&1"

Here is init scripts and runlevels details: http://pastebin.com/7HMnZTPX
I guess, it works on ubuntu due to LSB headers - there are matching S/K levels in /etc/rcX.d

As for centos, chckonfig header is ok, but chkonfig is not configured by puppet, so we should re-check out puppet manifests, do they ensure mcollective service enabled?

Note: as I reported, mcollective is running OK after reboot on Ubuntu, and stopped - at Centos

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

We have mcollective service enabled in puppet manifests https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/mcollective/manifests/server.pp#L62.

And if I run puppet apply -e "service { 'mcollective': ensure=>running, enable=>true }" at my lab, chkconfig gets runlevels configured. That is really strange why this don't work at deploy stage...

Revision history for this message
Alex Schultz (alex-schultz) wrote :

I found that mcollective is started via the cloud-init process post install and configured there as well. As far as I can tell we aren't actively managing mcollective via puppet as part of the deployment. I'm working on a fix to the fuel-agent cloud-init configuration to address this for centos.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/174005

Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Alex Schultz (alex-schultz)
status: Confirmed → In Progress
Changed in fuel:
assignee: Alex Schultz (alex-schultz) → Dmitry Pyzhov (dpyzhov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/174005
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=5a4556ea0cf943edffa33788994b7fb7abe150b3
Submitter: Jenkins
Branch: master

commit 5a4556ea0cf943edffa33788994b7fb7abe150b3
Author: Alex Schultz <email address hidden>
Date: Wed Apr 15 12:46:45 2015 -0500

    Ensure mcollective starts on boot for CentOS

    Change-Id: I7963dac00795245eb32ec9ad6e57ff7220fd95a4
    Closes-Bug: 1443291

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Anastasia Palkina (apalkina) wrote :

Verified on ISO #329

"build_id": "2015-04-20_22-54-31", "ostf_sha": "df8db1f48f03b18126ce5ec65317a1eb83a5a95f", "build_number": "329", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-04-20_22-54-31", "ostf_sha": "df8db1f48f03b18126ce5ec65317a1eb83a5a95f", "build_number": "329", "api": "1.0", "nailgun_sha": "69fca3bada96a48ab19b969621e5128390b7052e", "openstack_version": "2014.2-6.1", "production": "docker", "python-fuelclient_sha": "b4f1ddffd5263489090b65e662173e9e11aafd94", "astute_sha": "bf1751a4fe0d912325e3b4af629126a59c1b2b51", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "0c0e299dcc0276bb8262256148c6ee975857ee86", "fuellib_sha": "b878f1ae8e1d4c6f2f463e810b1f8014df5b22a5"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "69fca3bada96a48ab19b969621e5128390b7052e", "openstack_version": "2014.2-6.1", "production": "docker", "python-fuelclient_sha": "b4f1ddffd5263489090b65e662173e9e11aafd94", "astute_sha": "bf1751a4fe0d912325e3b4af629126a59c1b2b51", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "0c0e299dcc0276bb8262256148c6ee975857ee86", "fuellib_sha": "b878f1ae8e1d4c6f2f463e810b1f8014df5b22a5"

[root@node-6 ~]# chkconfig | grep mcollective
mcollective 0:off 1:off 2:on 3:on 4:on 5:on 6:off

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.