[library] Command exceeded timeout at /etc/puppet/modules/ceph/manifests/osd.pp:33

This happens on occasion and is likely the result of the older version of ceph-deploy. We should update the version of ceph-deploy or add retries = number of osd's on this system. Puppet will re-run and pass on the next iteration.

Reproduced for version 5.0.1

"build_id": "2014-07-08_13-57-45",
"mirantis": "yes",
"build_number": "107",
"ostf_sha": "09b6bccf7d476771ac859bb3c76c9ebec9da9e1f",
"nailgun_sha": "c0082e3a0e8544bad7bd45c15c5dd8632ea045b5",
"production": "docker",
"api": "1.0",
"fuelmain_sha": "b0f5151d12751b9b55dcd69bd1445d0d480012d6",
"astute_sha": "a4edb51661f50c66e247e0b8d00f2d01e0658fe6",
"release": "5.0.1",
"fuellib_sha": "d4cb36208efaf51a7c0ca012fa63d596d4ee2e29"

1. Create new environment (CentOS, simple mode)
2. Choose both Ceph
3. Choose Ceilometer
4. Add controller, compute+cinder+ceph, cinder+mongo, ceph
5. Start deployment. It was successful
6. But there is error in puppet.log on node with single ceph (node-12):

2014-07-08 14:00:31 ERR

 (/Stage[main]/Ceph::Osd/Exec[ceph-deploy osd activate]/returns) change from notrun to 0 failed: Command exceeded timeout

Ceph is alive:

[root@node-9 ~]# ceph osd tree
# id weight type name up/down reweight
-1 0.24 root default
-2 0.12 host node-10
0 0.06 osd.0 up 1
1 0.06 osd.1 up 1
-3 0.12 host node-12
2 0.06 osd.2 up 1
3 0.06 osd.3 up 1

[root@node-9 ~]# ceph -s
  cluster dd4ceee6-0ce0-4ae1-86b0-d6ae9ad10d99
   health HEALTH_OK
   monmap e1: 1 mons at {node-9=192.168.0.2:6789/0}, election epoch 1, quorum 0 node-9
   osdmap e35: 4 osds: 4 up, 4 in
    pgmap v71: 960 pgs: 960 active+clean; 14464 KB data, 8367 MB used, 245 GB / 253 GB avail; 112B/s rd, 0op/s
   mdsmap e1: 0/0/1 up

We now have a more recent ceph-deploy 1.5.9 in 5.1 so this problem should disappear. Since it's a low priority bug it will not be backported to stable release series.

Verified on ISO #436
"build_id": "2014-08-14_02-01-17",
"ostf_sha": "d2a894d228c1f3c22595a77f04b1e00d09d8e463",
"build_number": "436",
"auth_required": true,
"api": "1.0",
"nailgun_sha": "b5bdd19c2dbeb26ce3bd88270d09f5e7541a3aea",
"production": "docker",
"fuelmain_sha": "9f327045cdd72d406d89063393a499635be5e3d4",
"astute_sha": "b52910642d6de941444901b0f20e95ebbcb2b2e9",
"feature_groups": ["mirantis"],
"release": "5.1",
"fuellib_sha": "4b085bfbf7be973f0aa29d9d5e4f3ebd5bf789a1"