docker.service and docker-storage-setup.service services failed to start at fedora-21-atomic-5

Bug #1500062 reported by Egor Guz
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Magnum
New
Undecided
Unassigned

Bug Description

● docker-storage-setup.service - Docker Storage Setup
   Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; enabled)
   Active: failed (Result: exit-code) since Sat 2015-09-26 06:41:31 UTC; 1min 16s ago
  Process: 840 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE)
 Main PID: 840 (code=exited, status=1/FAILURE)

Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal docker-storage-setup[840]: Volume group "mapper" not found
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal docker-storage-setup[840]: ERROR: Old mode of passing data and metadata logical volumes to...ting.
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: docker-storage-setup.service: main process exited, code=exited, status=1/FAILURE
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: Failed to start Docker Storage Setup.
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: Unit docker-storage-setup.service entered failed state.
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: docker-storage-setup.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)
  Drop-In: /usr/lib/systemd/system/docker.service.d
           └─flannel.conf
   Active: failed (Result: exit-code) since Sat 2015-09-26 06:41:31 UTC; 1min 39s ago
     Docs: http://docs.docker.com
  Process: 852 ExecStart=/usr/bin/docker -d $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $INSECURE_REGISTRY (code=exited, status=1/FAILURE)
 Main PID: 852 (code=exited, status=1/FAILURE)

Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal docker[852]: time="2015-09-26T06:41:31.736762771Z" level=fatal msg="Error starting daemon...dm.fs"
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: Failed to start Docker Application Container Engine.
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: Unit docker.service entered failed state.
Sep 26 06:41:31 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: docker.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
bash-4.3#

Revision history for this message
Egor Guz (eghobo) wrote :

adding '--storage-driver devicemapper' to /etc/sysconfig/docker-storage, will fix the problem with docker.service.
but still failing

● docker-storage-setup.service - Docker Storage Setup
   Loaded: loaded (/usr/lib/systemd/system/docker-storage-setup.service; enabled)
   Active: failed (Result: exit-code) since Sat 2015-09-26 06:49:04 UTC; 5s ago
  Process: 1090 ExecStart=/usr/bin/docker-storage-setup (code=exited, status=1/FAILURE)
 Main PID: 1090 (code=exited, status=1/FAILURE)

Sep 26 06:49:04 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal docker-storage-setup[1090]: Volume group "mapper" not found
Sep 26 06:49:04 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal docker-storage-setup[1090]: ERROR: Old mode of passing data and metadata logical volumes to...ing.
Sep 26 06:49:04 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: docker-storage-setup.service: main process exited, code=exited, status=1/FAILURE
Sep 26 06:49:04 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: Failed to start Docker Storage Setup.
Sep 26 06:49:04 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: Unit docker-storage-setup.service entered failed state.
Sep 26 06:49:04 k8-vatrac7klp-0-saqpl7e33zr5-kube-master-crxqnj5zaioh.novalocal systemd[1]: docker-storage-setup.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

see details https://github.com/projectatomic/docker-storage-setup/blob/master/docker-storage-setup.sh

Revision history for this message
Egor Guz (eghobo) wrote :

adding /etc/sysconfig/docker-storage-setup
STORAGE_DRIVER=devicemapper
VG=atomicos

still failing
Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal docker-storage-setup[837]: Volume group "mapper" not found
  Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal docker-storage-setup[837]: Rounding up size to full physical extent 8.00 MiB
  Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal docker-storage-setup[837]: Volume group "atomicos" has insufficient free space (0 extents)...ired.
  Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal systemd[1]: docker-storage-setup.service: main process exited, code=exited, status=5/NOTINSSTALLED
  Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal systemd[1]: Failed to start Docker Storage Setup.
  Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal systemd[1]: Unit docker-storage-setup.service entered failed state.
  Sep 26 06:32:44 k8-mkoxrpb5ko-0-4zn5yga2zmpv-kube-master-pdvycqegmb7x.novalocal systemd[1]: docker-storage-setup.service failed.

need to check, with Ton about partition size

Adrian Otto (aotto)
Changed in magnum:
milestone: none → mitaka-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to magnum (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/267225

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/267996

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to magnum (master)

Reviewed: https://review.openstack.org/267225
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=0e7ba1e1a0df8812735f8fcb0f69502ce86227d8
Submitter: Jenkins
Branch: master

commit 0e7ba1e1a0df8812735f8fcb0f69502ce86227d8
Author: Egor Guz <email address hidden>
Date: Thu Jan 7 20:42:57 2016 -0800

    Increase size of Docker volume size

    Add logs for docker and docker-storage-setup services.
    docker-storage-setup uses MIN_DATA_SIZE=2G for Docker volume, so we
    cannot use Docker volume less that 2G (realistic 3G, because some
    space should be allocatedtd for medata-data)

    Related-Bug: #1500062

    Change-Id: I7a0c0023c164f890b492035c05efdb5ce32ef9f9

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/267996
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=ae817454c9e95afea82b525edda7a625caad4d5a
Submitter: Jenkins
Branch: master

commit ae817454c9e95afea82b525edda7a625caad4d5a
Author: Egor Guz <email address hidden>
Date: Fri Jan 8 00:56:50 2016 -0800

    Fix Docker storage configuration for Swarm

    Fix incorrect Docker storage configuration at agent nodes which cause
    failure of docker-storage-setup.service. Add Cinder volume to the
    master and configure Docker DeviceMapper to use them and set proper
    volume sizes for integration tests

    Change-Id: I57dfd6174143f8c7563db2920dd4e94b4f3c9883
    Related-Bug: #1500062

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to magnum (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/270634

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to magnum (master)

Reviewed: https://review.openstack.org/270634
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=ca34c5d813e00e4160f5651ca51fb75356670404
Submitter: Jenkins
Branch: master

commit ca34c5d813e00e4160f5651ca51fb75356670404
Author: Egor Guz <email address hidden>
Date: Wed Jan 20 23:12:58 2016 -0800

    Add volume to Kub master

    Add Cinder volume to the master and configure Docker DeviceMapper to use it

    Change-Id: I87a69175bae941d4e3bea95fce71fe3974b7cdc5
    Related-Bug: #1500062

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.