Charm does not surface errors when supplied block devices cannot be used
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Swift Storage Charm |
Triaged
|
Wishlist
|
Unassigned |
Bug Description
In my case juju storage provided the charm with a disk that was already formatted. This caused mkfs.xfs to fail and the deployment to be one zone down. The only visible symptom to the end user was the "Not enough storage zones for minimum replicas" message displayed by the swift-proxy charm. This is confusing to the user as the error was actually in the storage charm. The storage charm should clearly state in juju status that it failed to initialise the required disk(s).
$ juju ssh swift-storage-z3/0 "sudo grep mkfs /var/log/
2019-09-03 10:06:14 DEBUG config-changed mkfs.xfs: /dev/vdc appears to contain an existing filesystem (ext4).
2019-09-03 10:06:14 DEBUG config-changed mkfs.xfs: Use the -f option to force overwrite.
2019-09-03 10:06:14 WARNING juju-log Format device '/dev/vdc' failed (Command '
$ juju status swift-proxy swift-storage-z1 swift-storage-z2 swift-storage-z3
Model Controller Cloud/Region Version SLA Timestamp
mojo-b9a2fccb gnuoy-serverstack serverstack/
App Version Status Scale Charm Store Rev OS Notes
swift-hacluster active 3 hacluster local 12 ubuntu
swift-proxy 2.21.0 blocked 3 swift-proxy local 147 ubuntu
swift-storage-z1 2.21.0 waiting 1 swift-storage local 90 ubuntu
swift-storage-z2 2.21.0 waiting 1 swift-storage local 91 ubuntu
swift-storage-z3 2.21.0 waiting 1 swift-storage local 92 ubuntu
Unit Workload Agent Machine Public address Ports Message
swift-proxy/0 blocked executing 52 10.5.0.72 8080/tcp (config-changed) Not enough storage zones for minimum replicas
swift-hacluster/2 active idle 10.5.0.72 Unit is ready and clustered
swift-proxy/1 blocked idle 53 10.5.0.58 8080/tcp Not enough storage zones for minimum replicas
swift-hacluster/1 active idle 10.5.0.58 Unit is ready and clustered
swift-proxy/2* blocked idle 54 10.5.0.74 8080/tcp Not enough storage zones for minimum replicas
swift-
swift-storage-z1/0* waiting idle 55 10.5.0.81 Incomplete relations: proxy
swift-storage-z2/0* waiting idle 56 10.5.0.73 Incomplete relations: proxy
swift-storage-z3/0* waiting idle 57 10.5.0.77 Incomplete relations: proxy
Machine State DNS Inst id Series AZ Message
52 started 10.5.0.72 8ff6e455-
53 started 10.5.0.58 e6fc75cc-
54 started 10.5.0.74 41e94d83-
55 started 10.5.0.81 46a13c50-
56 started 10.5.0.73 df197d5e-
Changed in charm-swift-storage: | |
status: | New → Triaged |
importance: | Undecided → Wishlist |
This issue was hit in this review as well https:/ /review. opendev. org/#/c/ 678945/ 2 , ci report here : https:/ /openstack- ci-reports. ubuntu. com/artifacts/ test_charm_ pipeline_ func_full/ openstack/ charm-swift- storage/ 678945/ 2/3795/ index.html.
The swift proxy also complains of not enough storage zones for minimum replicas.
DEBUG:runner: swift-proxy: charmers- next/swift- proxy-170 master: ""
DEBUG:runner: charm: cs:~openstack-
DEBUG:runner: series: disco
DEBUG:runner: os: ubuntu
DEBUG:runner: charm-origin: jujucharms
DEBUG:runner: charm-name: swift-proxy
DEBUG:runner: charm-rev: 170
DEBUG:runner: exposed: false
DEBUG:runner: application-status:
DEBUG:runner: current: blocked
DEBUG:runner: message: Not enough storage zones for minimum replicas
DEBUG:runner: since: 03 Sep 2019 16:45:41Z
DEBUG:runner: relations:
DEBUG:runner: cluster:
DEBUG:runner: - swift-proxy
DEBUG:runner: identity-service:
DEBUG:runner: - keystone
DEBUG:runner: object-store:
DEBUG:runner: - glance
DEBUG:runner: swift-storage:
DEBUG:runner: - swift-storage
DEBUG:runner: units:
DEBUG:runner: swift-proxy/3:
DEBUG:runner: workload-status:
DEBUG:runner: current: blocked
DEBUG:runner: message: Not enough storage zones for minimum replicas
DEBUG:runner: since: 03 Sep 2019 16:45:41Z
DEBUG:runner: juju-status:
DEBUG:runner: current: idle
DEBUG:runner: since: 03 Sep 2019 16:48:13Z
DEBUG:runner: version: 2.6.8
DEBUG:runner: leader: true
DEBUG:runner: machine: "18"
DEBUG:runner: open-ports:
DEBUG:runner: - 8080/tcp
DEBUG:runner: public-address: 172.17.110.17
DEBUG:runner: version: 2.21.0
DEBUG:runner: endpoint-bindings:
DEBUG:runner: admin: ""
DEBUG:runner: amqp: ""
DEBUG:runner: certificates: ""
DEBUG:runner: cluster: ""
DEBUG:runner: ha: ""
DEBUG:runner: identity-service: ""
DEBUG:runner: internal: ""
DEBUG:runner: nrpe-external-
DEBUG:runner: object-store: ""
DEBUG:runner: public: ""
DEBUG:runner: swift-storage: ""