certificates-relation-departed breaks on error loop after removing barbican

Bug #1866553 reported by Pedro Guimarães
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
vault-charm
Incomplete
High
Unassigned

Bug Description

Using Bionic/stein (stable) charms on Juju 2.6.10.

To redeploy some applications, I ran the following:
juju remove-application --force octavia barbican

After that Vault moved to error state with following logs:
subprocess.CalledProcessError: Command '['relation-get', '--format=json', '-r', 'certificates:323', '-', 'barbican/0']' returned non-zero exit status 1.

More complete logs: https://pastebin.canonical.com/p/4b4rdNmZYR/

I've seen that happening before and I don't recall using --force all the time.
I am not entirely sure if that flag influenced or not this issue.

Tags: backport
Changed in vault-charm:
importance: Undecided → High
status: New → Triaged
milestone: none → 20.05
tags: added: backport
David Ames (thedac)
Changed in vault-charm:
milestone: 20.05 → 20.08
James Page (james-page)
Changed in vault-charm:
milestone: 20.08 → none
Changed in vault-charm:
status: Triaged → In Progress
assignee: nobody → Aurelien Lourot (aurelien-lourot)
Revision history for this message
Aurelien Lourot (aurelien-lourot) wrote :

Thanks for reporting! I can't find a specific charms.reactive or charm-helpers commit that would be the fix for this, yet it seems to have been fixed meanwhile. We see in the original logs [0] that in order to reproduce this we need a bundle with barbican connected to vault over the `certificates` relation. Thus I created a bionic-stein.yaml bundle [1] and did this a few times:

juju deploy ./bionic-stein.yaml
juju remove-application --force barbican

Each time in my case the vault unit managed to run the certificates-relation-departed and certificates-relation-joined with no issues. Also the `--force` flag shouldn't have any impact on the errors of applications not being removed. For the record I'm using Juju 2.8.7.

Closing this bug for now. Don't hesitate to re-open if you still encounter this and please provide more information (e.g. charm versions and bundle). Thanks!

[0] https://pastebin.canonical.com/p/4b4rdNmZYR/
[1] https://paste.ubuntu.com/p/nrsrSnQrMw/

Changed in vault-charm:
status: In Progress → Incomplete
assignee: Aurelien Lourot (aurelien-lourot) → nobody
Revision history for this message
Reginald Carey (reggiecarey) wrote :

This has occurred with me as well.

Attempting to force remove barbican to hopefully induce vault to exit hook failed:"certificates-relation-departed" state.

What is the resolution to the

hook failed:"certificates-relation-departed"

message that vault finds itself stuck in?

Revision history for this message
Billy Olsen (billy-olsen) wrote :

You should be able to get past the hook failure with:

juju resolved --no-retry vault/<errored_unit>

This will cause juju to mark the error as resolved and *not* retry the hook.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.