Better way to include a single line secret/password from external file into yaml bundle, while stripping trailing newline

Bug #1939368 reported by Trent Lloyd
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Confirmed
Wishlist
Unassigned

Bug Description

Some deployments want to keep secret files and passwords in a separate file that is stored outside of the git repository/main bundle definitions the rest of the bundle is in. An example:

cinder-oceanstor:
  charm: cs:~fe-staging/cinder-oceanstor-2
  options:
    username: include-file://../../.secrets/oceanstor_username

The problem we had in this case, is that the single-line username 'canonical' in the file has a trailing newline which is then faithfully reproduced in the charm settings and caused an authentication failure (in part because in this case, the value ended up base64 encoded in the final config storage, and so was able to preserve the newline in ways that many other cases may not have).

This seems like a not-unlikely trap for people to hit in a recurring way. There are many solutions to this exact situation, e.g. create a file with no trailing newline (but someone creating new credentials is likely to not remember to do that, and someone editing the file to change the secret is also likely to have their editor replace the newline). Similarly expecting all charms to always strip such inputs also seems likely to just result in recurring problems.

Hence I want to consider if there is a systematically better way to handle this that is obvious and right in most cases, to use for this specific pattern.

We equally have to consider cases where the newline is wanted. Though I think more rare for the single-line case, it's not unlikely to want a trailing new-line for multi-line data. Special casing the 'include-file://' handler to only strip a newline if there is a single-line in the file seems like a bad idea as much as it might often do the right thing.

Some thoughts on better ways to do this:
 - Instead of using include-file here, just put the secrets in a bundle overlay. This can sometimes however be high-maintenance in that you have to produce a full bundle format with all the hierachies including applications/charm application name/options. And a change to the charm application name could unintentionally un-link the configuration.
 - An easy way to include all options from a YAML formatted file, e.g. an "options-include:" without having to have it fully replicate the bundle format. e.g. accept a file with the simple YAML format "username: x\npassword: y\n" to merge with the "options" section. As it's clunky to maintain many different secrets files and to then also have to correctly duplicate the charm application name (as for example, if the charm name was changed, the included file would no longer apply and that may not be obvious)

I would appreciate input on either a Canonical way to do this better with the current tools or future improvement for the same.

Revision history for this message
Nobuto Murata (nobuto) wrote :
Revision history for this message
Harry Pidcock (hpidcock) wrote :

We are currently working on a secrets spec for Juju and likely that would help improve things here. We'll have to see how they work with bundles in particular, but the effort there is promising.

Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 1939368] Re: Better way to include a single line secret/password from external file into yaml bundle, while stripping trailing newline

Note that the charm can also say "this is a username, therefore I will
always 'strip' it before passing it as the value/config to the underlying
application"

I wonder if using an overlay for the concrete details and config would be
cleaner than using an 'include-file' description.
Is there a reason it is better to pull the username from an external file?

On Tue, Oct 12, 2021 at 1:30 AM Harry Pidcock <email address hidden>
wrote:

> We are currently working on a secrets spec for Juju and likely that
> would help improve things here. We'll have to see how they work with
> bundles in particular, but the effort there is promising.
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1939368
>
> Title:
> Better way to include a single line secret/password from external file
> into yaml bundle, while stripping trailing newline
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1939368/+subscriptions
>
>

Changed in juju:
importance: Undecided → Wishlist
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for juju because there has been no activity for 60 days.]

Changed in juju:
status: Incomplete → Expired
Trent Lloyd (lathiat)
Changed in juju:
status: Expired → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.