"juju config $app $item" prints multi-line strings as YAML

Bug #1780711 reported by Paul Collins
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Anastasia
2.3
Fix Released
High
Anastasia
2.4
Fix Released
High
Anastasia

Bug Description

When "juju config" prints a multi-line string, it encodes it as YAML. As well as being unexpected, this means that a simple and obvious command such as:

diff -u0 <(juju config g-s-s-amd64 mirror_list) scalingstack-bos01/glance-streams-amd64.yaml

must instead be written as the somewhat less obvious and hacky

diff -u0 <(juju config g-s-s-amd64 mirror_list | python -c 'import sys, yaml ; print(yaml.safe_load(sys.stdin.read()))') scalingstack-bos01/glance-streams-amd64.yaml

Observed with Juju 2.3.7 and Juju 2.4.0.

Tags: ui
Tim Penhey (thumper)
Changed in juju:
status: New → Triaged
importance: Undecided → High
tags: added: ui
Changed in juju:
milestone: none → 2.4.1
Changed in juju:
milestone: 2.4.1 → 2.4.2
Revision history for this message
Anastasia (anastasia-macmood) wrote :

Yes, indeed :) In fact, it only does it if you filter for a specific config value.
Responsible line https://github.com/juju/juju/blob/2.4/cmd/juju/application/config.go#L362
I'll work o a fix and will propose shortly.

Changed in juju:
status: Triaged → In Progress
assignee: nobody → Anastasia (anastasia-macmood)
milestone: 2.4.2 → 2.5-beta1
Revision history for this message
Anastasia (anastasia-macmood) wrote :

Actually, it looks like our default output format is yaml. This means that all output, filtered or not, from 'juju config' command is in yaml output.

You might find it easier to use '--format=json'?

I have proposed a PR against 2.4 to remove hard-coding of yaml for filtered output given a key though: https://github.com/juju/juju/pull/9026.

All other characters, like newline breaking of a long value or character escaping, is dictated by upsteam yaml and json formatters, external to Juju.

Changed in juju:
status: In Progress → Fix Committed
Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 1780711] Re: "juju config $app $item" prints multi-line strings as YAML

I think for single value output, we should be putting utf8 bytes into the
output, rather than yaml/json formatted.
val=(juju config foo)

should give you the right value as a utf-8 env variable.

On Tue, Aug 7, 2018 at 4:42 PM, Anastasia <email address hidden>
wrote:

> ** Changed in: juju
> Status: In Progress => Fix Committed
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1780711
>
> Title:
> "juju config $app $item" prints multi-line strings as YAML
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1780711/+subscriptions
>

Revision history for this message
Anastasia (anastasia-macmood) wrote :

@John A Meinel (jameinel),

That's an interesting suggestion. Would it work as you expect for settings that are not string though? And why utf-8? and not utf-16 or another format? Could we please file a separate a bug for it if we really think that this work should take place :)

Also would it then be worthwhile considering printing all settings values that are strings using this new format?

Revision history for this message
John A Meinel (jameinel) wrote :

If we could just take 8-byte strings and return them, I would be happy with
that. I don't think we want to force config to be 7-bit ASCII strings. And
UTF-8 is the standard mechanism beyond that in unix-land.

On Wed, Aug 8, 2018 at 4:00 AM, Anastasia <email address hidden>
wrote:

> @John A Meinel (jameinel),
>
> That's an interesting suggestion. Would it work as you expect for
> settings that are not string though? And why utf-8? and not utf-16 or
> another format? Could we please file a separate a bug for it if we
> really think that this work should take place :)
>
> Also would it then be worthwhile considering printing all settings
> values that are strings using this new format?
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1780711
>
> Title:
> "juju config $app $item" prints multi-line strings as YAML
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1780711/+subscriptions
>

Revision history for this message
John A Meinel (jameinel) wrote :

So, as I understand it, the essence of the bug is this (with juju-2.3):
$ juju config dummy-source token="a very long string that is too long for
the default content formatter, but it does need to be quite long"

$ juju config dummy-source token
a very long string that is too long for the default content formatter, but
it does
  need to be quite long

Specifically, I put in an explicit string, and the result I get back is a
reflowed version of what I put in.

That still happens with your patch in 2.4:
$ juju-2.4 config dummy-source token
a very long string that is too long for the default content formatter, but
it does
  need to be quite long

though you can at least get the JSON version:
$ juju-2.4 config dummy-source token --format=json
"a very long string that is too long for the default content formatter, but
it does need to be quite long"

Note that it is now wrapped in quotes, and still suffers if you have
characters in the content:
$ juju-2.4 config dummy-source token='abc "quoted"'
$ juju-2.4 config dummy-source token
abc "quoted"
$ juju-2.4 config dummy-source token --format=json
"abc \"quoted\""

What I think people would like to see is something where:

$ juju config application key=$VALUE
$ if [ juju config application key = "$VALUE" ] ; then echo true; else echo
false

always returns true for most definitions of $VALUE.

And I think that is more likely to be true if we used:
 fmt.Fprintf(os.Stdout, "%v\n", info["value"])

That will print out a 'string' version of any input (int, string, bool).
It doesn't handle unicode conversions, but I think it passes the "did I get
out what I put in" test.

On Sun, Aug 12, 2018 at 3:36 PM, John Meinel <email address hidden> wrote:

> If we could just take 8-byte strings and return them, I would be happy
> with that. I don't think we want to force config to be 7-bit ASCII strings.
> And UTF-8 is the standard mechanism beyond that in unix-land.
>
> On Wed, Aug 8, 2018 at 4:00 AM, Anastasia <<email address hidden>
> > wrote:
>
>> @John A Meinel (jameinel),
>>
>> That's an interesting suggestion. Would it work as you expect for
>> settings that are not string though? And why utf-8? and not utf-16 or
>> another format? Could we please file a separate a bug for it if we
>> really think that this work should take place :)
>>
>> Also would it then be worthwhile considering printing all settings
>> values that are strings using this new format?
>>
>> --
>> You received this bug notification because you are subscribed to juju.
>> Matching subscriptions: juju bugs
>> https://bugs.launchpad.net/bugs/1780711
>>
>> Title:
>> "juju config $app $item" prints multi-line strings as YAML
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/juju/+bug/1780711/+subscriptions
>>
>
>

Revision history for this message
Anastasia (anastasia-macmood) wrote :

Sounds good. I'll also add this difference in behavior as an explanation in help text of the command.

Changed in juju:
status: Fix Committed → In Progress
Revision history for this message
Anastasia (anastasia-macmood) wrote :
Revision history for this message
Anastasia (anastasia-macmood) wrote :
Revision history for this message
Anastasia (anastasia-macmood) wrote :

PR against develop (2.5): https://github.com/juju/juju/pull/9075

Changed in juju:
status: In Progress → Fix Committed
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.