Mojo collect fails on charmstore URLs when 'charm' is installed from a snap

Bug #1717089 reported by Casey Marshall
30
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Mojo: Continuous Delivery for Juju
Confirmed
Medium
Unassigned

Bug Description

I'm not sure if this is a bug in mojo, snappy, or the charm tool tbh, but in my experience, it uniquely affects mojo, so here goes...

If I 'apt install charm' from the Juju PPA, collect works fine.

If I collect from the charmstore, and I've gotten the charm tool from 'snap install charm', I get an error during the collect phase (I modified codetree out of /usr/lib/python2.7 to print the exception to check_output).

c@zeugmatic:~/mojo/prod-omnibus-trunk$ mojo --log-level DEBUG run
2017-09-13 17:55:09 [INFO] Using pre-existing container class lxc
2017-09-13 17:55:09 [WARNING] Project doesn't seem properly set up, please fix as project will break in future releases. /var/lib/lxc/mojo-prod-omnibus.xenial/config.nonet
2017-09-13 17:55:18 [INFO] Pulling /srv/mojo/mojo-prod-omnibus/xenial/mojo-prod-omnibus/spec from parent (bzr+ssh://bazaar.launchpad.net/~omnibus-dev/canonical-mojo-specs/trunk)
2017-09-13 17:55:26 [INFO] /srv/mojo/mojo-prod-omnibus/xenial/mojo-prod-omnibus/spec is at revno 1578
2017-09-13 17:55:26 [INFO] Retrieve the spec's manifest
2017-09-13 17:55:26 [INFO] Manifest comment:

#############################################################################
Deploy omnibus environment
#############################################################################

2017-09-13 17:55:26 [INFO] Manifest comment:

#############################################################################
manifest-deploy
Deploying new omnibus environment
#############################################################################

2017-09-13 17:55:26 [INFO] Manifest comment:

#############################################################################
Predeploy checks
#############################################################################

2017-09-13 17:55:26 [INFO] Checking Juju status
2017-09-13 17:55:28 [INFO] Environment has reached steady state
2017-09-13 17:55:28 [DEBUG] ### Running phase collect with options {'config': 'collect'} ###
2017-09-13 17:55:28 [INFO] Building resource tree
2017-09-13 17:55:30 [INFO] Downloading cs:nrpe from charm store to nrpe
2017-09-13 17:55:30 [INFO] Downloading cs:~prometheus-charmers/prometheus from charm store to prometheus
2017-09-13 17:55:30 [INFO] Downloading cs:haproxy-41 from charm store to haproxy
2017-09-13 17:55:30 [INFO] Downloading cs:~cmars/kpidev from charm store to kpidev
2017-09-13 17:55:30 [INFO] Downloading cs:postgresql from charm store to postgresql
2017-09-13 17:55:30 [INFO] Downloading cs:telegraf from charm store to telegraf
ERROR cannot expand cs:nrpe archive: cannot extract "revision": mkdir /var/lib/snapd/void/nrpe: permission denied

ERROR cannot expand cs:haproxy-41 archive: cannot extract "revision": mkdir /var/lib/snapd/void/haproxy: permission denied

ERROR cannot expand cs:~cmars/kpidev archive: cannot extract "revision": mkdir /var/lib/snapd/void/kpidev: permission denied

ERROR cannot expand cs:postgresql archive: cannot extract "revision": mkdir /var/lib/snapd/void/postgresql: permission denied

ERROR cannot expand cs:telegraf archive: cannot extract "revision": mkdir /var/lib/snapd/void/telegraf: permission denied

ERROR cannot expand cs:~prometheus-charmers/prometheus archive: cannot extract "revision": mkdir /var/lib/snapd/void/prometheus: permission denied

2017-09-13 17:58:45 [ERROR] Error calling external process
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mojo/cli.py", line 622, in run_with_args
    args.func(args)
  File "/usr/lib/python2.7/dist-packages/mojo/utils.py", line 305, in wrapped
    return method(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/mojo/cli.py", line 346, in run_from_manifest
    manifest.run(project, workspace, args.stage)
  File "/usr/lib/python2.7/dist-packages/mojo/manifest.py", line 99, in run
    auto_repo=auto_repo)
  File "/usr/lib/python2.7/dist-packages/mojo/phase.py", line 233, in run
    collect_success = collect_config.build()
  File "/usr/lib/python2.7/dist-packages/codetree/config.py", line 226, in build
    results.append(f.result())
  File "/usr/lib/python2.7/dist-packages/concurrent/futures/_base.py", line 405, in result
    return self.__get_result()
  File "/usr/lib/python2.7/dist-packages/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/codetree/config.py", line 70, in run_with_dependency
    return self.run()
  File "/usr/lib/python2.7/dist-packages/codetree/config.py", line 44, in run
    return self.source.get(self.location, self.source_options)
  File "/usr/lib/python2.7/dist-packages/codetree/handlers/charm.py", line 60, in get
    self.charmstore_url = downloader.resolve()
  File "/usr/lib/python2.7/dist-packages/codetree/handlers/charm.py", line 180, in resolve
    out = subprocess.check_output(cmd, stderr=subprocess.STDOUT, universal_newlines=True)
  File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '('charm', 'show', '--format=json', '--channel', 'beta', 'cs:~yellow/xenial/omnibus-directory-4', 'id')' returned non-zero exit status 1

Revision history for this message
Haw Loeung (hloeung) wrote :

This is related to the latest charm tools snap. I'm also seeing the same when trying to build charms:

[hloeung@dharkan nrpe-charm]$ charm build
build: Composing into /var/lib/snapd/void
build: Destination charm directory: /var/lib/snapd/void/trusty/void
build: Please add a `repo` key to your layer.yaml, with a url from which your layer can be cloned.
Traceback (most recent call last):
  File "/snap/charm/17/bin/charm-build", line 11, in <module>
    load_entry_point('charm-tools==2.1.2', 'console_scripts', 'charm-build')()
  File "/snap/charm/17/lib/python2.7/site-packages/charmtools/build/builder.py", line 679, in main
    build()
  File "/snap/charm/17/lib/python2.7/site-packages/charmtools/build/builder.py", line 522, in __call__
    self.generate()
  File "/snap/charm/17/lib/python2.7/site-packages/charmtools/build/builder.py", line 473, in generate
    layers = self.fetch()
  File "/snap/charm/17/lib/python2.7/site-packages/charmtools/build/builder.py", line 246, in fetch
    self.target_dir.makedirs_p()
  File "/snap/charm/17/lib/python2.7/site-packages/path.py", line 1222, in makedirs_p
    self.makedirs(mode)
  File "/snap/charm/17/lib/python2.7/site-packages/path.py", line 1215, in makedirs
    os.makedirs(self, mode)
  File "/snap/charm/17/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/snap/charm/17/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/lib/snapd/void/trusty'

Revision history for this message
Richard Harding (rharding) wrote :

I've hit this and I'm wondering if there's a workaround for this? The PPA of charm is deprecated and it's snap only from this point and I'm looking for some path forward.

Revision history for this message
Laurent Sesquès (sajoupa) wrote :

I could reproduce a similar issue (though not exactly the same error message):
ubuntu@epic-bear:~/spec$ mojo run
2018-03-26 06:29:57 [INFO] Using pre-existing container class lxd
/usr/lib/python2.7/dist-packages/pylxd/model.py:116: UserWarning: Attempted to set unknown attribute "description" on instance of "Container"
  key, self.__class__.__name__
2018-03-26 06:29:58 [INFO] Rsyncing /home/ubuntu/spec to /home/ubuntu/.local/share/mojo/lp1717089/xenial/lp1717089/spec
2018-03-26 06:29:58 [INFO] Retrieve the spec's manifest
2018-03-26 06:29:58 [INFO] Building resource tree
2018-03-26 06:29:59 [INFO] Downloading cs:ubuntu-12 from charm store to ubuntu
2018-03-26 06:29:59 [INFO] Downloading cs:nrpe-49 from charm store to nrpe
2018-03-26 06:29:59 [ERROR] Error calling external process
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mojo/cli.py", line 628, in run_with_args
    args.func(args)
  File "/usr/lib/python2.7/dist-packages/mojo/utils.py", line 305, in wrapped
    return method(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/mojo/cli.py", line 346, in run_from_manifest
    manifest.run(project, workspace, args.stage)
  File "/usr/lib/python2.7/dist-packages/mojo/manifest.py", line 100, in run
    auto_repo=auto_repo)
  File "/usr/lib/python2.7/dist-packages/mojo/phase.py", line 255, in run
    collect_success = collect_config.build()
  File "/usr/lib/python2.7/dist-packages/codetree/config.py", line 271, in build
    results.append(f.result())
  File "/usr/lib/python2.7/dist-packages/concurrent/futures/_base.py", line 405, in result
    return self.__get_result()
  File "/usr/lib/python2.7/dist-packages/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/codetree/config.py", line 115, in run_with_dependency
    return self.run()
  File "/usr/lib/python2.7/dist-packages/codetree/config.py", line 83, in run
    result = self.source.get(self.location, self.source_options)
  File "/usr/lib/python2.7/dist-packages/codetree/handlers/charm.py", line 85, in get
    if downloader.get(dest):
  File "/usr/lib/python2.7/dist-packages/codetree/handlers/charm.py", line 193, in get
    subprocess.check_output(cmd, stderr=subprocess.STDOUT, universal_newlines=True)
  File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '('charm', 'pull', '--channel', 'stable', u'cs:nrpe-49', 'nrpe')' returned non-zero exit status 1
ubuntu@epic-bear:~/spec$

Changed in mojo:
status: New → Confirmed
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.