No offline capability for bootstrap.py

Bug #542747 reported by Saint Germain
18
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Buildout
Confirmed
Undecided
Unassigned

Bug Description

The current bootstrap.py is trying to connect to the internet to retrieve the distribute_setup.py or ez_setup, as well as zc.buildout.
For corporate use and reproductibility, it would be better to have an offline mode by giving the path location to retrieve necessary files.

The attached file do just that.

Maybe it's clumsy coding, so I'm open to feedbacks !

Revision history for this message
Saint Germain (saintger) wrote :
Revision history for this message
Jonathan Ballet (multani) wrote :
Download full text (3.3 KiB)

Buildout 1.5.x released a newer version of bootstrap.py, which can keep the script to go over the network:

    $ python bootstrap.py --help
    Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
    ...
    Note that by using --setup-source and --download-base to point to
    local resources, you can keep this script from going over the network.

    Options:
      ...
      --setup-source=SETUP_SOURCE
                            Specify a URL or file location for the setup file. If
                            you use Setuptools, this will default to
                            http://peak.telecommunity.com/dist/ez_setup.py; if you
                            use Distribute, this will default to http://python-
                            distribute.org/distribute_setup.py.
      --download-base=DOWNLOAD_BASE
                            Specify a URL or directory for downloading zc.buildout
                            and either Setuptools or Distribute. Defaults to PyPI.
      ...

I have a script which contains the following:

    python bootstrap.py \
        --setup-source ~/.buildout/distribute_setup.py
        --download-base ~/.buildout/downloads/dist

~/.buildout/distribute_setup.py is the official file from Distribute and:

  $ ls -1 ~/.buildout/downloads/dist/distribute*
  /home/jon/.buildout/downloads/dist/distribute-0.6.14.tar.gz
  $ ls -1 ~/.buildout/downloads/dist/zc.buildout*
  /home/jon/.buildout/downloads/dist/zc.buildout-1.5.2.tar.gz

However, running the "offline" script gives me this (while being offline):

  $ ~/Scripts/bootstrap-offline
  Downloading file:///home/jon/.buildout/downloads/dist/distribute-0.6.14.tar.gz
  Extracting in /tmp/tmpV762wE
  Now working in /tmp/tmpV762wE/distribute-0.6.14
  Building a Distribute egg in /tmp/tmpW7Ksxw
  /tmp/tmpW7Ksxw/distribute-0.6.14-py2.6.egg
  Download error for 'http://pypi.python.org/simple/zc.buildout/': [Errno -2] Name or service not known -- Some packages may not be found!
  Download error for 'http://pypi.python.org/simple/zc.buildout/': [Errno -2] Name or service not known -- Some packages may not be found!
  Download error for 'http://pypi.python.org/simple/': [Errno -2] Name or service not known -- Some packages may not be found!
  install_dir /tmp/tmpW7Ksxw

(I patched Distribute to display the URL in case of Download errors).

It works anyway despite the "Download error" message. However, if my network connection goes online, the bootstrap script spends 3 more seconds to run (it doubles the time actually, I have a 64k connection).

It seems this is due to the creation of the "index" object in the bootstrap.py script:

       index = setuptools.package_index.PackageIndex(
           search_path=[setup_requirement_path])
       if find_links:
           index.add_find_links((find_links,))
       req = pkg_resources.Requirement.parse(requirement)
>>> if index.obtain(req) is not None:

Here, "find_links" is the address I gave through the "--download-base" option, but the index is queried anyway. So, if the network is online, it queries Pypi and download something from it (I guess?), otherwise it displays the error and use the find_links address, and so, bot...

Read more...

tags: added: offline
Stuart Bishop (stub)
Changed in zc.buildout:
status: New → Confirmed
Revision history for this message
Domen Kožar (ielectric+) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.