Add helper function to fetch many files at once

Bug #450629 reported by Free Ekanayaka
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Landscape Client
Fix Released
Medium
Free Ekanayaka
landscape-client (Ubuntu)
Fix Released
Undecided
Unassigned
Intrepid
Fix Released
Undecided
Unassigned
Jaunty
Fix Released
Undecided
Unassigned
Karmic
Fix Released
Undecided
Unassigned
Lucid
Fix Released
Undecided
Unassigned

Bug Description

We currently have a fetch_async function for downloading a single URL. It would be nice to have one to retrieve a list of URLs, and have it callback once all of them have completed. In particular this feature will be used to fetch the files of the release-ugprade tool used for dist-upgrades.

The attached branch implements the proposed behaviour, and also add a helper function to parse the /etc/lsb-release, which is used in a couple of spots already, and will be used in the release-upgrade feature as well.

Changed in landscape-client:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Free Ekanayaka (free.ekanayaka)
milestone: none → 1.4.0
tags: added: review
Revision history for this message
Sidnei da Silva (sidnei) wrote :

[1] parse_lsb_release: Please put the fd.close() in a 'finally:' statement.

[2] fetch_to_files: If the url ends with a '/', the filename will be an empty string:

>>> url = "X/"
>>> url.split("/")[-1]
''

[3] test_fetch_many_async: s/retrives/retrieves

Looks good! +1.

Revision history for this message
Thomas Herve (therve) wrote :

[1]
+ def completed(result):
+ self.fail()
+
+ def aborted(failure):
+ self.assertEquals(fetched_urls, ["http://right/", "http://wrong/"])
+
+ d.addCallback(completed)
+ d.addErrback(aborted)
+ return d

You should use assertFailure in this case: you can remove the self.fail(), and add aborted as a callback.

[2]
+ L{fetch_to_files} fetches a list of URLs and save they're content

Typo: their content

[3] Pyflakes:
landscape/package/tests/test_taskhandler.py:7: 'CommandError' imported but unused

+1!

tags: removed: review
Revision history for this message
Free Ekanayaka (free.ekanayaka) wrote :

Thanks Thomas and Sidnei! All comments addressed.

Jamu Kakar (jkakar)
Changed in landscape-client:
milestone: 1.4.0 → 1.4.1
Changed in landscape-client:
status: In Progress → Fix Committed
tags: added: needs-testing
tags: removed: needs-testing
Jamu Kakar (jkakar)
tags: added: 1.5-upgrade-client
Changed in landscape-client (Ubuntu Lucid):
status: New → Fix Committed
Changed in landscape-client (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Accepted landscape-client into intrepid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in landscape-client (Ubuntu Intrepid):
status: New → Fix Committed
tags: added: verification-needed
Changed in landscape-client (Ubuntu Jaunty):
status: New → Fix Committed
Revision history for this message
Martin Pitt (pitti) wrote :

Accepted landscape-client into jaunty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in landscape-client (Ubuntu Karmic):
status: New → Fix Committed
Revision history for this message
Martin Pitt (pitti) wrote :

Accepted landscape-client into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.4.0-0ubuntu0.8.10.0

---------------
landscape-client (1.4.0-0ubuntu0.8.10.0) intrepid-proposed; urgency=low

  * New upstream release (LP: #497351)

  * Bug fixes:
    - Fix landscape daemons fail to start when too many groups are
      available (LP: #456124)
    - Fix landscape programs wake up far too much. (LP: #340843)
    - Fix Package manager fails with 'no such table: task' (LP #465846)
    - Fix test suite leaving temporary files around (LP #476418)
    - Fix the 1hr long wait for user data to be uploaded following a
      resynchronisation (LP #369000)

  * Add support for Ubuntu release upgrades:
    - Add helper function to fetch many files at once (LP: #450629)
    - Handle release-upgrade messages in the packagemanager
      plugin (LP: #455217)
    - Add a release-upgrader task handler (LP: #462543)
    - Support upgrade-tool environment variables (LP: #463321)

  * Add initial support for Smart package locking:
    - Detect and report changes about Smart package locks (#488108)

  * Packaging fixes:
    - Turn unnecessary Pre-Depends on python-gobject into a regular Depends
    - If it's empty, remove /etc/landscape upon purge
 -- Free Ekanayaka <email address hidden> Wed, 16 Dec 2009 10:50:05 +0100

Changed in landscape-client (Ubuntu Intrepid):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.4.0-0ubuntu0.9.04.0

---------------
landscape-client (1.4.0-0ubuntu0.9.04.0) jaunty-proposed; urgency=low

  * New upstream release (LP: #497351)

  * Bug fixes:
    - Fix landscape daemons fail to start when too many groups are
      available (LP: #456124)
    - Fix landscape programs wake up far too much. (LP: #340843)
    - Fix Package manager fails with 'no such table: task' (LP #465846)
    - Fix test suite leaving temporary files around (LP #476418)
    - Fix the 1hr long wait for user data to be uploaded following a
      resynchronisation (LP #369000)

  * Add support for Ubuntu release upgrades:
    - Add helper function to fetch many files at once (LP: #450629)
    - Handle release-upgrade messages in the packagemanager
      plugin (LP: #455217)
    - Add a release-upgrader task handler (LP: #462543)
    - Support upgrade-tool environment variables (LP: #463321)

  * Add initial support for Smart package locking:
    - Detect and report changes about Smart package locks (#488108)

  * Packaging fixes:
    - Turn unnecessary Pre-Depends on python-gobject into a regular Depends
    - If it's empty, remove /etc/landscape upon purge
 -- Free Ekanayaka <email address hidden> Wed, 16 Dec 2009 10:50:05 +0100

Changed in landscape-client (Ubuntu Jaunty):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.4.0-0ubuntu0.9.10.0

---------------
landscape-client (1.4.0-0ubuntu0.9.10.0) karmic-proposed; urgency=low

  * New upstream release (LP: #497351)

  * Bug fixes:
    - Fix landscape daemons fail to start when too many groups are
      available (LP: #456124)
    - Fix landscape programs wake up far too much. (LP: #340843)
    - Fix Package manager fails with 'no such table: task' (LP #465846)
    - Fix test suite leaving temporary files around (LP #476418)
    - Fix the 1hr long wait for user data to be uploaded following a
      resynchronisation (LP #369000)

  * Add support for Ubuntu release upgrades:
    - Add helper function to fetch many files at once (LP: #450629)
    - Handle release-upgrade messages in the packagemanager
      plugin (LP: #455217)
    - Add a release-upgrader task handler (LP: #462543)
    - Support upgrade-tool environment variables (LP: #463321)

  * Add initial support for Smart package locking:
    - Detect and report changes about Smart package locks (#488108)

  * Packaging fixes:
    - Turn unnecessary Pre-Depends on python-gobject into a regular Depends
    - If it's empty, remove /etc/landscape upon purge
 -- Free Ekanayaka <email address hidden> Wed, 16 Dec 2009 10:50:05 +0100

Changed in landscape-client (Ubuntu Karmic):
status: Fix Committed → Fix Released
Changed in landscape-client:
status: Fix Committed → Fix Released
tags: removed: verification-needed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.