When installing multiple programs, downloading and installation should happen simultaneously

mpt if you want I could help with a patch

@shane: Help is certainly welcome, I like the idea, it should not be too hard, but requies a bit of work.

Sure ill attach a patch when its done.

Shane, I've assigned this to me only so I can add it to the specification, so feel free to implement it! I just need to know a couple of things:

1. Can the installation script for package A make package B uninstallable in a way that dpkg would normally know before downloading package B? Or is that something that only ever happens because of dependencies that can be calculated even before downloading?

2. What should happen if package B depends on A, and you request to install A then separately request to install B, and while B is downloading, the installation of A fails? Should B stop downloading immediately? Or should it complete downloading just in case you fix A later?

1. I would think the latter.

2. If package B requires package A and B fails it should stop downloading because a newer version of B that may be required might depend on a newer version of A. In that way the user doesnt waste bandwidth downloading something they my not need.

Shane, any update on this issue ?

It still doesnt do it and is still wishlist and isnt being worked on id say leave it as is.

I've now mentioned this in the specification. <https://wiki.ubuntu.com/SoftwareCenter#bug-438870>

Ideally, this shouldn't require any code changes to USC itself, just to aptdaemon and apt. USC should request every task immediately, if it doesn't do that already. apt should be able to do the concurrent downloads, then pause every task except for the one that's currently using dpkg. aptdaemon should be able to signal what's going on for every current task, and USC should just continue to reflect what aptdaemon tells it. Quite possibly, though, I don't know what I'm talking about and it's much more complicated than that...

This will need only work in aptdaemon.

yes !!! thanks guy , it is not fixed yet but thanks for the attention

On Thu, Sep 15, 2011 at 10:34 PM, Michael Vogt <email address hidden>wrote:

> This will need only work in aptdaemon.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (713137).
> https://bugs.launchpad.net/bugs/438870
>
> Title:
> Downloads should start when the previous one ends
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/aptdaemon/+bug/438870/+subscriptions
>

We could extract the to be downloaded packages during simulate and download them in the background. Since the download cache (archives) is locked during every operation we would have to add a secondary cache. Seems to get quite complicated. We would also have to deal with enabling additional repositories in the case of purchases. I am not sure if it is worth the effort.

"the download cache (archives) is locked during every operation"

Well then, what would it take to fix that? A change in apt, I guess?

You don't want a fix but a new feature. apt is completely transaction unaware - so this must be implemented in a level above apt (aptdaemon).

But is it really worth the effort? How much time would we really safe? It only makes sense to start the next download during the installation phase of the currently running transaction. Downloading packages from more than one transaction at the same time doesn't make any sense, since both would be slowed down. How long does it take to actually install a package after it has been downloaded? 5 or 10 seconds?

We don't need to care about inter-dependencies between transactions. We just try to download what we can get/guess. If we missed a new dependency since a previous transaction was cancelled, we will just download the missing packages during the normal processing of the transaction.

Mvo, purchases use chained transactions? We would skip downloads of chained transactions.

Mpt, how to handle progress information? This could result in a backwards running progress if we identify more packages to download.

You're probably right that downloading for more than one transaction at a time wouldn't be worth it. (It might save time if the downloads were from different servers, because some of the latency might be at the server's end rather than the client's. But in our case they will mostly be from the same server.) So let's concentrate on downloading for only one transaction at a time.

How much time that would save would depend mainly on four things.

First, on how what proportion of the average installation time is taken up by the download, and in particular, how close that average is to 50%. (The further away it is from 50%, in *either* direction, the more often there will be slack where only one task is in progress rather than two.)

Second, on how much variation there is in that average proportion. (Even if the average is exactly 50%, if it varies a lot, there will still be lots of slack.)

Third, on the length of the average queue: how often there's an X+1 at all, and an X+2 to take up any slack from the X+1 download finishing before X finishes, and an X+3 etc.

And fourth, how much the process of downloading X+1, and writing that downloaded file to disk, slows down the process of installing X. (Maybe this is negligible.)

As far as I know, we don't have data on any of these at the moment. I expect it would be fairly easy to write a script to measure the first two at least: choose a random package, time its download, time its installation, uninstall, repeat. And measuring those first two would be useful by itself, because it would help us make the existing progress bar smoother. <https://wiki.ubuntu.com/SoftwareCenter/SmoothProgress>

As for presenting progress of downloading packages that you thought would be downloaded already, a basic way to handle that would be: estimate/measure how often it happens, estimate/measure what proportion of the total time it takes when it does happen, multiply those two together, and allocate the resulting proportion of the total progress bar to it.

I started to work on a threading branch which would provide the base for this. Michael also sees an use case in splitting updates into smaller transactions.

I believe this works now in more recent versions?

No, it doesn't. In 5.4.1.2, a later download still waits until an earlier installation has finished.

Sorry, the title was confusing. Downloads of queued package installations could happen while already downloaded packages are installed.