Timed out while loading subscriptions for most-common-problems

Brian, I'm subscribing this to you for the moment. Any speed improvements we can make to this query?

We can probably also bring the time down by moving to the 24h for the day period column family set up:
https://bugs.launchpad.net/errors/+bug/1046269
https://bugs.launchpad.net/errors/+bug/1008730

For reference Laney is subscribed to a total of 403 packages.

In [4]: len(laney.getBugSubscriberPackages())
Out[4]: 403

Getting the source packages that a user is subscribed to does not take too long, it is looking up all the binaries for the source package that takes the most amount of time.

We work around this for teams that are subscribed to lots of packages, like foundations-bugs and ubuntu-x-swat, by caching their subscriptions in the UserBinaryPackages column family. We could do the same thing for Laney and other users we encounter with lots of subscriptions.

Alternatively, one way to speed this up is to switch from using the getPublishedBinaries Launchpad API request for a source package and just use binaryFileUrls and then do something like this:

bf.split('/')[-1].split('_')[0] for bf in json_request(bfu_url)

while this works it seems kind of nasty.

We could also create a column family with a cache of source packages to binary packages.

Further testing seems to indicate that using binaryFileUrls isn't any faster although the Launchpad cache may be changing the test results.

I think the best thing to do here is to add laney to list of cached users.

Cool.

Maybe you could do this a bit more dynamically by noticing the failure, checking the number of subscribed packages is over some threshold and caching the user. Or just cache for all users who have access and are subscribed to >threshold packages, if that's not too expensive.

Anyway, I just tried the URL in the description and got a list of problems. Kudos.

That is interesting because the version of daisy with this change has not been rolled out yet.

On Wed, Jun 26, 2013 at 03:32:04PM -0000, Brian Murray wrote:
> That is interesting because the version of daisy with this change has
> not been rolled out yet.

That /is/ interesting - https://errors.ubuntu.com/?user=laney&period=day
shows me some crashes. Does it for you?

Although they aren't for a very diverse range of packages, so I'm not
sure it's complete.

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]