compiling the WADL is mostly wasted CPU cycles

I agree, and Leonard and I have talked about the possibility of subdividing this information to get what you need only when you need it, but it's not a pattern that is supported by any standard AIUI, and would be us going off on our own again for both pattern and implementation.

There's already a standard for this: hyperlinks. WADL documents can link to each other. This is a standard feature of WADL that we've just never bothered to support.

Currently, the description of a 'person' is at http://api.launchpad.net/1.0/#person and the description of a 'bugtask' is at http://api.launchpad.net/1.0/#bug_task. There's no reason why we can't put the 'person' description at http://api.launchpad.net/1.0/descriptions/person and the 'bugtask' description at http://api.launchpad.net/1.0/descriptions/bug_task.

The problem is that wadllib will currently only dereference links within a single file. We'd need to enhance it to keep a collection of files and be able to cross-reference between them.

The smaller WADL files could be individually cached just like the big one is. The 'root' WADL file would be very small, containing nothing but a description and a link to the service root resource. We'd need to still serve the 'unified' WADL file to clients that were using old versions of wadllib.

(If we were serving XML representations, we could serve the WADL description of a resource *inline*, as part of the resource representation--but that would waste bandwidth, and we're not serving XML. Better to serve a link to the description IMO.)

In short: there's no reason why we can't do this, and I'm excited about it from a theoretical standpoint, but now that we serve the WADL with compression I don't think it's incredibly urgent.

splitting the definition would also further increase the penalty of
starting up a new client - rather than one [large] download, it would
be 10's or hundreds of downloads.

"splitting the definition would also further increase the penalty of starting up a new client..."

If the client needs all that stuff, then yes, a fragmented system will be much worse. But this bug was filed because a typical client only needs two or three resource definitions.

If we decide to do this, we can examine existing clients, do measurements, and see how much we improve the common case. One obvious optimization is to define "foo" and "page of foo" in the same WADL file. Another is to put the descriptions of the core resources--bugs, people, projects, maybe a couple others--into the root WADL file along with the service root, and keep everything else in individual WADL files. (Or packaged up into larger themed files--like, all the distro_* resource types could go into the same file.)

Largely for my own notes: I also mentioned subdividing WADL in passing in bug 607961, but the cause of that bug is not yet known so there's no reason to believe it would help the situation.