Now that bzr-svn is being used for code imports we shouldn't need that much hits anymore - just one or two TCP connections per Subversion branch that we import (bzr-svn does try to use persistent connections but needs two concurrent connections sometimes).
Since there's quite a few branches in their repository and they ask that scripts only pull at most once an hour, perhaps we need to consider maintaining a local clone of their svn repository to work from, which we sync from them on an hourly basis.
As discussed with mwhudson on IRC, this would require:
* setting up the mirror and adding a cron job that keeps it up to date using svnsync
* having a way in the importer scripts to automatically rewrite http://svn.apache.org/repos/asf -> local mirror url
The initial clone of their svn repo would probably take about a week or so given the size of the repo (almost 1M revisions).
We'd need to contact them either way to get the IP block removed.
Since this would be quite a bit of work, I guess the question is how important these mirrors are?
Now that bzr-svn is being used for code imports we shouldn't need that much hits anymore - just one or two TCP connections per Subversion branch that we import (bzr-svn does try to use persistent connections but needs two concurrent connections sometimes).
Since there's quite a few branches in their repository and they ask that scripts only pull at most once an hour, perhaps we need to consider maintaining a local clone of their svn repository to work from, which we sync from them on an hourly basis.
As discussed with mwhudson on IRC, this would require: svn.apache. org/repos/ asf -> local mirror url
* setting up the mirror and adding a cron job that keeps it up to date using svnsync
* having a way in the importer scripts to automatically rewrite http://
The initial clone of their svn repo would probably take about a week or so given the size of the repo (almost 1M revisions).
We'd need to contact them either way to get the IP block removed.
Since this would be quite a bit of work, I guess the question is how important these mirrors are?