Small note on your "bzr co --lightweight" times (colw.lsprof). _create_files seems to be the bulk of the time. Taking that perf, and filtering out the unimportant functions:
Sorry that the malone will mess up the formatting, but this is the best I can do for now.
And focusing just on the "get_file" path, which terminates here:
62598 0 61.5976 60.5614 bzrlib.workingtree:462(get_file_byname)
+62598 0 1.0362 0.2359 +bzrlib.workingtree:393(abspath)
Note that get_file_byname is:
def get_file_byname(self, filename, filtered=True):
path = self.abspath(filename)
f = file(path, 'rb')
if filtered and self.supports_content_filtering():
...
else:
return f
The key bit to notice is that this is effectively just file() or open().
My guess is that your encrypted filesystem actually has a lot more overhead
than you realize. Probably because the filenames themselves are encrypted, it
has to do a fair amount of work to figure out the path <=> encrypted path
mapping.
It seems to cost right about 1ms for every file we want to open (for reading
*or* writing). Which probably isn't a huge overhead for many operations that
only touch a couple of files, but if you end up reading/writing/statting the
entire 64k workingtree, that 1ms adds up fast.
That would also explain 'bzr status' performance, I think. Since you have the
overhead of determining the paths over and over again.
I certainly could be wrong. But as near as I can tell, the time is spent in
'open()' (aka file()). Which is normally pretty cheap.
Small note on your "bzr co --lightweight" times (colw.lsprof). _create_files seems to be the bulk of the time. Taking that perf, and filtering out the unimportant functions:
Sorry that the malone will mess up the formatting, but this is the best I can do for now.
1 0 173.9197 3.6865 bzrlib. transform: 2370(_create_ files) transform: 1175(create_ file) workingtree: 444(get_ file) osutils: 1676(_walkdirs_ utf8) tree:98( iter_changes)
+62598 0 65.5857 1.3641 +bzrlib.
...
+62598 0 67.3141 0.3384 +bzrlib.
...
+3801 0 27.7786 0.0733 +bzrlib.
...
+1 0 2.4488 0.0000 +bzrlib.
And focusing just on the "get_file" path, which terminates here: workingtree: 462(get_ file_byname) workingtree: 393(abspath)
62598 0 61.5976 60.5614 bzrlib.
+62598 0 1.0362 0.2359 +bzrlib.
Note that get_file_byname is: byname( self, filename, filtered=True): filename) content_ filtering( ):
def get_file_
path = self.abspath(
f = file(path, 'rb')
if filtered and self.supports_
...
else:
return f
The key bit to notice is that this is effectively just file() or open().
My guess is that your encrypted filesystem actually has a lot more overhead
than you realize. Probably because the filenames themselves are encrypted, it
has to do a fair amount of work to figure out the path <=> encrypted path
mapping.
It seems to cost right about 1ms for every file we want to open (for reading writing/ statting the
*or* writing). Which probably isn't a huge overhead for many operations that
only touch a couple of files, but if you end up reading/
entire 64k workingtree, that 1ms adds up fast.
That would also explain 'bzr status' performance, I think. Since you have the
overhead of determining the paths over and over again.
I certainly could be wrong. But as near as I can tell, the time is spent in
'open()' (aka file()). Which is normally pretty cheap.