Comment 15 for bug 593560

Revision history for this message
John A Meinel (jameinel) wrote :

Small note on your "bzr co --lightweight" times (colw.lsprof). _create_files seems to be the bulk of the time. Taking that perf, and filtering out the unimportant functions:

Sorry that the malone will mess up the formatting, but this is the best I can do for now.

           1 0 173.9197 3.6865 bzrlib.transform:2370(_create_files)
      +62598 0 65.5857 1.3641 +bzrlib.transform:1175(create_file)
...
      +62598 0 67.3141 0.3384 +bzrlib.workingtree:444(get_file)
...
       +3801 0 27.7786 0.0733 +bzrlib.osutils:1676(_walkdirs_utf8)
...
          +1 0 2.4488 0.0000 +bzrlib.tree:98(iter_changes)

And focusing just on the "get_file" path, which terminates here:
       62598 0 61.5976 60.5614 bzrlib.workingtree:462(get_file_byname)
      +62598 0 1.0362 0.2359 +bzrlib.workingtree:393(abspath)

Note that get_file_byname is:
    def get_file_byname(self, filename, filtered=True):
        path = self.abspath(filename)
        f = file(path, 'rb')
        if filtered and self.supports_content_filtering():
            ...
        else:
            return f

The key bit to notice is that this is effectively just file() or open().

My guess is that your encrypted filesystem actually has a lot more overhead
than you realize. Probably because the filenames themselves are encrypted, it
has to do a fair amount of work to figure out the path <=> encrypted path
mapping.

It seems to cost right about 1ms for every file we want to open (for reading
*or* writing). Which probably isn't a huge overhead for many operations that
only touch a couple of files, but if you end up reading/writing/statting the
entire 64k workingtree, that 1ms adds up fast.

That would also explain 'bzr status' performance, I think. Since you have the
overhead of determining the paths over and over again.

I certainly could be wrong. But as near as I can tell, the time is spent in
'open()' (aka file()). Which is normally pretty cheap.