We started with a local rescan that took: 121249081 function calls (121023286 primitive calls) in 665.469 CPU seconds Latest profile info for Local Rescan shows: 17240440 function calls (17018317 primitive calls) in 506.979 CPU seconds And the next bottleneck to tackle is cPickle.load, as per: ncalls tottime percall cumtime percall filename:lineno(function) 354949 180.393 0.001 183.109 0.001 {cPickle.load} 2581 83.872 0.032 319.376 0.124 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:507(get_mdobjs_in_dir) 4539 42.603 0.009 42.603 0.009 {posix.listdir} 58545 36.384 0.001 36.384 0.001 {posix.remove} 58476 26.243 0.000 26.243 0.000 {method 'remove' of 'collections.deque' objects} 354949 16.301 0.000 16.301 0.000 {open} 37542 12.212 0.000 12.212 0.000 {posix.lstat} 459845 12.203 0.000 271.947 0.001 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:212(__getitem__) 886460 10.724 0.000 17.639 0.000 /usr/lib/python2.6/posixpath.py:59(join) 354949 9.861 0.000 211.277 0.001 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:118(_load_pickle) 459845 8.220 0.000 11.197 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:292(purge) 393979 6.764 0.000 23.727 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:67(key_file) 459845 4.862 0.000 7.537 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:256(__getitem__) 2375292 3.857 0.000 3.857 0.000 {len} 1732359 3.702 0.000 3.702 0.000 {method 'startswith' of 'str' objects} 459845 3.496 0.000 5.244 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:287(update) 218207 3.465 0.000 46.556 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:950(changed) 444246/222123 3.455 0.000 4.842 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:141(__init__) 1674418 3.345 0.000 3.345 0.000 {method 'endswith' of 'str' objects} 354949 3.238 0.000 235.622 0.001 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:142(__getitem__) 354949 2.819 0.000 16.586 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:270(__setitem__) 2581 2.778 0.001 369.058 0.143 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/local_rescan.py:182(_get_share_info) 354813 2.715 0.000 2.715 0.000 /usr/lib/python2.6/os.py:721(_make_stat_result) 218207 1.992 0.000 3.211 0.000 /usr/lib/python2.6/posixpath.py:79(split) 900176 1.743 0.000 1.743 0.000 {method 'get' of 'dict' objects} 354949 1.474 0.000 184.583 0.001 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:111(_unpickle) 2581 1.443 0.001 77.223 0.030 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/local_rescan.py:494(_paths_filter) 444246 1.387 0.000 1.387 0.000 {method 'update' of 'dict' objects} 715986 1.305 0.000 1.305 0.000 {method 'append' of 'list' objects} 218207 1.268 0.000 1.943 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:922(_get_mdid_from_args) 2581 1.160 0.000 506.973 0.196 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/local_rescan.py:545(inner_scan) 81745 1.114 0.000 1.114 0.000 /usr/lib/python2.6/logging/__init__.py:1222(getEffectiveLevel) 19515 1.036 0.000 1.872 0.000 /usr/lib/python2.6/posixpath.py:308(normpath) 19515 0.856 0.000 39.860 0.002 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:163(__delitem__) 474845 0.817 0.000 0.817 0.000 {method 'append' of 'collections.deque' objects} 2581 0.816 0.000 450.035 0.174 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/local_rescan.py:231(_compare) 458022 0.763 0.000 0.763 0.000 {method 'popleft' of 'collections.deque' objects} 19515 0.700 0.000 71.420 0.004 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:678(delete_metadata) 81745 0.633 0.000 2.214 0.000 /usr/lib/python2.6/logging/__init__.py:1096(log) 19515 0.618 0.000 67.340 0.003 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:228(__delitem__) 354949 0.532 0.000 0.532 0.000 {method '__enter__' of 'file' objects} 19515 0.519 0.000 26.808 0.001 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:276(__delitem__) 222126 0.508 0.000 0.508 0.000 {method 'rfind' of 'str' objects} 81745 0.467 0.000 1.581 0.000 /usr/lib/python2.6/logging/__init__.py:1236(isEnabledFor) 222003 0.418 0.000 0.418 0.000 {method 'rstrip' of 'str' objects} 37530 0.403 0.000 0.403 0.000 {posix.access} 110027 0.177 0.000 0.177 0.000 /usr/lib/python2.6/stat.py:24(S_IFMT) 23431 0.167 0.000 0.167 0.000 {method 'split' of 'str' objects} 37542 0.147 0.000 12.359 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:1155(get_stat) 37542 0.137 0.000 0.200 0.000 /usr/lib/python2.6/stat.py:55(S_ISLNK) 37530 0.130 0.000 0.189 0.000 /usr/lib/python2.6/stat.py:40(S_ISDIR) 34955 0.119 0.000 0.174 0.000 /usr/lib/python2.6/stat.py:49(S_ISREG) 19515 0.101 0.000 0.101 0.000 {method 'join' of 'str' objects} 19516 0.096 0.000 0.096 0.000 {isinstance} 3916 0.088 0.000 0.381 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:543(get_by_mdid) 37510 0.075 0.000 0.075 0.000 {method 'pop' of 'dict' objects} 3659 0.058 0.000 0.058 0.000 {posix.stat} 19515 0.055 0.000 0.055 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/file_shelf.py:283(__contains__) 3916 0.048 0.000 0.075 0.000 /usr/lib/python2.6/posixpath.py:117(dirname) 3916 0.028 0.000 0.057 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:1048(get_abspath) 3659 0.025 0.000 0.083 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/local_rescan.py:265(check_stat) 3916 0.021 0.000 0.029 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/filesystem_manager.py:1039(_get_share) 5162 0.012 0.000 0.012 0.000 {method 'iteritems' of 'dict' objects} 2581 0.006 0.000 0.006 0.000 {method 'extend' of 'list' objects} 2581 0.005 0.000 0.005 0.000 {method 'disable' of '_lsprof.Profiler' objects} 10 0.001 0.000 0.001 0.000 {sum} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:226(__init__) 1 0.000 0.000 0.000 0.000 {method 'flush' of 'file' objects} 2 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:382(formatTime) 1 0.000 0.000 0.000 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/logger.py:86(shouldRollover) 2 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:423(format) 10 0.000 0.000 0.000 0.000 {method 'itervalues' of 'dict' objects} 2 0.000 0.000 0.000 0.000 {time.localtime} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:756(emit) 2 0.000 0.000 0.000 0.000 {time.strftime} 2 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:637(format) 2 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:289(getMessage) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:1113(findCaller) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:1196(callHandlers) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:660(handle) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/threading.py:116(acquire) 1 0.000 0.000 0.001 0.001 /usr/lib/python2.6/logging/__init__.py:1147(_log) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/handlers.py:63(emit) 3 0.000 0.000 0.000 0.000 /usr/lib/python2.6/string.py:351(find) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/threading.py:136(release) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:148(getLevelName) 6 0.000 0.000 0.000 0.000 {hasattr} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/genericpath.py:85(_splitext) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:1167(handle) 2 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:560(filter) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:520(filter) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:1134(makeRecord) 1 0.000 0.000 0.000 0.000 {method 'seek' of 'file' objects} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:76() 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:749(flush) 2 0.000 0.000 0.000 0.000 {time.time} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:843(emit) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/handlers.py:269(shouldRollover) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/posixpath.py:109(basename) 3 0.000 0.000 0.000 0.000 {method 'find' of 'str' objects} 1 0.000 0.000 0.000 0.000 /home/rtg/Projects/ubuntuone/speed-up-that-local-rescan/ubuntuone/syncdaemon/logger.py:124(filter) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:617(acquire) 4 0.000 0.000 0.000 0.000 {thread.get_ident} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/threading.py:808(currentThread) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/logging/__init__.py:624(release) 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/posixpath.py:94(splitext) 1 0.000 0.000 0.000 0.000 {method 'write' of 'file' objects} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/threading.py:662(name) 2 0.000 0.000 0.000 0.000 /usr/lib/python2.6/threading.py:64(_note) 1 0.000 0.000 0.000 0.000 {method 'tell' of 'file' objects} 1 0.000 0.000 0.000 0.000 {built-in method acquire} 1 0.000 0.000 0.000 0.000 {sys._getframe} 1 0.000 0.000 0.000 0.000 {built-in method release} 1 0.000 0.000 0.000 0.000 {posix.getpid} 1 0.000 0.000 0.000 0.000 /usr/lib/python2.6/posixpath.py:42(normcase)