EOFError in tritcask _deserialize - .hint and .data disagree on key availability

Bug #1064486 reported by Roman Yepishev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu One Client
Confirmed
High
Ubuntu One Client Engineering team

Bug Description

There are several reports of silent Ubuntu One startup that manifests as follows in syncdaemon.log:

2012-08-22 11:08:48,862 - ubuntuone.SyncDaemon.Main - INFO - Starting Ubuntu One client version 3.0.2b
2012-08-22 11:08:48,862 - ubuntuone.SyncDaemon.Main - INFO - Using 'C:\\Users\\username\\Ubuntu One' as root dir
2012-08-22 11:08:48,862 - ubuntuone.SyncDaemon.Main - INFO - Using 'C:\\Users\\username\\AppData\\Local\\xdg\\ubuntuone\\syncdaemon' as data dir
2012-08-22 11:08:48,862 - ubuntuone.SyncDaemon.Main - INFO - Using 'C:\\Users\\username\\AppData\\Local\\xdg\\ubuntuone\\shares' as shares root dir
2012-08-22 11:08:48,862 - ubuntuone.SyncDaemon.tritcask - INFO - Initializing Tritcask on: C:\Users\username\AppData\Local\xdg\ubuntuone\syncdaemon\tritcask
2012-08-22 11:08:48,862 - ubuntuone.SyncDaemon.tritcask - DEBUG - lookingup data files
2012-08-22 11:08:49,081 - ubuntuone.SyncDaemon.tritcask - INFO - found 5 data files, 0 dead and 0 broken files
2012-08-22 11:08:49,081 - ubuntuone.SyncDaemon.tritcask - DEBUG - building the keydir, using: [u'1344062980413809', u'1343543408748278', u'1343543406439616', u'1343631846410462', u'1344064244133051']
2012-08-22 11:08:49,081 - ubuntuone.SyncDaemon.tritcask - DEBUG - loading entries from hint of: C:\Users\username\AppData\Local\xdg\ubuntuone\syncdaemon\tritcask\1343543406439616.inactive.tritcask-v1.data
2012-08-22 11:08:49,158 - ubuntuone.SyncDaemon.tritcask - DEBUG - loading entries from hint of: C:\Users\username\AppData\Local\xdg\ubuntuone\syncdaemon\tritcask\1343543408748278.inactive.tritcask-v1.data
2012-08-22 11:08:49,158 - ubuntuone.SyncDaemon.tritcask - DEBUG - loading entries from hint of: C:\Users\username\AppData\Local\xdg\ubuntuone\syncdaemon\tritcask\1343631846410462.inactive.tritcask-v1.data
2012-08-22 11:08:49,158 - ubuntuone.SyncDaemon.tritcask - DEBUG - loading entries from hint of: C:\Users\username\AppData\Local\xdg\ubuntuone\syncdaemon\tritcask\1344062980413809.inactive.tritcask-v1.data
2012-08-22 11:08:49,158 - ubuntuone.SyncDaemon.tritcask - DEBUG - Ignoring empty live file.
2012-08-22 11:08:49,158 - ubuntuone.SyncDaemon.tritcask - INFO - keydir ready! (keys: 3946)
2012-08-22 11:08:49,158 - ubuntuone.SyncDaemon.VM.MD - DEBUG - metadata version: 7
2012-08-22 11:08:49,206 - ubuntuone.SyncDaemon.fsm - INFO - loading updated metadata

No more log entries are added until restart, which adds
2012-08-22 11:12:57,417 - ubuntuone.SyncDaemon - INFO - not starting twisted.manhole

The original issue for silent failure was fixed in LP: 1041163 but the underlying EOFError remains:

Here's the output of dump_metadata.py for such kind of broken tritcask data file:

Showing Root: 'C:\\Users\\username\\Ubuntu One' (id='')

Showing UDF: 'C:\\Users\\username\\Pictures - HTC Desire HD
A9191' (id='6c95c8cf-4c9d-440a-84f6-31b4e02e8209')

Showing UDF: 'C:\\Users\\username\\Pictures - GT-P7510'
(id='6fde87dd-1d53-47b8-8d57-8b3d882749a8')

Showing trash:
  (empty)

Showing move limbo:
Traceback (most recent call last):
  File "contrib/dump_metadata.py", line 166, in <module>
    main(basedir)
  File "contrib/dump_metadata.py", line 141, in main
    for key, value in fsm.move_limbo.iteritems():
  File "/usr/lib/python2.7/UserDict.py", line 111, in iteritems
    yield (k, self[k])
  File
"/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/filesystem_manager.py",
line 281, in __getitem__
    return super(TrashTritcaskShelf, self).__getitem__(raw_key)
  File
"/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py",
line 953, in __getitem__
    return self._deserialize(self._db.get(self.row_type, key))
  File
"/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py",
line 978, in _deserialize
    return cPickle.loads(raw_value)
EOFError

It looks like the .data file does not have the entry which gets pointed to by .hint file.

Roman Yepishev (rye)
tags: added: support
Changed in ubuntuone-client:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Ubuntu One Desktop+ team (ubuntuone-desktop+)
Leo Arias (elopio)
tags: added: desktop+
tags: added: u1-support
removed: support
tags: added: u1-support-escalated
removed: u1-support
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.