zimdicator.py : ValueError: Expecting , delimiter: line 1 column 25299 (char 25299)

Bug #583069 reported by Martin Wimpress 
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Zimdicator
Fix Released
Critical
Mitja Pagon

Bug Description

Hi,

I'm running zimdicator from bzr trunk with my desktop patch applied. Thanks for your help with that BTW :-) However, running zimdicator.py now causes a ValueError. See the traceback below.

Traceback (most recent call last):
  File "./zimdicator.py", line 173, in <module>
    zim = Zimdicator()
  File "./zimdicator.py", line 90, in __init__
    self.check_mail()
  File "./zimdicator.py", line 127, in check_mail
    json_data = json.loads(json_str)
  File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.6/json/decoder.py", line 336, in raw_decode
    obj, end = self._scanner.iterscan(s, **kw).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 183, in JSONObject
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 217, in JSONArray
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 193, in JSONObject
    raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
ValueError: Expecting , delimiter: line 1 column 25299 (char 25299)

I'm sorry I keep breaking your stuff but I will help test in any way I can :-)

Regards, Martin.

Revision history for this message
Mitja Pagon (sect2k) wrote :

Looks like more JSON decoding issues, the thing is, I can't reproduce this one. Is there anyway you could find out what part of JSON returned by zimbra causes the error (located at chat 25299 in your case).

To do that, try this. Login into zimbra web client, open a new tab in browser and type this url

https://<email address hidden>/?fmt=json&query=is:unread

copy the content of that page into text editor and look at what is at char 25299.

That would probably help a lot. You breaking my stuff, helps me improve it so keep on breaking ;-)

Cheers,
Mitja

Mitja Pagon (sect2k)
Changed in zimdicator:
importance: Undecided → Critical
status: New → Triaged
Revision history for this message
Martin Wimpress  (flexiondotorg) wrote :

Hi,

OK, since last night the number of unread mails has increased. Here is the revised traceback.

Traceback (most recent call last):
  File "./zimdicator.py", line 173, in <module>
    zim = Zimdicator()
  File "./zimdicator.py", line 90, in __init__
    self.check_mail()
  File "./zimdicator.py", line 127, in check_mail
    json_data = json.loads(json_str)
  File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.6/json/decoder.py", line 336, in raw_decode
    obj, end = self._scanner.iterscan(s, **kw).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 183, in JSONObject
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 217, in JSONArray
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 193, in JSONObject
    raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
ValueError: Expecting , delimiter: line 1 column 39725 (char 39725)

I added a line to print the contents of json_str and redirected the output of zimdicator .py to a file. The then out a few chars either side of the offending 39725 with the following.

cat json.txt | cut -c 39710-39740

Here is string that fragment.

e has vanished: \\"/var/lib/pos

Hope that helps.

Regards, Martin.

Revision history for this message
Mitja Pagon (sect2k) wrote :

Hi, could you expand the fragment a bit more? It' would help to maybe see the whole "string", that is causing this issue.

Thanks,
M

Revision history for this message
Martin Wimpress  (flexiondotorg) wrote :

Hi,

zimdicator is encountering un-escaped double back slashes (\\) in the string, despite your replacement for '\\' to '\\\\'?! Had me stumped, seemed to me your replacement should work. But evertime I debug it, it is choking on a double slash. So I did some reading, the following are informative but didn't help get me to a solution.

http://stackoverflow.com/questions/1505454/python-json-loads-chokes-on-escapes
http://stackoverflow.com/questions/983855/python-json-encoding

Mitja Pagon (sect2k)
Changed in zimdicator:
assignee: nobody → Mitja Pagon (sect2k)
Revision history for this message
Mitja Pagon (sect2k) wrote :

I've switched to XML as data transport format as it seems that Zimbra doesn't follow escaping conventions in JSON. Hope this resolves all this escaping issues and allows us to move forward.

Let me know if it works. Also there is initial notify-osd support in trunk, testing is needed. If it all works OK, i'll make a new release as it seems 0.1 is mostly unusable.

Cheers,
M

Changed in zimdicator:
status: Triaged → Fix Committed
Mitja Pagon (sect2k)
Changed in zimdicator:
milestone: none → 0.2
Mitja Pagon (sect2k)
Changed in zimdicator:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.