group search broken with KeyError: 'from_user' (identi.ca protocol)

I get a 404 occasionally. Related?

Traceback (most recent call last):
  File "/home/colin/Source/gwibber/gwibber/microblog/__init__.py", line 49, in get_data
    for message in method(client):
  File "/home/colin/Source/gwibber/gwibber/microblog/identica.py", line 167, in search
    for data in self.get_search(query):
  File "/home/colin/Source/gwibber/gwibber/microblog/identica.py", line 152, in get_search
    urllib.urlencode({"q": query}))))["results"]
  File "/usr/lib/python2.5/urllib2.py", line 124, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.5/urllib2.py", line 387, in open
    response = meth(req, response)
  File "/usr/lib/python2.5/urllib2.py", line 498, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.5/urllib2.py", line 425, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.5/urllib2.py", line 506, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found

@Colin Dean
I don't think so.

BTW I still get this with version 1.20 (bzr)

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/__init__.py", line 50, in get_data
    for message in method(client):
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/identica.py", line 179, in group
    yield SearchResult(self, data, query)
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/identica.py", line 99, in __init__
    self.sender = data["from_user"]
  File "/usr/lib/pymodules/python2.6/feedparser.py", line 200, in __getitem__
    return UserDict.__getitem__(self, realkey)
KeyError: 'from_user'

using trunk: 1.2.0~bzr340-0ubuntu1~daily1
Just find any dent with a !group, click it, the new tab loads is empty, and that error is logged.

I think I've tracked down the error.

When the user clicks on a group link (gwibber:group), gwibber calls the client.group() method. identica.Client.group() calls get_group().

get_group() hits an RSS feed of group activity instead of the standard JSON interface that identica.Client.search()->get_search() does.

This RSS feed does NOT have the same key->value pairs that the search results do.

I see two options at this point:

1.) Find out if there is a JSON feed for group activity and hit it instead.

2.) Finagle the RSS results by parsing the various fields to extract the information necessary to return a valid SearchResult.

I'm going to investigate #1 before attempting #2.

It appears that laconi.ca's proposed groups API has not yet been finalized.

That gives us two options for the current:

1.) Parse the RSS and extract as much info as we can to try to get it to fit the SearchResult model

2.) Execute a search instead by calling get_search() instead of get_group().

In either case, once the API is finalized, that should obviously be used.

Craig Andrews reports that he's implemented much of the groups API and that it should be available in 0.8.1 of laconica.

http://identi.ca/notice/8583041
http://identi.ca/notice/8583474

As it stands, my attached branch executes a search instead. While not perfect, I think it's better than throwing an error and not loading anything at all.

Once I get some kind of hint from Evan Prodromou as to when 0.8.1 will be out, I'll finish work on this.

What route should we go, folks? Parse the RSS or just use search for now? Or leave it broken?

The new version is out and it includes rudimentary groups API support.

I'll work on getting gwibber to talk groups API early next week.

http://identi.ca/api/statusnet/groups/list_all.json

http://status.net/wiki/ProposedGroupsAPI

@Colin: Thanks for getting the ball rolling on this. I fixed up the stuff in your branch and got it all working and merged into trunk. Identi.ca group streams are working again.

I'd appreciate it if somebody could help test this with laconi.ca and make sure that it works there too. I don't currently have a laconi.ca instance to test against.