Analyze differences in relevance ranking, zope.textindex vs. pgtextindex

Bug #604642 reported by Paul Everitt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL3
Fix Released
Medium
Chris Rossi

Bug Description

Estimated Effort: 2d

Goals
========

- Figure out why search results are different
- See in which cases one is better than the other
- Compile a list of suggested improvements
- Later tasks will analyze performance/scalability impacts

Details
=========

- Get both running side-by-side using OSI data
- Find the N most recent text searches for OSI
- Compare results, trying to gauge which would have been "better"
- Do analysis to see why each are yielding different results
- See how the text extraction, synonyms, stopwords, stemming, weighting, and
  other parts of the equation compare
- Propose changes in the index plugin protocol to improve relevance
- Write down any places (e.g. "Don't index the profile photo") that might yield
  better quality search results
- Include in the list any oddball ideas for boosting relevance (dynamic weighting
  based on count of tags, whatever)
- See if results are different/better when faced with foreign languages/characters

Question
==========

1) Are we doing an implicit prefix search on the advanced search page? I think
there was once a theory that it should return the same results yielded by the
LiveSearch. Which is, in hindsight, a dumb idea.

2) Does the pgtextindex URL use system ispell dictionaries?

Changed in karl3:
assignee: nobody → Chris Rossi (chris-archimedeanco)
importance: Undecided → Medium
milestone: none → m44
status: New → Confirmed
Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :
Changed in karl3:
status: Confirmed → Fix Committed
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

I'm going to re-open this one, as we might want to add to the report: advanced search and prefix search performance, memory usage.

Changed in karl3:
milestone: m44 → m45
status: Fix Committed → In Progress
Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

Prefix search and memory usage are already in the report and there is another ticket now for advanced search performance, so I'm marking this as committed.

Changed in karl3:
status: In Progress → Fix Committed
Revision history for this message
JimPGlenn (jpglenn09) wrote :

fixed

Changed in karl3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.