transliterate text/use collation before adding to xapian db and when searching
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Translations |
New
|
Undecided
|
Unassigned | ||
software-center (Ubuntu) |
Triaged
|
Medium
|
Unassigned | ||
Precise |
Won't Fix
|
Medium
|
Unassigned |
Bug Description
Binary package hint: software-center
As of now software center uses str.lower() when searching in the xapian db:
utils/query.py
22: s = search_term.lower()
33: query = xapian.
There are two problems with this:
* many languages have diacritic marks for characters but for fast typing users usually write the base character: (in Romanian: ăâșțî and ĂÂȘȚÎ are spelled AASTI by some users).
* characters in the Unicode set can appear in two forms: composed and decomposed: the character U+00C7 (LATIN CAPITAL LETTER C WITH CEDILLA) can also be expressed as the sequence U+0327 (COMBINING CEDILLA) U+0043 (LATIN CAPITAL LETTER C).
To solve both problems both the text entered in the xapian db and the user's text query must be normalized.
The search function in Chromium uses ICU rules to achieve this:
- http://
- http://
There is a python-icu library that could help achieve this. See for example http://
Or one could just remove the diacritical marks from the string altogether: http://
description: | updated |
tags: | added: db |
Changed in software-center (Ubuntu): | |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in software-center (Ubuntu Precise): | |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in software-center (Ubuntu Precise): | |
status: | Confirmed → Triaged |
This looks like a reasonable suggestion. Can you give an example of a search that would produce better results if this was implemented? That would help in prioritizing it.