I have not had the time to narrow down where exactly everything dies, but there needs to be some kind of wrapper system to ensure that everything that is written to sqlite and later retrieved is UTF-8. Currently, it handles some unicode but cannot convert Latin1 to UTF-8.
This occurs while scanning directories for files, while storing tag data, and when retrieving songs from the DB. It seems to be a real showstopper, because sqlite supports only UTF-8.
I'm afraid I won't have more time to play with this for now, so I'm posting a ticket in hopes that someone else will get around to it. I suggest either refactoring the database inserts a little, so you can add some latin1->utf8 decode/encode logic before inserts or maybe play with something similar to:
{{{
db.text_factory = lambda x: unicode(x, "utf-8", "replace")
}}}
Same goes for file scanning:
{{{
xl/tracks.py
for file in to_scan:
try:
+ file = unicode(os.path.join(dir, file), 'latin-1').encode("utf-8", 'ignore')
+ #file = unicode(os.path.join(dir, file))
except UnicodeDecodeError:
xlmisc.log("Error decoding filename %s" % file)
continue
}}}
This ticket was migrated from the old trac: re #79
Any news from this ?
I have filenames which have strange encodings. I guess something is wrong with them and I should fix this. But all previous player I used can still use them. Exaile fails to import them and stops, which is very anoying.