Invalid encoding returned for ca@valencia locale

While upgrading from 13.04 to 13.10 using locale ca@valencia, the process fails with the following traceback:

Traceback (most recent call last):

File "/tmp/ubuntu-release-upgrader-vh1jgw/saucy", line 10, in
<module>
sys.exit(main())

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeMain.py",
line 243, in main
if app.run():

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 1791, in run
return self.fullUpgrade()

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 1744, in fullUpgrade
if not self.askDistUpgrade():

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py",
line 996, in askDistUpgrade
self.cache.required_download)

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeViewText.py",
line 165, in confirmChanges
downloadSize, actions)

File
"/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeView.py",
line 387, in confirmChanges
downloadSizeStr = downloadSizeStr.decode(ENCODING)

LookupError: unknown encoding: utf_8_valencia

Looking at it in more detail, this comes from the following code in DistUpgrade/DistUpgradeView.py [1]

try:
    locale.setlocale(locale.LC_ALL, "")
    (code, ENCODING) = locale.getdefaultlocale()
except:
    logging.exception("getting the encoding failed")
    ENCODING = "utf-8" #pyflakes

if not ENCODING:
    ENCODING = "utf-8"

And digging deeper, it seems getdefaultlocale does not return the correct encoding when the locale is set to ca_ES

>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'LC_CTYPE=ca_ES.UTF-8@valencia;LC_NUMERIC=ca_ES.UTF-8;LC_TIME=ca_ES.UTF-8;LC_COLLATE=ca_ES.UTF-8@valencia;LC_MONETARY=ca_ES.UTF-8;LC_MESSAGES=ca_ES.UTF-8@valencia;LC_PAPER=ca_ES.UTF-8;LC_NAME=ca_ES.UTF-8;LC_ADDRESS=ca_ES.UTF-8;LC_TELEPHONE=ca_ES.UTF-8;LC_MEASUREMENT=ca_ES.UTF-8;LC_IDENTIFICATION=ca_ES.UTF-8'
>>> locale.getdefaultlocale()
('ca_ES', 'utf_8_valencia')

Instead of 'utf_8_valencia', here I'd expect 'UTF-8' to be returned. The returned locale is also wrong: it should be ca_ES@valencia.

While this needs to be fixed upstream, it might be worth adding a workaround (and a check for a valid encoding) in ubuntu-release-upgrader in the meantime.

This will also affect other @ locales, such as sr@Latin (Serbian with Latin encoding).

[1] http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/view/head:/DistUpgrade/DistUpgradeView.py#L41