Confusing new locale-gen behavior

Well, after having looked at the /usr/sbin/locale-gen script and done some further testing, I see that not *all* arguments are ignored. The synopsis seems to be:

locale-gen [[--keep-existing] LANGUAGE]

where LANGUAGE is the name of a file in /var/lib/locales/supported.d. (If you state multiple LANGUAGE arguments, only the last is recognized.) The --keep-existing option prevents purging of previously generated locales.

So basically you can

* Purge all existing locales and generate all locales in
  /var/lib/locales/supported.d

  sudo locale-gen

* Purge all existing locales and generate the locales of *one* file
  in /var/lib/locales/supported.d

  sudo locale-gen LANGUAGE

* [Re-]generate the locales of *one* file in
  /var/lib/locales/supported.d language-pack without purging

  sudo locale-gen --keep-existing LANGUAGE

This is quite different compared to the way the old locale-gen worked.

* All locales are purged by default

* No simple way to generate locales included in SUPPORTED in addition
  to the locales belonging to installed language packs.

* The man page is pretty useless.

I fear that this will result in a lot user confusion. Possibly there is some point with the new behavior, but at this time I fail to see it. In any case the change should be accompanied by some proper documentation.

I would suggest that we use the old locale-gen script (and the related man page) in 16.04. Then, post Xenial, we could consider more thoroughly what to do going forward.

I made a couple of minor changes to the script (attached). Those changes would improve it a bit IMO, if going back to the old script is considered 'too much'. Especially they would prevent that a command like

sudo locale-gen en_US.UTF-8

or

sudo locale-gen --purge

results in a broken locale without the user being aware of what happened. The modified script also allows multiple (valid) language code arguments, so

sudo locale-gen en ja

generates the English and Japanese locales, and not only the Japanese.

Status changed to 'Confirmed' because the bug affects multiple users.

glibc has been updated, and the locale-gen script, which is shipped with locales 2.23-0ubuntu2, addresses most of my concern.

I still found a reason to propose some minor changes:

* Allow multiple language arguments
* Make sure (via '|| true') that the error message is displayed if
  the language/locale isn't supported.

New diff attached.

Can you please have a look at
https://bugs.launchpad.net/ubuntu/+source/language-pack-en-base/+bug/1562279

Although I installed
language-pack-gnome-pl
language-pack-gnome-pl-base
language-pack-pl
language-pack-pl-base

I still get:

sudo locale-gen "pl_PL.UTF-8"/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Generating locales (this might take a while)...
  pl_PL.UTF-8...cannot open locale definition file `pl_PL': No such file or directory
 done
Generation complete.

@axel: I suspect that the locales package hasn't been properly installed. Try:

sudo apt-get install --reinstall locales
sudo locale-gen pl_PL.UTF-8

locale -a
C
C.UTF-8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
pl_PL.utf8
POSIX

locale
LANG=pl_PL.UTF-8
LANGUAGE=pl_PL:pl:en_GB:en
LC_CTYPE="pl_PL.UTF-8"
LC_NUMERIC="pl_PL.UTF-8"
LC_TIME="pl_PL.UTF-8"
LC_COLLATE="pl_PL.UTF-8"
LC_MONETARY="pl_PL.UTF-8"
LC_MESSAGES="pl_PL.UTF-8"
LC_PAPER="pl_PL.UTF-8"
LC_NAME="pl_PL.UTF-8"
LC_ADDRESS="pl_PL.UTF-8"
LC_TELEPHONE="pl_PL.UTF-8"
LC_MEASUREMENT="pl_PL.UTF-8"
LC_IDENTIFICATION="pl_PL.UTF-8"
LC_ALL=pl_PL.UTF-8

I also did
sudo dpkg-reconfigure locales

and it helped. It is ok now, also in wine. Thank you very much.

I encountered another issue at <http://askubuntu.com/q/753516>: If you for instance pass the argument "en_IN.UTF-8", locale-gen currently sees it as unsupported, since the real name in SUPPORTED is just "en_IN". New locales tend to be UTF-8 only, and I think locale-gen should accept arguments with the codeset appended in these cases.

@Adam: I made a patch out of my proposed changes. If other glibc stuff is on its way in before final freeze, you may want to consider it together with them. Otherwise I think the proposed changes would motivate a separate upload.

This bug was fixed in the package glibc - 2.23-0ubuntu3

---------------
glibc (2.23-0ubuntu3) xenial; urgency=medium

  * Merge with 2.23 from experimental, bringing in upstream updates:
    - Save/restore fprs/vrs while resolving symbols (LP: #1564918)
    - Fix _nss_dns_getnetbyname_r() stack overflow (CVE-2016-3075)
    - Merge libnss-dns-udeb and libnss-files-udeb into libc6-udeb.
  * Tidy up locale-gen, thanks to Gunnar Hjalmarsson (LP: #1560577):
    - Fix thinko that broke handling of multiple locale arguments.
    - Recognize UTF-8 locales without charset suffix in SUPPORTED.
    - Fix bug that led to the unsupported message not being shown.
  * Show reboot-required notification for all updates (LP: #1546457)

 -- Adam Conrad <email address hidden> Thu, 14 Apr 2016 10:26:16 -0600