Broken encoding in server's messages

Pretty serious one, marking for Dapper.

I have nailed the problem; now I need to find a solution that fixes this without reintroducing http://bugs.debian.org/343057 again (I'll add checks to the test suite to ensure that).

It seems that postmaster is very picky about its locale environment... :(

Fixed in postgresql-common 53, just uploaded to Debian:

postgresql-common (53) unstable; urgency=medium

  * Urgency medium since this fixes a quite serious bug; no intrusive changes
    otherwise.
  * Add Russian debconf translation, thanks to Yuriy Talakan'
    <email address hidden>! Closes: #367152
  * t/001_packages.t: Check that the locales used in the tests are installed
    so that the test suite fails early if not.
  * t/050_encodings.t, t/060_obsolete_confparams.t: Use ru_RU{,.UTF-8} for
    tests, since they have more potential for failure.
  * t/050_encodings.t:
    - Add check for https://launchpad.net/bugs/39177: Correct encoding of
      server error messages under various locales.
    - Add check for bug #343057: Correct startup if client_encoding and
      lc_messages settings do not match.
  * pg_ctlcluster: Set LC_CTYPE environment variable to unbreak server error
    messages. (Closes https://launchpad.net/bugs/39177). By only setting CTYPE
    we also avoid reintroducing bug #343057. (Yay for postmaster being so anal
    about its environment)
  * t/020_create_sql_remove.t: Consider LC_CTYPE a safe environment variable.
  * debian/postgresql-common.postinst: Bump version comparison for restarting
    postgresql servers to this version, to ensure that above bug fix becomes
    active.

I'll ask for a sync now.

Uploaded, 53ubuntu1 fixes this.

Martin: with the new upgrade now I have the following problem:

ricardo@yuggoth:~ $ psql -U examen examen
Contraseña para usuario examen:
Bienvenido a psql 8.1.3, la terminal interactiva de PostgreSQL.

Digite: \copyright para ver los términos de distribución
       \h para ayuda de comandos SQL
       \? para ayuda de comandos psql
       \g o or termine con punto y coma para ejecutar una consulta
       \q para salir

examen=# select * from test;
ERROR: no existe la relación «test»
examen=#

Obviously, the error message is not ok.
I have the following lines (among others) into my /etc/postgresql/8.1/main/postgresql.conf:

# - Locale and Formatting -

#datestyle = 'iso, mdy'
#timezone = unknown # actually, defaults to TZ
                                        # environment setting
#australian_timezones = off
#extra_float_digits = 0 # min -15, max 2
#client_encoding = sql_ascii # actually, defaults to database
                                        # encoding

# These settings are initialized by initdb -- they might be changed
lc_messages = 'es_ES.UTF-8' # locale for system error message
                                        # strings
lc_monetary = 'es_ES.UTF-8' # locale for monetary formattinglc_numeric = 'es_ES.UTF-8' # locale for number formatting
lc_time = 'es_ES.UTF-8' # locale for time formatting

I never change this file since its installation.

Ricardo,

the output looks as if the locale of your terminal is ISO-8859-X, not UTF-8. What does 'locale' say?

ricardo@yuggoth:~ $ locale
LANG=es_ES.UTF-8
LANGUAGE=es_ES:es:en_GB:en
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE="es_ES.UTF-8"
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=

By the way, I tried using "psql" in both gnome-terminal and tty console (Ctrl-Alt-F1), with the same results.

Changed from "Fix Released" to "In Progress". Dapper Final Release still has this problem. The messages are not easily readable in Spanish due to encoding issues.

This is a regression problem: this issue is not in Breezy.

More info: the problem can be seen in PgAdmin III frontend, too.

As you can see, the error messages from PostgreSQL backend are incorrect in PgAdmin III, too.

Ok, I discover where the error is. I've downloaded the .po file from Rosetta (template postgres-8.1, locale "es"), and I saw an issue:

The problem:
----------------------

The postgres-8.1.po file has an UTF-8 encoding, but the header claims that the encoding is ISO-8859-1:

ricardo@yuggoth:~ $ file postgres-8.1.po
postgres-8.1.po: UTF-8 Unicode PO (gettext message catalogue) text
ricardo@yuggoth:~ $ grep "charset=" postgres-8.1.po
"Content-Type: text/plain; charset=ISO-8859-1\n"

The solution:
---------------------

1. Replace ISO-8859-1 to UTF-8, so we can get the following output:

ricardo@yuggoth:~ $ grep "charset=" postgres-8.1.po
"Content-Type: text/plain; charset=UTF-8\n"

2. Compile the .po file to generate the .mo file:

ricardo@yuggoth:~ $ msgfmt -o postgres-8.1.mo postgres-8.1.po

3. Replace the old /usr/share/locale-langpack/es/LC_MESSAGES/postgres-8.1.mo file with the just generated one:

ricardo@yuggoth:~ $ sudo cp -f postgres-8.1.mo /usr/share/locale-langpack/es/LC_MESSAGES

4. Restart PostgreSQL:
ricardo@yuggoth:~ $ sudo /etc/init.d/postgresql-8.1 restart

Now, I get the messages perfectly corrected:

ricardo@yuggoth:~ $ psql -U examen examen
Contraseña para usuario examen:
Bienvenido a psql 8.1.4, la terminal interactiva de PostgreSQL.

Digite: \copyright para ver los términos de distribución
       \h para ayuda de comandos SQL
       \? para ayuda de comandos psql
       \g o or termine con punto y coma para ejecutar una consulta
       \q para salir

examen=# select * from test;
ERROR: no existe la relación «test»
examen=#

Martin, this is the corrected postgres-8.1.po file. The only change is to replace the (incorrect) line:

"Content-Type: text/plain; charset=ISO-8859-1\n"

with this (correct) one:

"Content-Type: text/plain; charset=UTF-8\n"

Can you upload this file into Rosetta, as template "postgres-8.1" and locale "es"?

Martin, according to Carlos guidelines, I just made the upload of the corrected .po file myself.

Ricardo, thank you! Sorry for the delay.

Don't worry :) I only didn't knew that I can upload the file myself, but then Carlos told me that I can, so I did it.

Thanks to you, Martin :)

Ricardo uploaded the fix, so I'm marking as fix released.