Should autodetect filename character encoding in zip files

that's known upstream, you can track it here: http://bugzilla.gnome.org/show_bug.cgi?id=306403

I confirm this bug. I use cyrillic symbols. And my partners often can not use archives from me on other OSes. Be cause they can read only CP1251.

I think, that better solution rathter than:
Expected results:
File-roller should attempt to detect the encoding of the filename and do the
appropriate conversion (iconv-style) to UTF-8. If it cannot do a conversion, it
should nevertheless make the file accessible. For example, unconverted
characters could be changed to 0xFFFD (Unicode Replacement Character,
http://www.fileformat.info/info/unicode/char/fffd/index.htm).

Is to allow user to pick input encoding from combobox (like html input encoding in firefox)
Of course, File-roller can try to detect.

I need to confirm this bug. I use Central European Windows cp1205. I have to use Windows apps to extract *.rar archives.

I have sam problem (CP1251) :/

Same problem here with german umlauts (äöü).
Please fix that as fast as possible.
Cannot use my computer with Ubuntu for my work while this isn´t fixed.

I have a very much the same trouble with Cyrillic letters. When in Windows, everything is OK. When I use unzip in the console, everything is OK. When I open the archive with file-roller, it shows garbaged encoding.

The problem should be solved if I had a chance to change the encoding in file-roller.

Btw, AFAIK, the issue should concern mainly zips, not 7z or rar - because they have Unicode chars support (or am I wrong here?).

"Btw, AFAIK, the issue should concern mainly zips, not 7z or rar - because they have Unicode chars support (or am I wrong here?)."
RAR had used until some version default windows codepages and then it was switched to Unicode.

2010 year, and this problem is still here. Whole world can't work with zip from Ubuntu because of small and very easy to fix problem. But zip is one of the most popular archives... Perfect!

Same problem with zip-archives, created on Windows, what contains cyrillic letters. Rar, 7z works perfect.
Interestingly, in Ubuntu 8.04, this error is absent. But at 9.04, 9.10 is present.
Bug is already 2 years! Is it so difficult to fix?!

same problem here with zip-archives, created on Windows with Ä, Ö, Ü letters

Ubuntu 9.04
File-Roller 2.28.1

File Roller 2.28.1, same issue on ubuntu 9.10, but I know, that on 8.04 I have no problem in zip archives!
gibt es irgendwelche Bewegung beim Lösen dieses Problems?

I mean, are there any movement to solve this problem?

please do not reassign bugs without giving a good explanation, this is a file-roller issue.

This issue has been well known since 2006 and no action has been taken to deal with the issue which affects users with zip files containing filenames encoded with non latin characters. It's a tremendous problem which involves batch renaming of thousands of files. The user must identify the filename encoding correctly and the specialized tools have to be found to handle the encoding conversion.

Repost from bug 34667.

Archives of files with ISO-8859-1 encoded filenames with special characters are not extracted correctly into the default UTF-8 encoded Ubuntu file system.

For example, björk.ogg -> bj?rk.ogg (invalid encoding) after extraction from an archive with ISO-8859-1 filenames.

The UTF-8 migration tool does not rectify this problem, and the migration tool does not work at all on partitions outside /home.

Within a UTF-8 encoded file system, File-Roller should be able to seemlessly extract files with non-UTF-8 encoded filenames, so that the user does not have to manually convert the filenames as a second step.

Many users are needlessly shocked to learn that they seem to have lost all their filename encoding after migrating to Ubuntu.

Bug first reported on 20 March 2006! Please upgrade the importance level from low to something higher!

Here's my old post on how to deal with the filename encoding issue.

http://ubuntuforums.org/showthread.php?t=144297

Apparently there is an application called utrac that can be used to identify the encoding of a given text, and then convmv can be used to re-encode the filename (text) to the encoding of the user's filesystem.

utrac
http://utrac.sourceforge.net/

convmv
http://j3e.de/linux/convmv/

The issue's been known since 3 June 2005!

Please, somebody, help with this bug! ZIP is still widely used, but ZIP is not aware about i18n an Unicode.

It's problem of backend - unzip or 7z. In some russian distributions, such as AltLinux, unzip patched to fix this bug.
P.S. Bug open since 2005 year, epic fail...

I have the same problem as long as I can remember. It's been almost five years! Can't you fix it already?

I confirm this bug. When it will be fixed?

The absence of any developers reaction may be caused, I mean, of that is not only file-roller fault. Backends (unzip and AFAIK p7zip too) do not have any option to convert codepage in filename. (BTW, ZIP archives use OEM codepage - that is 866 for cyrillic not 1251)

I confirm this bug.

The given problem is in a package unzip so so not very well what gui-archiver you use. All it use in the operation a package unzip. in the operation a package unzip.
In assemblage for Ubuntu 9.04 such problems were not therefore, makeshift variants:

1. To put a package from 9.04 (http://ubuntuforums.org/archive/index.php/t-37872.html) and to freeze it
sudo -s
echo unzip hold | dpkg --set-selections

2. To gather it it from under the necessary version Ubuntu from source codes for 9.04.

Excuse, here a real address on a package

http://packages.ubuntu.com/ru/jaunty/i386/unzip/download

Don't change bug settings if you don't know what you are doing

some prblem

This error (cp1251 codepage in zip archive) was not present in ubuntu 8.04, but in ubuntu is 10.04, it appeared again! The problem comes in two packages. The package Unzip 6.0 disappeared option to specify the character set (5.52 in older distributions ubuntu had options iconv-O-I) If we take unzip from the old distribution, then in the console and Midnight Commander all OK, but file-roller is still incorrectly displays the file names. Please fix this problem. As a system administrator must say that this is a very serious problem for many users! Please change importance to CRITICAL !

temporary fix:

install convmv:
    sudo apt-get install convmv

for greek encoding:

  LANG=el_GR.CP737 7z e file.zip
  convmv -f CP737 -t utf-8 *

I'll try to patch this in "unp"

The better way is to use natspec library and patched zip.
http://sisyphus.ru/en/srpm/Sisyphus/libnatspec
http://sisyphus.ru/en/srpm/Sisyphus/unzip/patches

I guess we should make this bug as duplicate of https://bugs.launchpad.net/ubuntu/+source/unzip/+bug/477755

When it will work correct with Russian lang in zip file???

I confirm this bug, it affects spanish "special characters" such as á and é. I'm using Lucid 10.04 RC.
This characters are really common in filenames in my country (Argentina).

I have a lot of iso-8859-2 or Windows-1250 encoded zip files. Non-ascii named files can't be extracted with File Roller. unzip(1) does it (keeping invalid utf8 file names), but at least I can rename and use those files. File Roller gives error message if I want to rename the contained files or if I want to extract it (caution: filename not matched: Gyakorlat hallgat\?i seg\?dlet \- 7. Webes alkalmaz\?sok fejleszt\?se.doc).

Problem with curillic

confirmed for Chinese! please fix

i have the same problem ubuntu 10.04

Problem with cirillic. Ubuntu 10.04.1

Unbelievable! 2006-03-20 - 2010-11-02! When this ancient bug will be fixed??? The importance is CRITICAL RED! Same problem (CP1251) Ubuntu 10.10

I confirm this bug. Ubuntu 10.10. The solution is very impotant - this bug is the reason why I don't recomend Ubuntu for my friends and paraents.

Confirm this bug in Ubuntu 10.10. Please fix it. This complicates the use of system in Ukraine, Russia, Belarus, Serbia, Bulgaria and others...

I don't know why, but reintalling unrar fixed the encoding in .rar files:

sudo apt-get remove rar
sudo apt-get remove --purge unrar
sudo apt-get install unrar

Everything opens fine now, at least in .rar archives.

Works for CP-1251 Ru_Cyrillic.
UNRAR 3.93 freeware
File Roller 2.32.0
Ubuntu 10.10 Maverick Kernel 2.6.35-23-generic

I confirm this bug in Ubuntu 10.04. Please fix it.

I confirm this bug in Ubuntu 9.10. Please fix it. ZIP

This is very critical bug for all of our PCs deployed in our offices in Russia. Please provide a fix. Or at least include these patches: http://sisyphus.ru/en/srpm/Sisyphus/unzip/patches

i think it's easy to compyle unzip independently and upload it to ppa, than to wait here.
http://www.opennet.ru/tips/2494_zip_rus_patch.shtml

Does your patch work only for Russian language users, or will it work for all languages?

Please do not add any more "fix it" comments to this bug. The only thing that does is make it much harder for any developer to read what is really happening. The more times that is added to the bug, the less chance the developer will be able to wade through the non-comments to determine if there is an issue and how to resolve it.

I can confirm the problem in a French environment : the accented characters are not dealt with correctly by file-roller (they become question marks). I'm aware that the problem lies with the zip format itself; invoking unzip directly from the command line does extract the "badly named" files, albeit with an " (invalid encoding)" comment appended to the name ---at least this way I can rename the offending files manually.

The problem lies in that file-roller issues a warning about the "filename not matched" but then does not provide a workaround. It should at the very least propose to decompress the files using some simple substitution scheme (keeping the question marks, replacing them with underscores, anything!) or maybe prompt the user to rename each offending file on the fly.

Bottom line: file-roller should decompress SOMETHING.

Can confirm this bug. Goddamn ubuntu arc. apps are useless crap with this bug!

As for workaround Izarc can be installed in wine. It is free, and can be configured to auto-open zip, rar and other orchives from file manager.
But it is sad that the native apps are almoust useless with widespread windows encoding (

Thank you for your comment. To maintain a respectful atmosphere, please follow the code of conduct - http://www.ubuntu.com/community/conduct/ . Bug reports are handled by humans, the majority of whom are volunteers, so please bear this in mind.

I can confirm comment #46.
In my case, simply uninstalling rar,
solved the problem with name encoding in rar files.
Now, file roller works as expected.

Using "unrar x file.rar" in command line
works correctly in both cases,
eg. with rar package installed or not.

Seems that rar, somehow,
interferes with file-roller behavior.

Ark, is not influenced by the presence or not of rar package.
Looking at the file-roller and ark source,
the problem with unrar command implementation,
lies in /file-roller-2.30.1.1/src/fr-command-rar.cin file

Not a developer, but in the aforementioned file,
changing the order of these two lines as follows,
and compiling solves the problem in file-roller...

if (have_rar ())
                fr_process_begin_command (comm->process, "unrar");
 else
  fr_process_begin_command (comm->process, "rar");

HopeThatHelps

Relative to my previous comment, the modified fr-command-rar.c

Where to get ppa with fixed error? I got wrong symbols in names in my zip-files. It is cyrillic.

I am marking the xarchiver task only as invalid. No where in this report and the duplicates can I find a reference to xarchiver failing to uncompress zip files.

Charlie is mincing words to get around the problem of facing the issue. The bug is valid, because xarchiver is NOT unzipping the contents of the archive: it is copying its contents to seemingly randomly-named other files. The original zipped files are NOT recovered: you get other-named files with the same content. This is untenable when unzipping a hierarchy of files and folders, or just multiple files: how is the user going to match the weird output with his expectation? He can't.

Definitely not "invalid", inasmuch as the original (Unicode-named) files are NOT recreated by xarchiver.

Daniel: Please tell us which comment in this report refers to the xarchiver application. file-roller and xarchiver are not the same application. My words are not minced. xarchiver is never reported to fail in this report.

I was sure I had seen someone explaining the problem as arising from xarchiver, but I seem to have been mistaken. I now understand (I'm new to this bugs.launchpad) that this bug had been assigned to multiple applications (file-roller and xarchiver). In that sense, xarchiver is possibly blameless (I'm no expert on this), the problem lying only with file-roller. I apologise for my premature over-reaction.

As long as we're all agreed that the lack of Unicode filename support from file-roller is a problem...

I confirm this bug on 10.04.1 - 2.6.32-28-generic

Got around it by installing p7zip and p7zip-full

Ed

fix this problem please

fix bug please! rar and zip

HERE IS THE FIX:
sudo apt-get remove --purge rar

I confirm this annoying bug!

I confirm this annoying bug too!

Please, dear developers, fix it!

I confirm this bug in Ubuntu 10.04 in .zip files, what contains cyrillic letters.

Obviously, the purge-fix above is not the fix but the "break" :). Actually, it seems that the problem is inside the p7zip package. The 7z program does not convert the character coding. It has the "scs" switch to convert the charset but it is not implemented in versions up to the p7zip_9.20.1.
The cyrillic names in Windows-created ZIP file are stored in cp866. Surprisingly it is pretty difficult to add cp866 locale to Ubuntu. The typical solution:
$ sudo /usr/share/locales/install-language-pack ru_RU.CP866
$ sudo locale-gen
fails with silent error on the first command. Other solutions fail too (cannot open locale definition file ru_RU.CP866: No such file or directory). The following command does not list cp866 too:
cat /usr/share/i18n/SUPPORTED | grep ^ru
The easiest way to deal with cp866 is to set LC_CTYPE to C (it means no conversion), and convert the output with iconv.
--
The workaround is attached as a patch. Apply it to /usr/bin/7z.
7z-natspec.patch: autodetects your DOS charset. Requires libnatspec to be installed (see the next attachment).
7z.patch: please set manually windows_codepage (2nd line) to your DOS charset.
--
The solution described is applicable to any region-specific charset.

For those who do not want to install the libnatspec.

libnatspec can be found at https://launchpad.net/~r0lf/+archive/ppa/+build/1911465 (ver. 0.2.6)
The project page seems to be http://sisyphus.ru/ru/srpm/libnatspec
A bit fresher amd64 build downloaded from there and converted from RPM to DEB is in the attachment.

You can download a bit fresher build from http://sisyphus.ru/ru/srpm/libnatspec manually and convert it from RPM to DEB with alien.
Note: If you choose to do so, please ignore the "broken package" warning and do not forget that starting with Ubuntu 11.10 the library path is not /lib64 but /lib. libnatspec installs into /lib64, thus you will have to move it to /lib manually.
That's why I've decided not to post an attachment.

Correct natspec 7z version.
Works only for viewing files.

The attachment "Autodetects our DOS charset. Requires libnatspec to be installed." of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

OK, some investigation and code work helped reveal the fix.
You need to do the following operations:
1. download the p7zip source package.
2. unpack the package and cd to its parent directory.
3. download the patch (see attachment).
4. patch -p0 <p7zip-9.20.1-manual-iconv-lc_dos.patch
5. "make 7z" inside the p7zip src package and copy ./bin/7z and ./bin/7z.so to /usr/lib/p7zip/ to fix the File Roller.
You need to "make 7z 7za sfx" and copy ./bin/* to /usr/lib/p7zip to replace p7zip binaries completely.
---
How to use the patched p7zip:
Just call p7zip / any program using it (e.g. file-roller) with LC_DOS environment variable set to the respective desired DOS charset. The correspondence between your Ubuntu language and DOS charset can be detected in 2 ways:
1. libnatspec library: sources at http://sisyphus.ru/ru/srpm/libnatspec ; binaries at https://launchpad.net/~r0lf/+archive/ppa/+build/1911465 . Just use the construction like LC_DOS=cp`natspec -c`
2. gettext itself provides the language-charset table because of DJGPP support. Use the following construction:
echo `/usr/share/gettext/intl/config.charset msdosdjgpp | grep \`echo $LANG | cut -f1 -d.\` | cut -f2 -d' '`
Of course, the 2nd solution won't work perfectly inside a .desktop file.
Example:
LC_DOS=cp866 file-roller
You also can download the attachment in the post below. It contains .desktop file and compiled 7z binaries for amd64. Then:
1. Drag and drop that .desktop file into the Main Menu application (alacarte). After that you can create DOS/Win32-compliant ZIP archive by starting the "Archive Manager - natspec DOS charset" from main menu.
2. Right click on any ZIP archive and select Properties->Open with->Show other ...->Archive Manager - natspec DOS charset. After that you can open DOS/Win32 archives by right-clicking on the file and selecting "Open withArchive Manager - natspec ....".

This attachment contains .desktop file and compiled 7z binaries for amd64. It also contains 2 patches:
1. The patch described above.
2. The old 2009-patch for p7zip-4.65 with the similar fix. The difference is the environment variable name: it is ARCHIVE_CP (for the p7zip-9.20 patch it is LC_DOS).

I am using Mint 10 (Julia) based on Ubuntu 10.10. I had problem when trying to extract files with Turkish letter (ş, ç, ö, ğ ...) from and archive. It gave naming error. There was question marks in place of those letters.
I search "archive" in software manager and I uninstalled many relevant (at least I thought so) components like p7zip, unzip, unrar, file-roller etc. Then I installed deb package of Peazip from its website. I didn't like it then I uninstalled it and installed file-roller only (from software manager). I don't know how but the problem has disappeared.

Confirming bug in Oneric (11.10) (x64) for cyrillic (Win 1251) filenames.

Reinstallation of rar as stated in post #46 by Artem fixes the issues.

In terminal:
sudo apt-get remove rar
sudo apt-get remove --purge unrar
sudo apt-get install unrar

Confirmed Turkish character erronously showed as '?' question marks on one of my clients.
OS: Ubuntu 12.04 LTS
File Roller version: 3.4.1
Attached a test Turkish encoded (both file name and zip content) zip sample
Kind Regards
--
Ali Geyik

I can confirm that the workaround described in #46 worked for me. Thank you, temik!

(Ubuntu 12.04.3 LTS)

и меня это бесит тоже, поправьте, пожалуйста

I confirm this bug on Ubuntu 14.04.2 LTS.
Can't read zip files with cyrillic filenames created in Windows.
Почините, пожалуйста :)

Не могу прочитать zip-файлы с кириллическими именами файлов, созданными в Windows.

Wrote a patch for unzip fixing this issue:
https://sourceforge.net/p/infozip/patches/29/

The same patch for p7zip:
https://sourceforge.net/p/p7zip/bugs/187/

This is what we call Distributed Bureaucracy out of Centralized Anarchy!