scim-gtk2-immodule will cause apps linked to libstdc++ 5 to crash

Description of problem:
When an application links to an old version of libstdc++, symbols collide,
beacuse scim links to the latest version.

Version-Release number of selected component (if applicable):
1.4.0

How reproducible:
everytime

Steps to Reproduce:
1. Install application like acroread 7.0 which use older stdc++
2. acroread

Actual results:
crash

Expected results:
running okay

I tried --enable-ld-version-script on RHEL4 where the toolchain is older and
more mature than that in FC4+. It succeeds in allowing openoffice.org to run
(which relies on compat-libstdc++ 3.3 instead of the native 3.4 of RHEL4.
Unfortunately it fails to fix acroread.

According to Su Zhe, acroread has internal symbol conflicts where even
--enable-ld-version-script is unable to fix conflicts. acroread crashes when
you launch it with environment variables disabled but with scim installed. I
suspect this is because it is loading gtk+, which loads the immodule into memory.

Jens mentioned that XIM should still work, but this is doubtful given the above.

OK... Jens was right. GTK_IM_MODULE=gtk-im-context-simple allows acroread to work.

--enable-ld-version-script is working in RHEL4 and FC3, but FC4+ it causes total
desktop software failure. Even gdm fails to start and crashes immediately.

We will need Jakub's help on this. There is something about the FC4+ toolchain
or glibc that is causing --enable-ld-version-script built binaries to totally
fail during runtime.

Of course you can't mix different libstdc++.so (major) versions within one
program, unless using dlmopen rather than dlopen.
If you limit to C++ without STL (i.e. limit yourself to just <new>, <exception>
and <typeinfo> headers from C++ headers), you could link libsupc++.a statically
and using a version script avoid exporting those symbols to the rest of the
arch.
Otherwise, your only choices are dlmopen (but in that case it would mean
that a lot of libraries would be loaded twice, unless you can squeeze some of
the 20 libraries immmodule.so loads (are they all really necessary?)),
having separate immmodule.so versions depending on libstdc++.so version, not
using C++ for this kind of library at all or running it in a different process
from the main app.

Thanks, Jakub, but that doesn't explain why --enable-ld-version-script
isn't working with FC4 and later, or does it? :)

Right, Jakub I believe you misunderstand our problem. We know the limitations
of having more than one libstdc++. --enable-ld-version-script for scim's build
allows it to workaround some (but not all) of those limitations. It works fine
on RHEL4 and SuSE's devel with gcc-4.0.0. There is something different about
FC4's toolchains or glibc that causes scim built in this way to totally fail.
That is the specific issue where we need your help.

I don't see how can using a version script for immmodule.so (which is of course a
good thing) and libscim*.so help this. Those libraries link dynamically against
libstdc++.so, so the version script has no influence on what symbols are exported
from other shared libraries. libstdc++.so uses symbol versioning as well, but
if you use STL in the main program linked say with the old libstdc++.so (3.3 and
earlier), whatever makes into the binary and/or its libraries and is being
exported will be unversioned. This results in mixing of incompatible interfaces.
You can be lucky, as you from the above were in RHEL4. You aren't lucky anymore.
That's it.
It all depends on what exact symbols do get used and exported from where and
how they changed. Simply live with the fact that you really can't do this
reliably.

We are aware that we cannot avoid all problems due to apps that are linked to an
older libstdc++. The problem that we need help solving is the complete failure
of scim when it is built with --enable-ld-version-script, when dealing with only
C and the current version of libstdc++.

There are no details here about how it crashes, backtraces etc.
One possible thing could be that some C++ symbols really ought to be always
exported.
Say if you hide using version script
_ZZN9__gnu_cxx20__common_pool_policyINS_6__poolELb1EE11_S_get_poolEvE7_S_pool
and
_ZGVZN9__gnu_cxx20__common_pool_policyINS_6__poolELb1EE11_S_get_poolEvE7_S_pool
and then allocate some object using STL allocator in one shared library and
destroy it in a different library, you lose, as here libstdc++ relies on ODR.
Similarly with other _S_pool variable objects. Also, if you use RTTI, libstdc++
uses simple pointer comparison for RTTI checks; see
/usr/include/c++/4.0*/typeinfo header.
If you hide the typeinfo symbols (I think these are _ZTI* ones) and try to use
RTTI in one shared library on an object created in another shared library, RTTI
might give wrong answers.

qwang apparently found that RHEL4's libstdc++-3.3 built openoffice.org is not
perfect with the ld version script of libsstdc++-3.4 scim, with scim failing
permanently after a while during usage.

I'm afraid we need to explore other ways of avoiding this problem, because even
if we find the solution to the total failure issue with ld version scripts of
FC4+, it isn't anywhere near good enough to do what we need.

Jakub, upstream told me that this symbol conflict problem will go away
once everyone is using libstdc++ >= 4.0. Is that correct?

qwang's finding in Comment #10 turned out to happen with GTK_IM_MODULE=xim too,
so it is unlikely related to this C++ compat issue.

> Jakub, upstream told me that this symbol conflict problem will go away
> once everyone is using libstdc++ >= 4.0. Is that correct?

Isn't it overly optimistic to think that the C++ ABI will never change again? I
also don't like the idea of abandoning compatibility with all apps compiled with
an older C++ compiler.

Warren, comment 11 is just a side question:
whether gcc 4 and later avoid this problem or not.

https://bugzilla.novell.com/show_bug.cgi?id=85416&x=8&y=7
Novell's equivalent bug where they marked it as "FIXED" despite it being not
perfect.

From Comment #7, we need details of the complete failure of SCIM under normal
conditions (no mixing of libstdc++ versions) in order for Jakub to be able to
help us diagnose this problem. Setting NEEDINFO_ENG.

Here is a gdb backtrace with gedit. The backtrace occurs when switching
to SCIM in the IM menu on Button3. This is with scim built configured with
--enable-ld-version-script.

$ gdb gedit
GNU gdb Red Hat Linux (6.3.0.0-1.65rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib64/libthread_db.so.1".

(gdb) r
Starting program: /usr/bin/gedit
(no debugging symbols found)
:
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 46912501217584 (LWP 24437)]
(no debugging symbols found)
:
(no debugging symbols found)
Detaching after fork from child process 24441.
(no debugging symbols found)
:
(no debugging symbols found)
Detaching after fork from child process 24444.
GTK Panel of SCIM 1.4.2

*** glibc detected *** /usr/bin/gedit: free(): invalid pointer:
0x0000000000b04d00 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3107b6b0be]
/lib64/libc.so.6(__libc_free+0x6e)[0x3107b6b5ee]
/usr/lib64/scim-1.0/1.4.0/IMEngine/socket.so[0x2aaab6c0370a]
/usr/lib64/scim-1.0/1.4.0/IMEngine/socket.so[0x2aaab6c0608a]
/usr/lib64/scim-1.0/1.4.0/IMEngine/socket.so[0x2aaab6c064c3]
/usr/lib64/gtk-2.0/immodules/im-scim.so[0x2aaab66ac71a]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a58e3f6]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a50e206]
/usr/lib64/libgobject-2.0.so.0(g_closure_invoke+0x110)[0x310880ac61]
/usr/lib64/libgobject-2.0.so.0[0x3108818116]
/usr/lib64/libgobject-2.0.so.0(g_signal_emit_valist+0x3ce)[0x3108818e54]
/usr/lib64/libgobject-2.0.so.0(g_signal_emit+0x83)[0x31088194d2]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a5d8328]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a5e54c0]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a5e9252]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a5e92cc]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a50e206]
/usr/lib64/libgobject-2.0.so.0(g_closure_invoke+0x110)[0x310880ac61]
/usr/lib64/libgobject-2.0.so.0[0x3108818116]
/usr/lib64/libgobject-2.0.so.0(g_signal_emit_valist+0x3ce)[0x3108818e54]
/usr/lib64/libgobject-2.0.so.0(g_signal_emit+0x83)[0x31088194d2]
/usr/lib64/libgtk-x11-2.0.so.0[0x310a5d8328]
/usr/lib64/libgtk-x11-2.0.so.0(gtk_main_do_event+0x325)[0x310a50cf3c]
/usr/lib64/libgdk-x11-2.0.so.0[0x3109842951]
/usr/lib64/libglib-2.0.so.0(g_main_context_dispatch+0x1d5)[0x3108625dae]
/usr/lib64/libglib-2.0.so.0[0x3108628a64]
/usr/lib64/libglib-2.0.so.0(g_main_loop_run+0x192)[0x3108628f50]
/usr/lib64/libgtk-x11-2.0.so.0(gtk_main+0xa1)[0x310a50c4aa]
/usr/bin/gedit(main+0x211)[0x41ef9e]
/lib64/libc.so.6(__libc_start_main+0xef)[0x3107b1cd1f]
/usr/bin/gedit[0x41e059]
======= Memory map: ========
00400000-0046f000 r-xp 00000000 fd:00 4062283
/usr/bin/gedit
0056f000-0057d000 rw-p 0006f000 fd:00 4062283
/usr/bin/gedit
0057d000-00c25000 rw-p 0057d000 00:00 0 [heap]
3107900000-310791a000 r...

Here is a valgrind output with gedit. The crash occurs when switching
to SCIM in the IM menu on Button3. This is with scim built configured with
--enable-ld-version-script. The scim version is 1.4.2-5, scim-pinyin version is
0.5.91-2.

-----------------------------------------------------------------
[root@localhost gedit]# valgrind --leak-check=yes gedit
==3529== Memcheck, a memory error detector.
==3529== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==3529== Using LibVEX rev 1367, a library for dynamic binary translation.
==3529== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==3529== Using valgrind-3.0.1, a dynamic binary instrumentation framework.
==3529== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==3529== For more details, rerun with: -v
==3529==
==3529== Syscall param write(buf) points to uninitialised byte(s)
==3529== at 0x1C461143: __write_nocancel (in /lib/libpthread-2.3.90.so)
==3529== by 0x1C5E420D: (within /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C5E442B: _X11TransWrite (in /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C5C978A: (within /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C5C98A5: _XReply (in /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C5B5924: XInternAtom (in /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C5D2208: XSetWMProperties (in /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C25CA0B: (within /usr/lib/libgdk-x11-2.0.so.0.800.6)
==3529== by 0x1C25F59E: gdk_window_new (in /usr/lib/libgdk-x11-2.0.so.0.800.6
)
==3529== by 0x1C23F279: gdk_display_open (in /usr/lib/libgdk-x11-2.0.so.0.800
6)
==3529== by 0x1C220893: gdk_display_open_default_libgtk_only (in /usr/lib/lib
gdk-x11-2.0.so.0.800.6)
==3529== by 0x1C0124B6: gtk_init_check (in /usr/lib/libgtk-x11-2.0.so.0.800.6
)
==3529== Address 0x1CC8B470 is 128 bytes inside a block of size 16384 alloc'd
==3529== at 0x1B904AD4: calloc (vg_replace_malloc.c:279)
==3529== by 0x1C5B9D59: XOpenDisplay (in /usr/X11R6/lib/libX11.so.6.2)
==3529== by 0x1C23F19D: gdk_display_open (in /usr/lib/libgdk-x11-2.0.so.0.800
6)
==3529== by 0x1C220893: gdk_display_open_default_libgtk_only (in /usr/lib/lib
gdk-x11-2.0.so.0.800.6)
==3529== by 0x1C0124B6: gtk_init_check (in /usr/lib/libgtk-x11-2.0.so.0.800.6
)
==3529== by 0x1C0124E9: gtk_init (in /usr/lib/libgtk-x11-2.0.so.0.800.6)
==3529== by 0x1BAE0CDD: (within /usr/lib/libbonoboui-2.so.0.0.0)
==3529== by 0x1BB3B875: gnome_program_postinit (in
/usr/lib/libgnome-2.so.0.1200.0)
==3529== by 0x1BB3C216: (within /usr/lib/libgnome-2.so.0.1200.0)
==3529== by 0x1BB3C523: gnome_program_init (in
/usr/lib/libgnome-2.so.0.1200.0)
==3529== by 0x806096D: main (in /usr/bin/gedit)
==3529==
==3529== Syscall param write(buf) points to uninitialised byte(s)
==3529== at 0x1C461143: __write_nocancel (in /lib/libpthread-2.3.90.so)
==3529== by 0x1BA943A0: (within /usr/X11R6/lib/libICE.so.6.3)
==3529== by 0x1BA945B0: _IceTransWrite (in /usr/X11R6/lib/libICE.so.6.3)
==3529== by 0x1BA8D4C6: _IceWrite (in /usr/X11R6/lib/libICE.so.6.3)
==3529== by 0x1BA8D59F: IceFlush (in /usr/X11R6/lib/libICE.so.6.3)
==3529== by 0x1BA7E94F: SmcSetPropert...

Here's how I was told to reproduce.

Anyway it is pretty easy to reproduce:
1. checkout scim from package cvs, or we can provide you with
an srpm say if you prefer.
2. move the commented out "--enable-ld-version-script" to the %configure line.
3. build scim.
4. install built scim package.
5. install say scim-pinyin.
6. start a new desktop session in English locale say.
7. start gedit say.
8. button3 menu -> Input Method -> scim
9. watch gedit crash.

If you have any questions or if there is any way we can help
please don't hesitate to ask. :)

Jens

Created attachment 120441
force default visibility for mt_alloc pools

I use the patch above to rebuild gcc, then rebuid scim and scim-pinyin with
"--enable-ld-version-script", but gedit still crash.
The os version is rawhide 20051014, gcc package is from cvs in devel section.

The above patch is not useful toward solving this problem. Benjamin later
determined that versioning weak symbols in libstdc++ is the proper and elegant
solution to solve this problem. Unfortunately it will take a while to implement.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=24660

To be clear, we really have two separate issues in FC4+ affecting us.

1) The above issue where we need versioning of weak symbols in libstdc++ to
solve. Benjamin describes this as an elegant solution to the problem and there
is strong confidence that this will work. It is only a matter of resource
allocation and schedules to work on it.
2) The crashing issue is due to the use of two different allocators. This will
be solved before FC5test2 with gcc-4.1 by reverting back to the previous
libstdc++ allocator used in RHEL4.

Right. (2) has been split out into bug 173220.

NOTE: Please keep all communication regarding libstdc++7 private.

http://porkchop.redhat.com/beehive/comps/playpen/fc5/libstdc++so7/
First packaged attempt of libstdc++7. The -devel package includes the
libstdc++so7-g++ wrapper which you use instead of g++ if you want to build or
link a C++ application against this library.

I attempted to build scim and scim-anthy with this procedure:
1) export CXX=libstdc++so7-g++ before configure and make.
2) Remove --enable-ld-version-script

The end result had /usr/lib/gtk-2.0/immodules/im-scim.so still linked to
libstdc++.so.6 so I did something wrong. Further work is necessary in order to
complete this.

After scim and other scim plugins are fixed, Jakub said that special work must
be taken in order to build scim-qt because some parts of it links against
libstdc++.so.6 and the scim parts need libstdc++7. Here are some notes where he
describes what is necessary.

<jakub> warren: probably split the sources, so that there are some source files
that are using KDE/Qt headers (and those are compiled with g++)
<jakub> warren: then these files comunicate over a C API (well, it can be e.g.
some C++ classes, but they must never contain any STL stuff)
<jakub> warren: with other C++ files that use the SCIM headers and are compiled
with libstdc++so7-g++
<jakub> warren: finally, the link line will be slighly complicated, as you need
to link against both libstdc++'s
<jakub> warren: guess something like libstdc++so7-g++ ... -shared ... `g++
$CXXFLAGS -print-file-name=libstdc++.so`

Status Update:
==============
libstdc++so7 package and scim have been tested, and are now ready. scim-qtimm
requires moderate hacking in order to work with the dual C++ ABI. scim-qtimm
can be fixed later because it is far less important.

libstdc++so7 is based on the libstdc++_7 C++ ABI which would be included in a
yet undecided gcc release. The libstdc++_7 ABI adds the needed "weak symbol
version" feature needed by this scim dual C++ requirement, in addition to other
C++ feature benefits. Benjamin Kosnik wants libstdc++so7 in FC5 in order to
satisfy the scim requirement and also expose libstdc++_7 to wide user testing.
Because libstdc++_7 is pre-alpha, we will guarantee in documentation and release
notes that the ABI and so-version (like libstdc++-20060125.so.7) will change in
future updates.

PackageListProcess has initiated and discussion currently ensues.

libstdc++so7 has been included in dist-fc5 and scim updated and rebuilt against it.

Bug #182177 has been filed to deal with the scim-qtimm NEEDSWORK issue.

I'd consider this to be done. Work continues in Bug #182117 for scim-qtimm and
elsewhere to possibly rewrite gtkimmodule in order to get rid of this
libstdc++so7 entire, but otherwise we're in good shape.

Update for Historical Purposes
==============================
Bug #185693 is where the gtkimmodule was rewritten in C with an abstraction over
a socket, thereby solving this C++ conflict problem. Thanks to Ryo Dairiki for
his great efforts in writing scim-bridge.

*** Bug 217865 has been marked as a duplicate of this bug. ***