python2.5 compiled with libffi does not support ctypes 64 bits integer

Ok, finally, I know where the problem lies.

The equivalence ctype <-> libffi seems to be done in the file Modules/_ctypes/cfield.c
This file builds a table for the equivalence:

#ifdef HAVE_LONG_LONG
    { 'q', q_set, q_get, &ffi_type_slong, q_set_sw, q_get_sw},
    { 'Q', Q_set, Q_get, &ffi_type_ulong, Q_set_sw, Q_get_sw},
#endif

The type ffi_type_[su]long are defined in ffi.h. In the libffi included with python sources, we have the following:

#define ffi_type_ulong ffi_type_uint64
#define ffi_type_slong ffi_type_sint64
#if LONG_MAX == 2147483647
# if FFI_LONG_LONG_MAX != 9223372036854775807
  #error "no 64-bit data type supported"
# endif
#elif LONG_MAX != 9223372036854775807
 #error "long size not supported"
#endif

Which set the size of fft_type_[us]long to 8 bytes

But in /usr/include/ffi.h, corresponding to the packaged libffi from dapper, we have the following:

#if LONG_MAX == 2147483647
# define ffi_type_ulong ffi_type_uint32
# define ffi_type_slong ffi_type_sint32
#elif LONG_MAX == 9223372036854775807
# define ffi_type_ulong ffi_type_uint64
# define ffi_type_slong ffi_type_sint64
#else
 #error "long size not supported"
#endif

There is a mixup somewhere between long and long long

I can reproduce this behaviour on several Ubuntu boxes with '--with-system-ffi'.

Actually the ctypes unittests throw errors with this configuration, and eventually die with a segfault.

It would probably be a good idea to insert some runtime checks in ctypes itself, the sizes of the various c_... types should be compared with the sizes that the struct module calculates. This will not fix the bug but would avoid that users are bitten by it.

closing the python2.5 task; 2.5-5ubuntu1 is configured without -with-system-ffi

Thomas, please could you followup on http://gcc.gnu.org/12782 ?

closing this report.

current libffi doesn't support long long, the copy in the ctypes extension did "hijack" the implementation for long. so unfortunately on platforms where sizeof(long) != sizeof(long long), --with-system-ffi cannot be used.

Matthias, sorry that it took so long for me to understand the problem. Your last comment made it clear that there is indeed a ctypes bug. Here is a simple fix that should correct the problem (tested on x86 ubuntu feisty, with --with-system-ffi). Should the report be opened again?

Index: Modules/_ctypes/cfield.c
===================================================================
--- Modules/_ctypes/cfield.c (revision 58153)
+++ Modules/_ctypes/cfield.c (working copy)
@@ -1616,18 +1616,22 @@
 /* XXX Hm, sizeof(int) == sizeof(long) doesn't hold on every platform */
 /* As soon as we can get rid of the type codes, this is no longer a problem */
 #if SIZEOF_LONG == 4
- { 'l', l_set, l_get, &ffi_type_sint, l_set_sw, l_get_sw},
- { 'L', L_set, L_get, &ffi_type_uint, L_set_sw, L_get_sw},
+ { 'l', l_set, l_get, &ffi_type_sint32, l_set_sw, l_get_sw},
+ { 'L', L_set, L_get, &ffi_type_uint32, L_set_sw, L_get_sw},
 #elif SIZEOF_LONG == 8
- { 'l', l_set, l_get, &ffi_type_slong, l_set_sw, l_get_sw},
- { 'L', L_set, L_get, &ffi_type_ulong, L_set_sw, L_get_sw},
+ { 'l', l_set, l_get, &ffi_type_sint64, l_set_sw, l_get_sw},
+ { 'L', L_set, L_get, &ffi_type_uint64, L_set_sw, L_get_sw},
 #else
 # error
 #endif
 #ifdef HAVE_LONG_LONG
- { 'q', q_set, q_get, &ffi_type_slong, q_set_sw, q_get_sw},
- { 'Q', Q_set, Q_get, &ffi_type_ulong, Q_set_sw, Q_get_sw},
+#if SIZEOF_LONG_LONG == 8
+ { 'q', q_set, q_get, &ffi_type_sint64, q_set_sw, q_get_sw},
+ { 'Q', Q_set, Q_get, &ffi_type_uint64, Q_set_sw, Q_get_sw},
+#else
+# error
 #endif
+#endif
        { 'P', P_set, P_get, &ffi_type_pointer},
        { 'z', z_set, z_get, &ffi_type_pointer},
 #ifdef CTYPES_UNICODE
@@ -1744,7 +1748,10 @@
 ffi_type ffi_type_uint32 = { 4, 4, FFI_TYPE_UINT32 };
 ffi_type ffi_type_sint32 = { 4, 4, FFI_TYPE_SINT32 };

+ffi_type ffi_type_int = { 4, 4, FFI_TYPE_INT };
+
 ffi_type ffi_type_uint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_UINT64 };
+ffi_type ffi_type_long_long = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 };
 ffi_type ffi_type_sint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 };

 ffi_type ffi_type_float = { sizeof(float), FLOAT_ALIGN, FFI_TYPE_FLOAT };

thanks for looking; I'll check this next week.

tested with the 2.5 branch on i486 and x86_64 without regressions. please consider this for the python25-branch.

python2.5 (2.5.1-5ubuntu4) gutsy; urgency=low

  * Fix segfault in _ctypes module when configured with --with-system-ffi.
    LP: #72505.
  * Configure --with-system-ffi.
  * Enable running the testsuite.
  * Update to 20070929, taken from the 2.5 release branch.
  * Regenerate the documentation.

 -- Matthias Klose <email address hidden> Sat, 29 Sep 2007 12:23:27 +0200