PHP5 Segfault - Backtrace included - ZEND_DECLARE_FUNCTION_SPEC_HANDLER / do_bind_function

Bug #1294399 reported by William Attwood
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
php5 (Ubuntu)
New
Medium
Unassigned

Bug Description

Backtrace:
#0 0x00007f6892c1230d in do_bind_function (opline=0x7f680b35f010, function_table=0x7f689800b640, compile_time=0 '\000') at /build/buildd/php5-5.3.10/Zend/zend_compile.c:2978
#1 0x00007f6892c50bbc in ZEND_DECLARE_FUNCTION_SPEC_HANDLER (execute_data=0x7f68985386b0) at /build/buildd/php5-5.3.10/Zend/zend_vm_execute.h:586
#2 0x00007f6892c5093b in execute (op_array=0x7f689806b260) at /build/buildd/php5-5.3.10/Zend/zend_vm_execute.h:107
#3 0x00007f6892c2bea0 in zend_execute_scripts (type=0, retval=0x898036ef8, file_count=3) at /build/buildd/php5-5.3.10/Zend/zend.c:1308
#4 0x00007f6892bd8513 in php_execute_script (primary_file=0x0) at /build/buildd/php5-5.3.10/main/main.c:2323
#5 0x00007f6892cbb3ad in php_handler (r=0x7f6892cbb3ad) at /build/buildd/php5-5.3.10/sapi/apache2handler/sapi_apache2.c:688
#6 0x00007f6896f21508 in ap_run_handler ()
#7 0x00007f6896f2197e in ap_invoke_handler ()
#8 0x00007f6896f30bdc in ap_internal_redirect ()
#9 0x00007f6890eb45e5 in ?? () from /usr/lib/apache2/modules/mod_rewrite.so
#10 0x00007f6896f21508 in ap_run_handler ()
#11 0x00007f6896f2197e in ap_invoke_handler ()
#12 0x00007f6896f31570 in ap_process_request ()
#13 0x00007f6896f2e398 in ?? ()
#14 0x00007f6896f27fa8 in ap_run_process_connection ()
#15 0x00007f6896f361d0 in ?? ()
#16 0x00007f6896f3693a in ?? ()
#17 0x00007f6896f374e7 in ap_mpm_run ()
#18 0x00007f6896f0c4a4 in main ()

This is not consistent, with many hundreds of successful loads, then a few failures:
Mar 18 16:51:49 localhost haproxy[4897]: 121.205.241.0:50119 [18/Mar/2014:16:51:49.190] http_80 http_80/tcsweb23 4/0/0/-1/798 502 204 - - SH-- 81/81/23/0/0 0/0 {} "POST /catalog/request_quote HTTP/1.1"
Mar 18 16:52:58 localhost haproxy[4897]: 166.70.206.46:49593 [18/Mar/2014:16:52:53.732] http_80 http_80/tcsweb20 3063/0/0/-1/4894 502 204 - - SH-- 89/88/35/3/0 0/0 {} "GET /tires-auto-repair-burlington-nc HTTP/1.1"
Mar 18 16:54:17 localhost haproxy[4897]: 183.60.213.30:56075 [18/Mar/2014:16:52:25.769] http_80 http_80/tcsweb20 2/0/0/-1/111316 502 204 - - SH-- 46/46/21/0/0 0/0 {} "GET /engine-diagnostics-tips HTTP/1.1"
Mar 18 17:17:28 localhost haproxy[4897]: 183.60.213.30:34079 [18/Mar/2014:17:14:54.823] http_80 http_80/tcsweb20 3119/0/0/-1/153536 502 204 - - SH-- 77/77/29/1/0 0/0 {} "GET /blog/index/tag:windshield-replacement HTTP/1.1"
Mar 18 17:18:15 localhost haproxy[4897]: 121.205.249.25:56370 [18/Mar/2014:17:18:10.976] http_80 http_80/tcsweb24 38/0/0/-1/4451 502 204 - - SH-- 54/54/25/1/0 0/0 {} "POST /catalog/request_quote HTTP/1.1"

Restarting Apache clears it up most times. One restart event did not clear it up.

7 different core dumps, 3 different web servers, show the same details:
[0x7f68985386b0] ??? ...elements/nap.ctp:105
[0x7f6898537638] ??? ...view.php:1159

Where nap.ctp:105 is:
if(!function_exists('napitemcheck')){
        function napitemcheck($name, $setting, $schema, $highlight, $exclude, $newline) {

This is in alignment with ZEND_DECLARE_FUNCTION_SPEC_HANDLER, then the crash on do_bind_function.

php -v
PHP 5.3.10-1ubuntu3.10 with Suhosin-Patch (cli) (built: Feb 28 2014 23:14:25)
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2012 Zend Technologies

php -m
[PHP Modules]
apc
bcmath
bz2
calendar
Core
ctype
curl
date
dba
dom
ereg
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
imagick
json
libxml
mbstring
mcrypt
memcache
mhash
mssql
mysql
mysqli
openssl
pcntl
pcre
PDO
pdo_dblib
pdo_mysql
Phar
posix
readline
Reflection
session
shmop
SimpleXML
soap
sockets
SPL
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlwriter
zip
zlib

[Zend Modules]

APC details:
   APC Version 3.1.7
   PHP Version 5.3.10-1ubuntu3.10
   APC Host tcsweb20 ()
   Server Software Apache
   Shared Memory 1 Segment(s) with 2.0 GBytes
                       (mmap memory, pthread mutex Locks locking)
   Start Time 2014/03/18 16:28:37
   Uptime 58 minutes
   File Upload Support 1

File Cache Information

   Cached Files 4886 (219.8 MBytes)
   Hits 466156
   Misses 4960
   Request Rate (hits, misses) 134.26 cache requests/second
   Hit Rate 132.85 cache requests/second
   Miss Rate 1.41 cache requests/second
   Insert Rate 1.39 cache requests/second
   Cache full count 0

User Cache Information

   Cached Variables 1233 ( 43.7 MBytes)
   Hits 43780
   Misses 4185
   Request Rate (hits, misses) 13.67 cache requests/second
   Hit Rate 12.48 cache requests/second
   Miss Rate 1.19 cache requests/second
   Insert Rate 0.89 cache requests/second
   Cache full count 0

Runtime Settings

   apc.cache_by_default 1
   apc.canonicalize 1
   apc.coredump_unmap 0
   apc.enable_cli 0
   apc.enabled 1
   apc.file_md5 0
   apc.file_update_protection 2
  apc.filters
   apc.gc_ttl 600
   apc.include_once_override 0
   apc.lazy_classes 0
   apc.lazy_functions 0
   apc.max_file_size 20M
   apc.mmap_file_mask /dev/zero
   apc.num_files_hint 2700
   apc.preload_path
   apc.report_autofilter 0
   apc.rfc1867 0
   apc.rfc1867_freq 0
   apc.rfc1867_name APC_UPLOAD_PROGRESS
   apc.rfc1867_prefix upload_
   apc.rfc1867_ttl 3600
   apc.serializer default
   apc.shm_segments 1
   apc.shm_size 2048M
   apc.slam_defense 0
   apc.stat 1
   apc.stat_ctime 0
   apc.ttl 7200
   apc.use_request_time 1
   apc.user_entries_hint 2700
   apc.user_ttl 3600
   apc.write_lock 1

Host Status Diagrams

   Memory Usage
   (multiple slices indicate fragments) Hits & Misses
    Free: 1.7 GBytes (87.1%) Hits: 466156 (98.9%)
    Used: 264.9 MBytes (12.9%) Misses: 4960 (1.1%)

Detailed Memory Usage and Fragmentation

   Fragmentation: 0.05% (997.8 KBytes out of 1.7 GBytes in 687 fragments)

free -m
             total used free shared buffers cached
Mem: 4963 2841 2121 0 15 2427
-/+ buffers/cache: 399 4563
Swap: 371 33 338

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/tcsweb09-root 2.8G 2.4G 273M 90% /
....

Robie Basak (racb)
Changed in php5 (Ubuntu):
importance: Undecided → Medium
Revision history for this message
William Attwood (wattwood-p) wrote :
Download full text (3.5 KiB)

Upgraded to APC 3.1.13. No change, still segfaulting. Upgraded to latest kernal:
Linux 3.2.0-26-generic #41-Ubuntu SMP Thu Jun 14 17:49:24 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Program terminated with signal 11, Segmentation fault.
t#0 0x00007fc6d9d4330d in do_bind_function (opline=0x7fc64eac5ab0, function_table=0x7fc6de5a2a90, compile_time=0 '\000') at /build/buildd/php5-5.3.10/Zend/zend_compile.c:2978
2978 /build/buildd/php5-5.3.10/Zend/zend_compile.c: No such file or directory.
(gdb) bt
#0 0x00007fc6d9d4330d in do_bind_function (opline=0x7fc64eac5ab0, function_table=0x7fc6de5a2a90, compile_time=0 '\000') at /build/buildd/php5-5.3.10/Zend/zend_compile.c:2978
#1 0x00007fc6d9d81bbc in ZEND_DECLARE_FUNCTION_SPEC_HANDLER (execute_data=0x7fc6d7601e50) at /build/buildd/php5-5.3.10/Zend/zend_vm_execute.h:586
#2 0x00007fc6d9d8193b in execute (op_array=0x7fc6de8f0800) at /build/buildd/php5-5.3.10/Zend/zend_vm_execute.h:107
#3 0x00007fc6d9d5cea0 in zend_execute_scripts (type=0, retval=0x8de63e558, file_count=3) at /build/buildd/php5-5.3.10/Zend/zend.c:1308
#4 0x00007fc6d9d09513 in php_execute_script (primary_file=0x0) at /build/buildd/php5-5.3.10/main/main.c:2323
#5 0x00007fc6d9dec3ad in php_handler (r=0x7fc6d9dec3ad) at /build/buildd/php5-5.3.10/sapi/apache2handler/sapi_apache2.c:688
#6 0x00007fc6de04d508 in ap_run_handler ()
#7 0x00007fc6de04d97e in ap_invoke_handler ()
#8 0x00007fc6de05cbdc in ap_internal_redirect ()
#9 0x00007fc6d7fe4635 in ?? () from /usr/lib/apache2/modules/mod_rewrite.so
#10 0x00007fc6de04d508 in ap_run_handler ()
#11 0x00007fc6de04d97e in ap_invoke_handler ()
#12 0x00007fc6de05d570 in ap_process_request ()
#13 0x00007fc6de05a398 in ?? ()
#14 0x00007fc6de053fa8 in ap_run_process_connection ()
#15 0x00007fc6de0621d0 in ?? ()
#16 0x00007fc6de06293a in ?? ()
#17 0x00007fc6de0634e7 in ap_mpm_run ()
#18 0x00007fc6de0384a4 in main ()

However, it is not including a different file. Some are still nap.ctp, while others are social.ctp. Each one is when a function is defined:
[0x7fc6d7601e50] ??? /.../social.ctp:58
[0x7fc6d7600dd8] ??? /.../view.php:1159

social.ctp:58
57: if(!function_exists('iconCreate')){
58: function iconCreate($imgName, $iconClass, $icon_set, $icon_set_path){

Also, this appears to happen most at 1:30-1:40AM server time, and the servers impacted are random between WEB13 and WEB27. With some sporadic segfaults during the day.

[Wed Mar 26 01:39:18 2014] [notice] child pid 18776 exit signal Segmentation fault (11)
[Wed Mar 26 01:39:33 2014] [notice] child pid 18761 exit signal Segmentation fault (11)
[Wed Mar 26 01:39:38 2014] [notice] child pid 18781 exit signal Segmentation fault (11)
[Wed Mar 26 01:40:16 2014] [notice] child pid 18759 exit signal Segmentation fault (11)
[Wed Mar 26 01:40:40 2014] [notice] child pid 18767 exit signal Segmentation fault (11)
[Wed Mar 26 01:42:31 2014] [notice] child pid 18773 exit signal Segmentation fault (11)
[Wed Mar 26 01:43:49 2014] [notice] child pid 20663 exit signal Segmentation fault (11)
[Wed Mar 26 01:44:11 2014] [notice] child pid 20505 exit signal Segmentation fault (11)
[Wed Mar 26 01:45:31 2014] [notice] child pid 18779 exit signal Segm...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.