php5-clamavlib delays php script execution

Hello,

Thanks for reporting this bug and helping to make Ubuntu better. Can you list which PHP modules you have installed? I tried your test script on my machine and didn't experience any delays.

Thanks

ii php-fpdf 1.53.dfsg-5 PHP class to generate PDF files
ii php-pear 5.2.4-2ubuntu5.1 PEAR - PHP Extension and Application Reposit
ii php5 5.2.4-2ubuntu5.1 server-side, HTML-embedded scripting languag
ii php5-adodb 5.04-1 Extension optimising ADOdb database abstract
ii php5-cgi 5.2.4-2ubuntu5.1 server-side, HTML-embedded scripting languag
ii php5-clamavlib 0.13-1ubuntu1 PHP ClamAV Lib - ClamAV Interface for PHP5 S
ii php5-cli 5.2.4-2ubuntu5.1 command-line interpreter for the php5 script
ii php5-common 5.2.4-2ubuntu5.1 Common files for packages built from the php
ii php5-curl 5.2.4-2ubuntu5.1 CURL module for php5
ii php5-gd 5.2.4-2ubuntu5.1 GD module for php5
ii php5-mcrypt 5.2.3-0ubuntu1 MCrypt module for php5
ii php5-memcache 2.2.0-1 memcache extension module for PHP5
ii php5-mysql 5.2.4-2ubuntu5.1 MySQL module for php5
ii php5-pspell 5.2.4-2ubuntu5.1 pspell module for php5
ii php5-sqlite 5.2.4-2ubuntu5.1 SQLite module for php5
ii php5-sqlrelay 1:0.38-3ubuntu1 SQL Relay PHP API
ii php5-suhosin 0.9.22-1 advanced protection module for php5
ii php5-xmlrpc 5.2.4-2ubuntu5.1 XML-RPC module for php5

executing the test script above, hit a 100% CPU usage then drop to normal when execution is finished.

by the way, thank you for your attention on this bug

So I started installing each of the modules you listed which I didn't have, and I think clamavlib is the culprit. What happens, I believe, is that php5-libclamav will load the signature database when PHP is called from the command line. I think the library was written to be used with Apache which should load the db only when Apache starts or the db is updated programatically.

Anyway, can you test this by commenting the following lines in /etc/php5/cli/php.ini:
  #extension=clamav.so
  #[clamav]
  #clamav.dbpath=/var/lib/clamav
  #clamav.maxreclevel=0
  #clamav.maxfiles=0
  #clamav.archivememlim=0
  #clamav.maxfilesize=0
  #clamav.maxratio=0

Then rerun the script from the CLI? When I commented the lines the test script you posted above executed at normal speed.

Thanks

Adam,

 Yes that fixed it!, many thanks but can we still pursue this as a real bug or this case can be closed?
 by the way many thanks for your great time!!!

> Yes that fixed it!, many thanks but can we still pursue this as a real bug
> or this case can be closed?

That's a good question, it may be feasible to only add the php5-clamav
php.ini to the cli only after a dpk-reconfigure or something. I can take a
look at the package and talk to the ClamAV maintainer to see which aproach
is best.

>
> by the way many thanks for your great time!!!

You're quite welcome.

--
Party On,
Adam

Adam,

I'm looking forward on this, i hope the right fix will be committed soon and push on update-manager. more power to ubuntu, this rocks!!!

This is an addition this bug.
My webserver is acting normal in the first 24 hours (after start/restart) then becomes very sluggish.
If I try to open 30-40 urls in 30-40 seconds, they all take 100% cpu and lots of memory, and then hang.
At first they eat 100% CPU, then they eat-up all the memory and swap space, and then change to wait state (hang).

Test case:
I forked around 50 apache2/php5s in 60 seconds, and left them for half an hour.
The application is var_dump() of a large object containing few inherited classes, and few objects;
In normal conditions, this causes load average < 0.5.

here is a Top copy with around 50 active apache2/php5 forks after 10 minutes:
top - 10:46:15 up 1 day, 42 min, 2 users, load average: 60.40, 59.84, 40.41
Tasks: 143 total, 4 running, 139 sleeping, 0 stopped, 0 zombie
Cpu0 : 6.5%us, 14.3%sy, 0.0%ni, 5.1%id, 72.8%wa, 0.1%hi, 1.2%si, 0.0%st
Mem: 1018856k total, 997360k used, 21496k free, 14252k buffers
Swap: 1959912k total, 1959364k used, 548k free, 16936k cached

Didn't catch ps tho to take exact count of forks.
When I restart apache or machine everything works fine for approx 1 day, then starts bugging all over again.
When I disable php5-clamavlib everything works fine and stays fine.
I have also this issue with php-cli and 99% load, and it also works fine when php5-clamavlib is disabled.

I presumed that this has something to do with clamav-freshclam daemon, and tried to repeat this problem by doing $freshclam update manually, but nothing happened.
My biggest concern is that kernel doesn't get ENOMEM error or ignores it or something very unusual happens since dmesg, /var/log/messages and /var/log/apache2/error.log are all clean, and kernel isn't killing any forks. They just deadlock and stay like that forever (30 minutes, and none of them was killed so I presumed they will stay in wait state forever)

hardware:
Sempron64 2800+, 1Gig RAM, 2Gig swap, 2x160Gig RAID1 hdd
distro:
ubuntu server hardy
kernel:
Linux srv1.intranea 2.6.24-19-server #1 SMP Wed Jun 4 17:16:58 UTC 2008 i686 GNU/Linux
apache:
Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.1 with Suhosin-Patch mod_ssl/2.2.8 OpenSSL/0.9.8g
apache modules:
core mod_log_config mod_logio prefork http_core mod_so mod_alias mod_auth_basic mod_authn_file mod_authz_default mod_authz_groupfile mod_authz_host mod_authz_user mod_autoindex mod_cgi mod_dir mod_env mod_mime mod_negotiation mod_php5 mod_rewrite mod_setenvif mod_ssl mod_status mod_userdir
php get_loaded_extensions says:
zip xmlwriter libxml xml wddx tokenizer sysvshm sysvsem sysvmsg session SimpleXML sockets soap SPL shmop standard Reflection posix mime_magic mbstring json iconv hash gettext ftp filter exif dom dba date ctype calendar bz2 bcmath zlib pcre openssl xmlreader apache2handler ADOdb clamav curl ffmpeg gd ldap mcrypt mhash mysql mysqli odbc PDO pdo_mysql PDO_ODBC pdo_pgsql pdo_sqlite pgsql SQLite xmlrpc

Reassign to php-clamavlib where it belongs. BTW last release affected is hardy, since php-clamavlib is no longer in the archive.

Except for maintaining clamav compatibility, no work is being done on this package in Ubuntu.