Unix socket continues to grow if the object-auditor process to print a log.

Bug #1089215 reported by Masaharu ARAI
26
This bug affects 6 people
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
High
Hodong Hwang

Bug Description

[Description]
  Unix socket continues to grow if the object-auditor process to print a log.
  Because object-auditor process lacks close().

[Patch]
  I attached patch in this report.

[How to reproduce this issue]
# swift-init all start
Starting container-updater...(/etc/swift/container-server/1.conf)
Starting container-updater...(/etc/swift/container-server/2.conf)
Starting container-updater...(/etc/swift/container-server/3.conf)
Starting container-updater...(/etc/swift/container-server/4.conf)
Starting account-auditor...(/etc/swift/account-server/1.conf)
Starting account-auditor...(/etc/swift/account-server/2.conf)
Starting account-auditor...(/etc/swift/account-server/3.conf)
Starting account-auditor...(/etc/swift/account-server/4.conf)
Starting object-replicator...(/etc/swift/object-server/1.conf)
Starting object-replicator...(/etc/swift/object-server/2.conf)
Starting object-replicator...(/etc/swift/object-server/3.conf)
Starting object-replicator...(/etc/swift/object-server/4.conf)
Starting proxy-server...(/etc/swift/proxy-server.conf)
Starting container-replicator...(/etc/swift/container-server/1.conf)
Starting container-replicator...(/etc/swift/container-server/2.conf)
Starting container-replicator...(/etc/swift/container-server/3.conf)
Starting container-replicator...(/etc/swift/container-server/4.conf)
Starting object-auditor...(/etc/swift/object-server/1.conf)
Starting object-auditor...(/etc/swift/object-server/2.conf)
Starting object-auditor...(/etc/swift/object-server/3.conf)
Starting object-auditor...(/etc/swift/object-server/4.conf)
Unable to locate config for object-expirer
Starting container-auditor...(/etc/swift/container-server/1.conf)
Starting container-auditor...(/etc/swift/container-server/2.conf)
Starting container-auditor...(/etc/swift/container-server/3.conf)
Starting container-auditor...(/etc/swift/container-server/4.conf)
Starting container-server...(/etc/swift/container-server/1.conf)
Starting container-server...(/etc/swift/container-server/2.conf)
Starting container-server...(/etc/swift/container-server/3.conf)
Starting container-server...(/etc/swift/container-server/4.conf)
Starting account-server...(/etc/swift/account-server/1.conf)
Starting account-server...(/etc/swift/account-server/2.conf)
Starting account-server...(/etc/swift/account-server/3.conf)
Starting account-server...(/etc/swift/account-server/4.conf)
Starting account-reaper...(/etc/swift/account-server/1.conf)
Starting account-reaper...(/etc/swift/account-server/2.conf)
Starting account-reaper...(/etc/swift/account-server/3.conf)
Starting account-reaper...(/etc/swift/account-server/4.conf)
Starting container-sync...(/etc/swift/container-server/1.conf)
Starting container-sync...(/etc/swift/container-server/2.conf)
Starting container-sync...(/etc/swift/container-server/3.conf)
Starting container-sync...(/etc/swift/container-server/4.conf)
Starting account-replicator...(/etc/swift/account-server/1.conf)
Starting account-replicator...(/etc/swift/account-server/2.conf)
Starting account-replicator...(/etc/swift/account-server/3.conf)
Starting account-replicator...(/etc/swift/account-server/4.conf)
Starting object-updater...(/etc/swift/object-server/1.conf)
Starting object-updater...(/etc/swift/object-server/2.conf)
Starting object-updater...(/etc/swift/object-server/3.conf)
Starting object-updater...(/etc/swift/object-server/4.conf)
Starting object-server...(/etc/swift/object-server/1.conf)
Starting object-server...(/etc/swift/object-server/2.conf)
Starting object-server...(/etc/swift/object-server/3.conf)
Starting object-server...(/etc/swift/object-server/4.conf)
# ps -efa | grep object-auditor
swift 16663 1 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/1.conf
swift 16664 1 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/2.conf
swift 16665 1 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/3.conf
swift 16666 1 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/4.conf
swift 16882 16663 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/1.conf
swift 16890 16664 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/2.conf
swift 16895 16665 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/3.conf
swift 16896 16666 0 18:21 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server/4.conf
root 16930 16157 0 18:21 pts/0 00:00:00 grep object-auditor
# lsof -p 16663
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
swift-obj 16663 swift cwd DIR 253,0 4096 2 /
swift-obj 16663 swift rtd DIR 253,0 4096 2 /
swift-obj 16663 swift txt REG 253,0 9032 9845 /usr/bin/python
swift-obj 16663 swift mem REG 253,0 1754624 9838 /usr/lib64/libpython2.6.so.1.0
swift-obj 16663 swift mem REG 253,0 156872 388712 /lib64/ld-2.12.so
swift-obj 16663 swift mem REG 253,0 22536 388739 /lib64/libdl-2.12.so
swift-obj 16663 swift mem REG 253,0 1922112 388713 /lib64/libc-2.12.so
swift-obj 16663 swift mem REG 253,0 145720 388728 /lib64/libpthread-2.12.so
swift-obj 16663 swift mem REG 253,0 598800 388746 /lib64/libm-2.12.so
swift-obj 16663 swift mem REG 253,0 90952 388744 /lib64/libz.so.1.2.3
swift-obj 16663 swift mem REG 253,0 124624 389038 /lib64/libselinux.so.1
swift-obj 16663 swift mem REG 253,0 113952 388983 /lib64/libresolv-2.12.so
swift-obj 16663 swift mem REG 253,0 18936 389046 /lib64/libuuid.so.1.3.0
swift-obj 16663 swift mem REG 253,0 17256 389064 /lib64/libcom_err.so.2.1
swift-obj 16663 swift mem REG 253,0 181632 389062 /lib64/libk5crypto.so.3.1
swift-obj 16663 swift mem REG 253,0 915736 389065 /lib64/libkrb5.so.3.3
swift-obj 16663 swift mem REG 253,0 12592 389050 /lib64/libkeyutils.so.1.3
swift-obj 16663 swift mem REG 253,0 1665328 9816 /usr/lib64/libcrypto.so.1.0.0
swift-obj 16663 swift mem REG 253,0 46336 389060 /lib64/libkrb5support.so.0.1
swift-obj 16663 swift mem REG 253,0 272360 389069 /lib64/libgssapi_krb5.so.2.2
swift-obj 16663 swift mem REG 253,0 17520 389116 /lib64/libutil-2.12.so
swift-obj 16663 swift mem REG 253,0 375352 9818 /usr/lib64/libssl.so.1.0.0
swift-obj 16663 swift mem REG 253,0 65928 388663 /lib64/libnss_files-2.12.so
swift-obj 16663 swift mem REG 253,0 9928 29324 /usr/lib64/python2.6/site-packages/netifaces.so
swift-obj 16663 swift mem REG 253,0 7208 44775 /usr/lib64/python2.6/lib-dynload/_weakref.so
swift-obj 16663 swift mem REG 253,0 31720 9237 /usr/lib64/libffi.so.5.0.6
swift-obj 16663 swift mem REG 253,0 124760 9789 /usr/lib64/python2.6/lib-dynload/_ctypes.so
swift-obj 16663 swift mem REG 253,0 39767 42464 /usr/lib64/python2.6/site-packages/xattr/_xattr.so
swift-obj 16663 swift mem REG 253,0 23752 9844 /usr/lib64/python2.6/lib-dynload/zlibmodule.so
swift-obj 16663 swift mem REG 253,0 34096 28279 /usr/lib64/python2.6/site-packages/simplejson/_speedups.so
swift-obj 16663 swift mem REG 253,0 17816 42508 /usr/lib64/python2.6/lib-dynload/_bytesio.so
swift-obj 16663 swift mem REG 253,0 19656 44767 /usr/lib64/python2.6/lib-dynload/_fileio.so
swift-obj 16663 swift mem REG 253,0 42304 44776 /usr/lib64/python2.6/lib-dynload/arraymodule.so
swift-obj 16663 swift mem REG 253,0 14632 9819 /usr/lib64/python2.6/lib-dynload/fcntlmodule.so
swift-obj 16663 swift mem REG 253,0 12904 9803 /usr/lib64/python2.6/lib-dynload/_randommodule.so
swift-obj 16663 swift mem REG 253,0 27816 9826 /usr/lib64/python2.6/lib-dynload/mathmodule.so
swift-obj 16663 swift mem REG 253,0 15368 44768 /usr/lib64/python2.6/lib-dynload/_hashlib.so
swift-obj 16663 swift mem REG 253,0 81256 42526 /usr/lib64/python2.6/lib-dynload/datetime.so
swift-obj 16663 swift mem REG 253,0 76464 44779 /usr/lib64/python2.6/lib-dynload/cPickle.so
swift-obj 16663 swift mem REG 253,0 24432 9835 /usr/lib64/python2.6/lib-dynload/selectmodule.so
swift-obj 16663 swift mem REG 253,0 21232 42522 /usr/lib64/python2.6/lib-dynload/binascii.so
swift-obj 16663 swift mem REG 253,0 58072 11362 /usr/lib64/python2.6/site-packages/OpenSSL/SSL.so
swift-obj 16663 swift mem REG 253,0 74560 11364 /usr/lib64/python2.6/site-packages/OpenSSL/crypto.so
swift-obj 16663 swift mem REG 253,0 12848 11365 /usr/lib64/python2.6/site-packages/OpenSSL/rand.so
swift-obj 16663 swift mem REG 253,0 20112 44780 /usr/lib64/python2.6/lib-dynload/cStringIO.so
swift-obj 16663 swift mem REG 253,0 33216 44773 /usr/lib64/python2.6/lib-dynload/_ssl.so
swift-obj 16663 swift mem REG 253,0 60752 44771 /usr/lib64/python2.6/lib-dynload/_socketmodule.so
swift-obj 16663 swift mem REG 253,0 22240 44769 /usr/lib64/python2.6/lib-dynload/_heapq.so
swift-obj 16663 swift mem REG 253,0 9872 9776 /usr/lib64/python2.6/lib-dynload/_bisectmodule.so
swift-obj 16663 swift mem REG 253,0 54992 42529 /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so
swift-obj 16663 swift mem REG 253,0 21808 26444 /usr/lib64/python2.6/site-packages/greenlet.so
swift-obj 16663 swift mem REG 253,0 37872 44774 /usr/lib64/python2.6/lib-dynload/_struct.so
swift-obj 16663 swift mem REG 253,0 21608 9799 /usr/lib64/python2.6/lib-dynload/_localemodule.so
swift-obj 16663 swift mem REG 253,0 12256 42512 /usr/lib64/python2.6/lib-dynload/_functoolsmodule.so
swift-obj 16663 swift mem REG 253,0 25448 42533 /usr/lib64/python2.6/lib-dynload/stropmodule.so
swift-obj 16663 swift mem REG 253,0 41936 9829 /usr/lib64/python2.6/lib-dynload/operator.so
swift-obj 16663 swift mem REG 253,0 30352 9786 /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so
swift-obj 16663 swift mem REG 253,0 20328 44790 /usr/lib64/python2.6/lib-dynload/timemodule.so
swift-obj 16663 swift mem REG 253,0 99154480 42477 /usr/lib/locale/locale-archive
swift-obj 16663 swift DEL REG 253,0 400126 /tmp/ffiHYH4Dc
swift-obj 16663 swift 0u CHR 1,3 0t0 3643 /dev/null
swift-obj 16663 swift 1u CHR 1,3 0t0 3643 /dev/null
swift-obj 16663 swift 2u CHR 1,3 0t0 3643 /dev/null
swift-obj 16663 swift 3r FIFO 0,8 0t0 833161 pipe
swift-obj 16663 swift 4r FIFO 0,8 0t0 833164 pipe
swift-obj 16663 swift 5r FIFO 0,8 0t0 833167 pipe
swift-obj 16663 swift 6u REG 253,0 4096 399760 /tmp/ffiwYlRow (deleted)
swift-obj 16663 swift 7r FIFO 0,8 0t0 833170 pipe
swift-obj 16663 swift 8r FIFO 0,8 0t0 833173 pipe
swift-obj 16663 swift 9r FIFO 0,8 0t0 833176 pipe
swift-obj 16663 swift 10r FIFO 0,8 0t0 833179 pipe
swift-obj 16663 swift 11r FIFO 0,8 0t0 833182 pipe
swift-obj 16663 swift 12r FIFO 0,8 0t0 833185 pipe
swift-obj 16663 swift 13r FIFO 0,8 0t0 833188 pipe
swift-obj 16663 swift 14r FIFO 0,8 0t0 833191 pipe
swift-obj 16663 swift 15r FIFO 0,8 0t0 833194 pipe
swift-obj 16663 swift 16r FIFO 0,8 0t0 833197 pipe
swift-obj 16663 swift 17r FIFO 0,8 0t0 833200 pipe
swift-obj 16663 swift 18r FIFO 0,8 0t0 833203 pipe
swift-obj 16663 swift 19r FIFO 0,8 0t0 833206 pipe
swift-obj 16663 swift 20r FIFO 0,8 0t0 833209 pipe
swift-obj 16663 swift 21u unix 0xffff880099602980 0t0 833526 socket
swift-obj 16663 swift 22u unix 0xffff880099602680 0t0 833527 socket
swift-obj 16663 swift 23u unix 0xffff880099602380 0t0 833530 socket
swift-obj 16663 swift 25u REG 253,0 4096 400126 /tmp/ffiHYH4Dc (deleted)

---- There are 4 unix sockets when swift startup.

---- after 30 minutes
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
swift-obj 16663 swift cwd DIR 253,0 4096 2 /
swift-obj 16663 swift rtd DIR 253,0 4096 2 /
swift-obj 16663 swift txt REG 253,0 9032 9845 /usr/bin/python
swift-obj 16663 swift mem REG 253,0 1754624 9838 /usr/lib64/libpython2.6.so.1.0
swift-obj 16663 swift mem REG 253,0 156872 388712 /lib64/ld-2.12.so
swift-obj 16663 swift mem REG 253,0 22536 388739 /lib64/libdl-2.12.so
swift-obj 16663 swift mem REG 253,0 1922112 388713 /lib64/libc-2.12.so
swift-obj 16663 swift mem REG 253,0 145720 388728 /lib64/libpthread-2.12.so
swift-obj 16663 swift mem REG 253,0 598800 388746 /lib64/libm-2.12.so
swift-obj 16663 swift mem REG 253,0 90952 388744 /lib64/libz.so.1.2.3
swift-obj 16663 swift mem REG 253,0 124624 389038 /lib64/libselinux.so.1
swift-obj 16663 swift mem REG 253,0 113952 388983 /lib64/libresolv-2.12.so
swift-obj 16663 swift mem REG 253,0 18936 389046 /lib64/libuuid.so.1.3.0
swift-obj 16663 swift mem REG 253,0 17256 389064 /lib64/libcom_err.so.2.1
swift-obj 16663 swift mem REG 253,0 181632 389062 /lib64/libk5crypto.so.3.1
swift-obj 16663 swift mem REG 253,0 915736 389065 /lib64/libkrb5.so.3.3
swift-obj 16663 swift mem REG 253,0 12592 389050 /lib64/libkeyutils.so.1.3
swift-obj 16663 swift mem REG 253,0 1665328 9816 /usr/lib64/libcrypto.so.1.0.0
swift-obj 16663 swift mem REG 253,0 46336 389060 /lib64/libkrb5support.so.0.1
swift-obj 16663 swift mem REG 253,0 272360 389069 /lib64/libgssapi_krb5.so.2.2
swift-obj 16663 swift mem REG 253,0 17520 389116 /lib64/libutil-2.12.so
swift-obj 16663 swift mem REG 253,0 375352 9818 /usr/lib64/libssl.so.1.0.0
swift-obj 16663 swift mem REG 253,0 65928 388663 /lib64/libnss_files-2.12.so
swift-obj 16663 swift mem REG 253,0 9928 29324 /usr/lib64/python2.6/site-packages/netifaces.so
swift-obj 16663 swift mem REG 253,0 7208 44775 /usr/lib64/python2.6/lib-dynload/_weakref.so
swift-obj 16663 swift mem REG 253,0 31720 9237 /usr/lib64/libffi.so.5.0.6
swift-obj 16663 swift mem REG 253,0 124760 9789 /usr/lib64/python2.6/lib-dynload/_ctypes.so
swift-obj 16663 swift mem REG 253,0 39767 42464 /usr/lib64/python2.6/site-packages/xattr/_xattr.so
swift-obj 16663 swift mem REG 253,0 23752 9844 /usr/lib64/python2.6/lib-dynload/zlibmodule.so
swift-obj 16663 swift mem REG 253,0 34096 28279 /usr/lib64/python2.6/site-packages/simplejson/_speedups.so
swift-obj 16663 swift mem REG 253,0 17816 42508 /usr/lib64/python2.6/lib-dynload/_bytesio.so
swift-obj 16663 swift mem REG 253,0 19656 44767 /usr/lib64/python2.6/lib-dynload/_fileio.so
swift-obj 16663 swift mem REG 253,0 42304 44776 /usr/lib64/python2.6/lib-dynload/arraymodule.so
swift-obj 16663 swift mem REG 253,0 14632 9819 /usr/lib64/python2.6/lib-dynload/fcntlmodule.so
swift-obj 16663 swift mem REG 253,0 12904 9803 /usr/lib64/python2.6/lib-dynload/_randommodule.so
swift-obj 16663 swift mem REG 253,0 27816 9826 /usr/lib64/python2.6/lib-dynload/mathmodule.so
swift-obj 16663 swift mem REG 253,0 15368 44768 /usr/lib64/python2.6/lib-dynload/_hashlib.so
swift-obj 16663 swift mem REG 253,0 81256 42526 /usr/lib64/python2.6/lib-dynload/datetime.so
swift-obj 16663 swift mem REG 253,0 76464 44779 /usr/lib64/python2.6/lib-dynload/cPickle.so
swift-obj 16663 swift mem REG 253,0 24432 9835 /usr/lib64/python2.6/lib-dynload/selectmodule.so
swift-obj 16663 swift mem REG 253,0 21232 42522 /usr/lib64/python2.6/lib-dynload/binascii.so
swift-obj 16663 swift mem REG 253,0 58072 11362 /usr/lib64/python2.6/site-packages/OpenSSL/SSL.so
swift-obj 16663 swift mem REG 253,0 74560 11364 /usr/lib64/python2.6/site-packages/OpenSSL/crypto.so
swift-obj 16663 swift mem REG 253,0 12848 11365 /usr/lib64/python2.6/site-packages/OpenSSL/rand.so
swift-obj 16663 swift mem REG 253,0 20112 44780 /usr/lib64/python2.6/lib-dynload/cStringIO.so
swift-obj 16663 swift mem REG 253,0 33216 44773 /usr/lib64/python2.6/lib-dynload/_ssl.so
swift-obj 16663 swift mem REG 253,0 60752 44771 /usr/lib64/python2.6/lib-dynload/_socketmodule.so
swift-obj 16663 swift mem REG 253,0 22240 44769 /usr/lib64/python2.6/lib-dynload/_heapq.so
swift-obj 16663 swift mem REG 253,0 9872 9776 /usr/lib64/python2.6/lib-dynload/_bisectmodule.so
swift-obj 16663 swift mem REG 253,0 54992 42529 /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so
swift-obj 16663 swift mem REG 253,0 21808 26444 /usr/lib64/python2.6/site-packages/greenlet.so
swift-obj 16663 swift mem REG 253,0 37872 44774 /usr/lib64/python2.6/lib-dynload/_struct.so
swift-obj 16663 swift mem REG 253,0 21608 9799 /usr/lib64/python2.6/lib-dynload/_localemodule.so
swift-obj 16663 swift mem REG 253,0 12256 42512 /usr/lib64/python2.6/lib-dynload/_functoolsmodule.so
swift-obj 16663 swift mem REG 253,0 25448 42533 /usr/lib64/python2.6/lib-dynload/stropmodule.so
swift-obj 16663 swift mem REG 253,0 41936 9829 /usr/lib64/python2.6/lib-dynload/operator.so
swift-obj 16663 swift mem REG 253,0 30352 9786 /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so
swift-obj 16663 swift mem REG 253,0 20328 44790 /usr/lib64/python2.6/lib-dynload/timemodule.so
swift-obj 16663 swift mem REG 253,0 99154480 42477 /usr/lib/locale/locale-archive
swift-obj 16663 swift DEL REG 253,0 400126 /tmp/ffiHYH4Dc
swift-obj 16663 swift 0u CHR 1,3 0t0 3643 /dev/null
swift-obj 16663 swift 1u CHR 1,3 0t0 3643 /dev/null
swift-obj 16663 swift 2u CHR 1,3 0t0 3643 /dev/null
swift-obj 16663 swift 3r FIFO 0,8 0t0 833161 pipe
swift-obj 16663 swift 4r FIFO 0,8 0t0 833164 pipe
swift-obj 16663 swift 5r FIFO 0,8 0t0 833167 pipe
swift-obj 16663 swift 6u REG 253,0 4096 399760 /tmp/ffiwYlRow (deleted)
swift-obj 16663 swift 7r FIFO 0,8 0t0 833170 pipe
swift-obj 16663 swift 8r FIFO 0,8 0t0 833173 pipe
swift-obj 16663 swift 9r FIFO 0,8 0t0 833176 pipe
swift-obj 16663 swift 10r FIFO 0,8 0t0 833179 pipe
swift-obj 16663 swift 11r FIFO 0,8 0t0 833182 pipe
swift-obj 16663 swift 12r FIFO 0,8 0t0 833185 pipe
swift-obj 16663 swift 13r FIFO 0,8 0t0 833188 pipe
swift-obj 16663 swift 14r FIFO 0,8 0t0 833191 pipe
swift-obj 16663 swift 15r FIFO 0,8 0t0 833194 pipe
swift-obj 16663 swift 16r FIFO 0,8 0t0 833197 pipe
swift-obj 16663 swift 17r FIFO 0,8 0t0 833200 pipe
swift-obj 16663 swift 18r FIFO 0,8 0t0 833203 pipe
swift-obj 16663 swift 19r FIFO 0,8 0t0 833206 pipe
swift-obj 16663 swift 20r FIFO 0,8 0t0 833209 pipe
swift-obj 16663 swift 21u unix 0xffff880099602980 0t0 833526 socket
swift-obj 16663 swift 22u unix 0xffff880099602680 0t0 833527 socket
swift-obj 16663 swift 23u unix 0xffff880099602380 0t0 833530 socket
swift-obj 16663 swift 24u unix 0xffff8800b237c080 0t0 834320 socket
swift-obj 16663 swift 25u REG 253,0 4096 400126 /tmp/ffiHYH4Dc (deleted)
swift-obj 16663 swift 26u unix 0xffff880107b0d080 0t0 834450 socket
swift-obj 16663 swift 27u unix 0xffff8800a1e03cc0 0t0 834581 socket
swift-obj 16663 swift 28u unix 0xffff880091113380 0t0 834714 socket
swift-obj 16663 swift 29u unix 0xffff8800b23cf980 0t0 834821 socket
swift-obj 16663 swift 30u unix 0xffff8800baec40c0 0t0 834928 socket
swift-obj 16663 swift 31u unix 0xffff8800a9ec0680 0t0 835037 socket
swift-obj 16663 swift 32u unix 0xffff8800b23a6cc0 0t0 835146 socket
swift-obj 16663 swift 33u unix 0xffff8800bad7a380 0t0 835254 socket
swift-obj 16663 swift 34u unix 0xffff880091125980 0t0 835361 socket
swift-obj 16663 swift 35u unix 0xffff8800b23b10c0 0t0 835463 socket
swift-obj 16663 swift 36u unix 0xffff88010d091980 0t0 835581 socket
swift-obj 16663 swift 37u unix 0xffff8801368a60c0 0t0 835689 socket
swift-obj 16663 swift 38u unix 0xffff8801395de680 0t0 835798 socket
swift-obj 16663 swift 39u unix 0xffff8801395a5c80 0t0 835912 socket
swift-obj 16663 swift 40u unix 0xffff88010ba033c0 0t0 836067 socket
swift-obj 16663 swift 41u unix 0xffff8801078d39c0 0t0 836180 socket
swift-obj 16663 swift 42u unix 0xffff880139a4d080 0t0 836318 socket
swift-obj 16663 swift 43u unix 0xffff88009110b3c0 0t0 836432 socket
swift-obj 16663 swift 44u unix 0xffff880091100cc0 0t0 836547 socket
swift-obj 16663 swift 45u unix 0xffff88009110d380 0t0 836654 socket
swift-obj 16663 swift 46u unix 0xffff8800b2381980 0t0 836779 socket
swift-obj 16663 swift 47u unix 0xffff8800911340c0 0t0 836887 socket
swift-obj 16663 swift 48u unix 0xffff8800910833c0 0t0 836996 socket
swift-obj 16663 swift 49u unix 0xffff8800a9e44cc0 0t0 837112 socket
swift-obj 16663 swift 50u unix 0xffff8800b23cb080 0t0 837308 socket
swift-obj 16663 swift 51u unix 0xffff8800b23cb380 0t0 837417 socket
swift-obj 16663 swift 52u unix 0xffff88010d311380 0t0 837526 socket
swift-obj 16663 swift 53u unix 0xffff88013361b980 0t0 837634 socket
swift-obj 16663 swift 54u unix 0xffff8800911c50c0 0t0 837741 socket
swift-obj 16663 swift 55u unix 0xffff8800b23ab380 0t0 837848 socket
swift-obj 16663 swift 56u unix 0xffff8800911e1080 0t0 837956 socket
swift-obj 16663 swift 57u unix 0xffff88009115d3c0 0t0 838064 socket
swift-obj 16663 swift 58u unix 0xffff88009115c9c0 0t0 838174 socket
swift-obj 16663 swift 59u unix 0xffff880091107080 0t0 838283 socket
swift-obj 16663 swift 60u unix 0xffff8800911b93c0 0t0 838394 socket
swift-obj 16663 swift 61u unix 0xffff8801398e69c0 0t0 838507 socket
swift-obj 16663 swift 62u unix 0xffff88010d2af0c0 0t0 838675 socket
swift-obj 16663 swift 63u unix 0xffff8801398ea3c0 0t0 838783 socket
swift-obj 16663 swift 64u unix 0xffff8801078920c0 0t0 838894 socket
swift-obj 16663 swift 65u unix 0xffff880137a0a380 0t0 839001 socket
swift-obj 16663 swift 66u unix 0xffff8800b23b9980 0t0 839108 socket
swift-obj 16663 swift 67u unix 0xffff8801385f00c0 0t0 839216 socket
swift-obj 16663 swift 68u unix 0xffff880138577680 0t0 839325 socket
swift-obj 16663 swift 69u unix 0xffff8800911afc80 0t0 839432 socket
swift-obj 16663 swift 70u unix 0xffff8800911633c0 0t0 839539 socket
swift-obj 16663 swift 71u unix 0xffff8800911c79c0 0t0 839648 socket
swift-obj 16663 swift 72u unix 0xffff8800911aa080 0t0 839766 socket
swift-obj 16663 swift 73u unix 0xffff8801336a66c0 0t0 839874 socket
swift-obj 16663 swift 74u unix 0xffff8801078b1cc0 0t0 839981 socket
swift-obj 16663 swift 75u unix 0xffff880091133380 0t0 840088 socket
swift-obj 16663 swift 76u unix 0xffff8800a9f3f980 0t0 840195 socket
swift-obj 16663 swift 77u unix 0xffff8800911170c0 0t0 840303 socket
swift-obj 16663 swift 78u unix 0xffff88010d315680 0t0 840412 socket
swift-obj 16663 swift 79u unix 0xffff880107a49c80 0t0 840521 socket
swift-obj 16663 swift 80u unix 0xffff880107abc3c0 0t0 840678 socket
swift-obj 16663 swift 81u unix 0xffff8801398589c0 0t0 840787 socket
swift-obj 16663 swift 82u unix 0xffff88010d29e380 0t0 840921 socket
swift-obj 16663 swift 83u unix 0xffff8800b2383680 0t0 841046 socket
swift-obj 16663 swift 84u unix 0xffff88010d326080 0t0 841329 socket
swift-obj 16663 swift 85u unix 0xffff8800b23f30c0 0t0 841509 socket
swift-obj 16663 swift 86u unix 0xffff8800b23ac6c0 0t0 841700 socket
swift-obj 16663 swift 87u unix 0xffff8800b23a3080 0t0 841824 socket

---- There are 65 unix sockets!!

Revision history for this message
Masaharu ARAI (masaharu-arai) wrote :
Revision history for this message
Mitsuru Kosaka (kosaka-mitsuru) wrote :
Download full text (20.9 KiB)

I confirmed that this bug had been fixed with the patch.

[Result]
----------------------------------------------------------------------
[root@rcsop07 ~]# swift-init all start
Starting container-updater...(/etc/swift/container-server/1.conf)
Starting container-updater...(/etc/swift/container-server/2.conf)
Starting container-updater...(/etc/swift/container-server/3.conf)
Starting container-updater...(/etc/swift/container-server/4.conf)
Starting account-auditor...(/etc/swift/account-server/1.conf)
Starting account-auditor...(/etc/swift/account-server/2.conf)
Starting account-auditor...(/etc/swift/account-server/3.conf)
Starting account-auditor...(/etc/swift/account-server/4.conf)
Starting object-replicator...(/etc/swift/object-server/1.conf)
Starting object-replicator...(/etc/swift/object-server/2.conf)
Starting object-replicator...(/etc/swift/object-server/3.conf)
Starting object-replicator...(/etc/swift/object-server/4.conf)
Starting proxy-server...(/etc/swift/proxy-server.conf)
Starting container-replicator...(/etc/swift/container-server/1.conf)
Starting container-replicator...(/etc/swift/container-server/2.conf)
Starting container-replicator...(/etc/swift/container-server/3.conf)
Starting container-replicator...(/etc/swift/container-server/4.conf)
Starting object-auditor...(/etc/swift/object-server/1.conf)
Starting object-auditor...(/etc/swift/object-server/2.conf)
Starting object-auditor...(/etc/swift/object-server/3.conf)
Starting object-auditor...(/etc/swift/object-server/4.conf)
Unable to locate config for object-expirer
Starting container-auditor...(/etc/swift/container-server/1.conf)
Starting container-auditor...(/etc/swift/container-server/2.conf)
Starting container-auditor...(/etc/swift/container-server/3.conf)
Starting container-auditor...(/etc/swift/container-server/4.conf)
Starting container-server...(/etc/swift/container-server/1.conf)
Starting container-server...(/etc/swift/container-server/2.conf)
Starting container-server...(/etc/swift/container-server/3.conf)
Starting container-server...(/etc/swift/container-server/4.conf)
Starting account-server...(/etc/swift/account-server/1.conf)
Starting account-server...(/etc/swift/account-server/2.conf)
Starting account-server...(/etc/swift/account-server/3.conf)
Starting account-server...(/etc/swift/account-server/4.conf)
Starting account-reaper...(/etc/swift/account-server/1.conf)
Starting account-reaper...(/etc/swift/account-server/2.conf)
Starting account-reaper...(/etc/swift/account-server/3.conf)
Starting account-reaper...(/etc/swift/account-server/4.conf)
Starting container-sync...(/etc/swift/container-server/1.conf)
Starting container-sync...(/etc/swift/container-server/2.conf)
Starting container-sync...(/etc/swift/container-server/3.conf)
Starting container-sync...(/etc/swift/container-server/4.conf)
Starting account-replicator...(/etc/swift/account-server/1.conf)
Starting account-replicator...(/etc/swift/account-server/2.conf)
Starting account-replicator...(/etc/swift/account-server/3.conf)
Starting account-replicator...(/etc/swift/account-server/4.conf)
Starting object-updater...(/etc/swift/object-server/1.conf)
Starting object-updater...(/etc/swift/object-server/2.conf)
S...

Revision history for this message
John Dickinson (notmyname) wrote :

Great! Can you please submit the patch through gerrit (http://wiki.openstack.org/HowToContribute) this week (or ASAP) so that it can be included in the next release?

Changed in swift:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/19102

Changed in swift:
status: New → In Progress
Revision history for this message
Hodong Hwang (hodong-hwang) wrote :

Hello.
I have experienced same problem on my cluster.
And fix that the other way.
Here is my patch.
Thanks.

Revision history for this message
Hodong Hwang (hodong-hwang) wrote :
Revision history for this message
John Dickinson (notmyname) wrote :

Hodong Hwang, please either comment on the existing patch in gerrit (link above) or submit your patch through gerrit so it can be reviewed appropriately.

Revision history for this message
Coleman Corrigan (coleman-corrigan) wrote :

I've a 1.7.5 test system that doesn't appear to reproduce this behaviour - can you let me know what's in your object-server.conf ?
In my case I've a default log_facility going to LOG_LOCAL1 and the auditor being redirected to LOG_LOCAL2.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/19598

Changed in swift:
assignee: nobody → Hodong Hwang (hodong-hwang)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/19599

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/19600
Committed: http://github.com/openstack/swift/commit/d46f90e17ad75ae647de002f33cbb362d9f8da78
Submitter: Jenkins
Branch: master

commit d46f90e17ad75ae647de002f33cbb362d9f8da78
Author: Hodong Hwang <email address hidden>
Date: Thu Jan 10 16:46:48 2013 +0900

    Make object-auditor to use one logger

    This commit makes that AuditorWorker gets a logger from ObjectAuditor class
    (instead of creating new one) so the auditor uses minimum unix-sockets.

    Fixes: bug #1089215
    Change-Id: Ia47d862cbe7505db821784b01fcce6f22196e79f

Changed in swift:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in swift:
milestone: none → 1.7.6
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.