SAIO: logs filling up with "exception dumping recon cache"

Bug #1024477 reported by Chris Almond
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Undecided
Florian Hines

Bug Description

I've got a running SAIO system extended to use swauth, https & Swift3. I noticed the log files /var/log/swift/storageX.log and storage.error were growing quit large. Every few seconds the following would appear in the error log:

Jul 13 09:48:36 swift-saio-0 object-replicator Exception dumping recon cache: #012Traceback (most recent call last):#012 File "/home/saio/swift/swift/common/utils.py", line 1196, in dump_recon_cache#012 with lock_file(cache_file, lock_timeout, unlink=False) as cf:#012 File "/usr/lib/python2.6/contextlib.py", line 16, in __enter__#012 return self.gen.next()#012 File "/home/saio/swift/swift/common/utils.py", line 761, in lock_file#012 fd = os.open(filename, flags)#012OSError: [Errno 13] Permission denied: '/var/cache/swift/object.recon'
Jul 13 09:49:41 swift-saio-0 object-replicator last message repeated 2 times

The /var/cache/swift directory did not exist. After creating it and chowning it to my swift process UID/GID ("saio") the log files stopped growing. And, the following files were immediately created inside /var/cache/swift:

-rw------- 1 saio saio 270 2012-07-13 09:49 account.recon
-rw------- 1 saio saio 272 2012-07-13 09:49 container.recon
-rw------- 1 saio saio 52 2012-07-13 09:49 obje ct.recon

Note - I don't have any pipeline or filter settings for recon in any of my account/object/container server conf files. Looks like the SAIO config documentation should at a minimum call out creation of the /var/cache/swift directory to prevent log files from blowing up. (or add recon config to all of the conf files?)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/9798

Changed in swift:
assignee: nobody → Florian Hines (pandemicsyn)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/9798
Committed: http://github.com/openstack/swift/commit/0da1b4f82d105e89729e41e45a22cf5ddc3d719a
Submitter: Jenkins
Branch: master

commit 0da1b4f82d105e89729e41e45a22cf5ddc3d719a
Author: Florian Hines <email address hidden>
Date: Fri Jul 13 17:13:53 2012 -0500

    Create and configure /var/cache/swift in SAIO

    Fixes bug #1024477

    Change-Id: Id09bbff6bb4a5d032727d3ca53b3276b4769d178

Changed in swift:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in swift:
milestone: none → 1.6.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.