This appears to be some kind of race condition. It is kind of hard to debug this particular one because we don't appear to have all the logs from the proxy server. Not sure why, but we only see the object and container server data for this operation.
The sequence of events is:
1. Container create of: TestContainer-tempest-122946756
a. Auto account-create of: AUTH_3673914f25b046aa815ff03fd0f035fd
2. POST Container TestContainer-tempest-122946756
3. HEAD Container TestContainer-tempest-122946756
4. Object PUT TestContainer-tempest-122946756/Object-tempest-846598070
a. container PUT object
b. PUT object response
5. GET Container, json format
6. DELETE object - 500 Error, .data file not found
Nov 23 09:38:47 devstack-precise-hpcloud-az2-726791 object-server: ERROR __call__ error with DELETE /sdb1/237/AUTH_3673914f25b046aa815ff03fd0f035fd/TestContainer-tempest-122946756/Object-tempest-846598070 :
Traceback (most recent call last):
File "/opt/stack/new/swift/swift/obj/server.py", line 665, in __call__
res = method(req)
File "/opt/stack/new/swift/swift/common/utils.py", line 1919, in wrapped
return func(*a, **kw)
File "/opt/stack/new/swift/swift/common/utils.py", line 687, in _timing_stats
resp = func(ctrl, *args, **kwargs)
File "/opt/stack/new/swift/swift/obj/server.py", line 613, in DELETE
disk_file.delete(req_timestamp)
File "/opt/stack/new/swift/swift/obj/diskfile.py", line 1364, in delete
deleter.put({'X-Timestamp': timestamp})
File "/opt/stack/new/swift/swift/obj/diskfile.py", line 686, in put
self._finalize_put, metadata, target_path)
File "/opt/stack/new/swift/swift/common/utils.py", line 2216, in force_run_in_thread
return self._run_in_eventlet_tpool(func, *args, **kwargs)
File "/opt/stack/new/swift/swift/common/utils.py", line 2199, in _run_in_eventlet_tpool
raise result
OSError: [Errno 2] No such file or directory: '/opt/stack/data/swift/1/sdb1/objects/237/7a0/768dbb513bd1ee5ace0d0f9650d097a0/1385199527.85480.data' (txn: txdb6be9b4235d41cfabb9d-00529077a7)
This appears to be some kind of race condition. It is kind of hard to debug this particular one because we don't appear to have all the logs from the proxy server. Not sure why, but we only see the object and container server data for this operation.
The sequence of events is:
1. Container create of: TestContainer- tempest- 122946756 b046aa815ff03fd 0f035fd tempest- 122946756 tempest- 122946756 tempest- 122946756/ Object- tempest- 846598070
a. Auto account-create of: AUTH_3673914f25
2. POST Container TestContainer-
3. HEAD Container TestContainer-
4. Object PUT TestContainer-
a. container PUT object
b. PUT object response
5. GET Container, json format
6. DELETE object - 500 Error, .data file not found
Nov 23 09:38:47 devstack- precise- hpcloud- az2-726791 object-server: ERROR __call__ error with DELETE /sdb1/237/ AUTH_3673914f25 b046aa815ff03fd 0f035fd/ TestContainer- tempest- 122946756/ Object- tempest- 846598070 : new/swift/ swift/obj/ server. py", line 665, in __call__ new/swift/ swift/common/ utils.py" , line 1919, in wrapped new/swift/ swift/common/ utils.py" , line 687, in _timing_stats new/swift/ swift/obj/ server. py", line 613, in DELETE file.delete( req_timestamp) new/swift/ swift/obj/ diskfile. py", line 1364, in delete put({'X- Timestamp' : timestamp}) new/swift/ swift/obj/ diskfile. py", line 686, in put _finalize_ put, metadata, target_path) new/swift/ swift/common/ utils.py" , line 2216, in force_run_in_thread in_eventlet_ tpool(func, *args, **kwargs) new/swift/ swift/common/ utils.py" , line 2199, in _run_in_ eventlet_ tpool data/swift/ 1/sdb1/ objects/ 237/7a0/ 768dbb513bd1ee5 ace0d0f9650d097 a0/1385199527. 85480.data' (txn: txdb6be9b4235d4 1cfabb9d- 00529077a7)
Traceback (most recent call last):
File "/opt/stack/
res = method(req)
File "/opt/stack/
return func(*a, **kw)
File "/opt/stack/
resp = func(ctrl, *args, **kwargs)
File "/opt/stack/
disk_
File "/opt/stack/
deleter.
File "/opt/stack/
self.
File "/opt/stack/
return self._run_
File "/opt/stack/
raise result
OSError: [Errno 2] No such file or directory: '/opt/stack/