When creating a object larger than 5GB with the content-length header set, the server does not error until after the first 5GB of data has been sent

I've been unable to reproduce this issue using curl and have confirmed the code required to immediately return a response has been there for quite some time. After further examination using python-swiftclient and reading the httplib/httplib2 documentation I believe this is actually a limitation with those libraries. The complete request (including body) must be sent before the response can be retrieved. This is because the act of getting the response blocks waiting for the response itself. Under normal circumstances you won't get a response until the end of your upload, so attempting to get the response before sending the data itself causes things to wait indefinitely.

Wayne - can you reproduce your error using curl ?

Exactly, it's a limitation of httplib where it doesn't know how to use "Expect: 100-continue" headers. That's the only way for HTTP requests to fail early. Curl I think does it by default when sending a PUT.

Of course expect headers add an extra RTT to requests, so if latency is important, you should probably just make sure your content is valid as best you can and forego that step.

I've got some time away from compute today. I'll try for a curl recreation.

I presume if you do a PUT with curl and do a "-H Expect:" it'll turn off the expect header and will have to upload the whole file before getting a 413. That's not really something we can fix.

You are correct. Curl creation was a pain for this since it wanted to hold all 5GB in memory, but it does error early on the 5GB boundary using curl.

Marked as Invalid