Request using Chunked encoding as a header in object GET raises httlib Error

Bug #1483997 reported by janonymous
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Confirmed
Low
janonymous

Bug Description

curl -i http://127.0.0.1:8080/v1/AUTH_Jai/Form/abc -X GET -H "Transfer-Encoding: chunked" raises error in httplib:

if self.chunked:
        return self._read_chunked(amt)

in line :
def _read_chunked(self.amt):
    ....
    ....
    chunk_left = int(line, 16)

Revision history for this message
janonymous (janonymous) wrote :

8 2015-08-12T12:32:30.792721+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: self.handle_one_response() (txn: tx3c9937c0b4b0490bad5ed -0055caef86) (client_ip: 127.0.0.1)
909189 2015-08-12T12:32:30.792786+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: File "/usr/local/lib/python2.7/dist-packages/eventlet/ws gi.py", line 507, in handle_one_response (txn: tx3c9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909190 2015-08-12T12:32:30.792857+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: while self.environ['eventlet.input'].read(MINIMUM_CHUNK_ SIZE): (txn: tx3c9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909191 2015-08-12T12:32:30.792922+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: File "/usr/local/lib/python2.7/dist-packages/eventlet/ws gi.py", line 189, in read (txn: tx3c9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909192 2015-08-12T12:32:30.793002+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: return self._chunked_read(self.rfile, length) (txn: tx3c 9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909193 2015-08-12T12:32:30.793170+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: File "/usr/local/lib/python2.7/dist-packages/eventlet/ws gi.py", line 179, in _chunked_read (txn: tx3c9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909194 2015-08-12T12:32:30.793204+05:30 alok-HP-Compaq-8200-Elite-MT-PC object-server: STDERR: File "/usr/local/lib/python2.7/dist-packages/eventlet/w sgi.py", line 189, in read (txn: tx3c9937c0b4b0490bad5ed-0055caef86)
909195 2015-08-12T12:32:30.793285+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: self.chunk_length = int(rfile.readline().split(b";", 1)[ 0], 16) (txn: tx3c9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909196 2015-08-12T12:32:30.793351+05:30 alok-HP-Compaq-8200-Elite-MT-PC object-server: STDERR: return self._chunked_read(self.rfile, length) (txn: tx3 c9937c0b4b0490bad5ed-0055caef86)
909197 2015-08-12T12:32:30.793373+05:30 alok-HP-Compaq-8200-Elite-MT-PC proxy-server: STDERR: ValueError: invalid literal for int() with base 16: '' ( txn: tx3c9937c0b4b0490bad5ed-0055caef86) (client_ip: 127.0.0.1)
909198 2015-08-12T12:32:30.793419+05:30 alok-HP-Compaq-8200-Elite-MT-PC object-server: STDERR: File "/usr/local/lib/python2.7/dist-packages/eventlet/w sgi.py", line 179, in _chunked_read (txn: tx3c9937c0b4b0490bad5ed-0055caef86)
909199 2015-08-12T12:32:30.793492+05:30 alok-HP-Compaq-8200-Elite-MT-PC object-server: STDERR: self.chunk_length = int(rfile.readline().split(b";", 1) [0], 16) (txn: tx3c9937c0b4b0490bad5ed-0055caef86)
909200 2015-08-12T12:32:30.793563+05:30 alok-HP-Compaq-8200-Elite-MT-PC object-server: STDERR: ValueError: invalid literal for int() with base 16: '' (txn: tx3c9937c0b4b0490bad5ed-0055caef86)

Changed in swift:
assignee: nobody → janonymous (janonymous)
Revision history for this message
Samuel Merritt (torgomatic) wrote :

Yes, I can reproduce this, and yes, I get a pair of stack traces in my logs when I do. Also of note: the client receives the normal GET response; they can't tell that anything broke.

However, those stack traces don't contain any references to Swift code. Everything is in eventlet.wsgi or Python stdlib; there's not a single frame that refers to Swift code, so it seems highly unlikely that Swift can do anything about it.

The only improvement I can see right now is that Swift shouldn't pass Transfer-Encoding to the object server on object GET since it's not going to send a body on the GET request. That'd take it from two stack traces to one.

Also, if I actually *send* a GET body, then the proxy doesn't log any stack trace. Only the object server does.

Changed in swift:
importance: Undecided → Low
status: New → Confirmed
Revision history for this message
sajuptpm (sajuptpm) wrote :
Revision history for this message
janonymous (janonymous) wrote :

It's reason was Chunk requests handling. I will upload a patch soon for it's handling.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.