Comment 22 for bug 1493303

Revision history for this message
Kota Tsuyuzaki (tsuyuzaki-kota) wrote : Re: Swift proxy memory leak on unfinished read

It seems that I can reproduce this with current master (exactly after 12d8a53 ) in 2 environment, one is ubuntu 14.04.02 (3.13.0-48-generic kernel) on AWS and the other is 14.04.01 (3.13.0-36-generic) on my laptop virtual machine.

I didn't look at the reason in detail yet but it looks like that we have an issue for that.

My procedure is:

1. upload a 50MB slo object via swift command (i.e. swift upload ---use-slo)
2. Download the slo object via curl, and then stop the download stream via -m (timeout) option
3. cat /proc/net/sockstat shows as follows:

sockets: used 314
TCP: inuse 20 orphan 0 tw 0 alloc 123 mem 3919 (<- I found the "mem" increased by the curl requests)
UDP: inuse 3 mem 0
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0

4. It looks also increase proxy-server memory usage (confilemed by ps and top command)

It looks that larger number of timeout increased the used memory rapidly. i.e. "-m 0.1" made proxy-server to spend more memory than "-m 0.01" (both of them killed the connections before the task finished)

However, I couldn't find CLOSE_WAIT/FIN_WAIT in netstat. (it might be related that I turned on tcp_recycle in sysctl)

Right now I'm not sure but it might be depends on kernel version because I tested in closer version with reported environment so I'll try to make sure another environment if we can reproduce this anytime.