Possible memory leak related to crypto pkg usage
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
goamz |
New
|
Undecided
|
Unassigned |
Bug Description
I'm debugging a program built with Go 1.2, that does lots of operations in S3 using goamz and MySQL using go-sql-driver.
However, lately it started using too much memory and just growing until it is caught by the OOM killer. I'm not sure when it started, because I made some changes to this program to fix some bugs and after working properly I was cough into this memory issue.
I enabled the pprof http server and got some stats during a test run.
(pprof) top20
Total: 2235.0 MB
1010.0 45.2% 45.2% 1010.0 45.2% crypto/
243.5 10.9% 56.1% 463.5 20.7% crypto/
189.5 8.5% 64.6% 189.5 8.5% bytes.makeSlice
170.5 7.6% 72.2% 390.5 17.5% crypto/
147.0 6.6% 78.8% 147.0 6.6% reflect.
130.0 5.8% 84.6% 130.0 5.8% bufio.NewReaderSize
80.5 3.6% 88.2% 80.5 3.6% crypto/
72.5 3.2% 91.4% 72.5 3.2% crypto/
46.0 2.1% 93.5% 46.0 2.1% math/big.nat.make
27.5 1.2% 94.7% 1283.0 57.4% net/http.
25.0 1.1% 95.8% 25.0 1.1% encoding/
18.5 0.8% 96.7% 18.5 0.8% runtime.convT2E
17.0 0.8% 97.4% 17.0 0.8% crypto/sha1.New
13.0 0.6% 98.0% 30.0 1.3% crypto/hmac.New
4.0 0.2% 98.2% 4.0 0.2% crypto/
4.0 0.2% 98.4% 50.0 2.2% encoding/
3.5 0.2% 98.5% 8.0 0.4% net.socket
3.5 0.2% 98.7% 485.0 21.7% net/http.
3.5 0.2% 98.8% 3.5 0.2% newdefer
2.5 0.1% 98.9% 2.5 0.1% concatstring
I disabled HTTPS in the Region connection I'm using and the looks like the problem went away. I'm not sure if this is a Go issue or a goamz one.
Total: 14.0 MB
10.0 71.4% 71.4% 10.0 71.4% bufio.NewReaderSize
1.5 10.7% 82.1% 1.5 10.7% net/http.
1.0 7.1% 89.3% 1.0 7.1% fmt.Sprintf
0.5 3.6% 92.9% 0.5 3.6% net.sockaddrToTCP
0.5 3.6% 96.4% 0.5 3.6% newdefer
0.5 3.6% 100.0% 0.5 3.6% runtime.allocm
0.0 0.0% 100.0% 0.5 3.6% bitbucket.
0.0 0.0% 100.0% 10.0 71.4% bufio.NewReader