Golang中的内存泄露及解决方案

写的一个定时删除文件的服务，结果无缘无故被系统杀死了。

于是我设置了参数，每隔30s启动一次。并用 dmesg 命令查看

[1700172.849656] Out of memory: Kill process 1195 (SliceDyson) score 730 or sacrifice child
[1700172.849665] Killed process 1195 (SliceDyson) total-vm:13314968kB, anon-rss:8693980kB, file-rss:116kB

确定是内存泄露造成的。

使用pprof分析造成内存泄露的原因

准备工作

• golang pprof 的文档 里面给了一个例子，如何把 pprof 信息输出到文件里面

• Reddit 上这篇文章 ，提到了如何通过对比两次 heap 的结构，来判断到底哪些内存没有释放：

One thing you can do is to compare 2 heap pprofs. You can do it like this:

    Extract a heap profile heap0.pprof

    Add some load to the application

    Extract another heap profile heap1.pprof

    Compare them with go tool pprof -base heap0.pprof <bin> heap1.pprof

This way you can see exactly what is increasing over time.

• https://blog.golang.org/profiling-go-programs

分析

• 分析生成的 .pprof 文件

$ go tool pprof 120.pprof 
Type: inuse_space
Time: Mar 6, 2019 at 9:27pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 603.20MB, 99.67% of 605.20MB total
Dropped 1 node (cum <= 3.03MB)
      flat  flat%   sum%        cum   cum%
  603.20MB 99.67% 99.67%   605.20MB   100%  main.main
         0     0% 99.67%   605.20MB   100%  runtime.main
(pprof) ^C


$ go tool pprof 240.pprof 
Type: inuse_space
Time: Mar 6, 2019 at 11:42pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 2.81GB, 99.86% of 2.81GB total
Dropped 6 nodes (cum <= 0.01GB)
      flat  flat%   sum%        cum   cum%
    2.81GB 99.86% 99.86%     2.81GB   100%  main.main
         0     0% 99.86%     2.81GB   100%  runtime.main

• 分析各个函数的占比情况

File: SliceDyson
Type: inuse_objects
Time: Mar 7, 2019 at 2:52pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) web
(pprof) top
Showing nodes accounting for 6810, 100% of 6810 total
      flat  flat%   sum%        cum   cum%
      6554 96.24% 96.24%       6554 96.24%  main.GetXMLFiles
       256  3.76%   100%        256  3.76%  vendor/golang_org/x/net/http2/hpack.addDecoderNode
         0     0%   100%        256  3.76%  main.init
         0     0%   100%       6554 96.24%  main.main
         0     0%   100%        256  3.76%  net/http.init
         0     0%   100%       6810   100%  runtime.main
         0     0%   100%        256  3.76%  vendor/golang_org/x/net/http2/hpack.init
         0     0%   100%        256  3.76%  vendor/golang_org/x/net/http2/hpack.init.0

最后定位到造成内存泄露的原因，是 GetXMLFiles 函数造成的。

var xmlFilesPath []string

var LoopGetXmlFilesPath []string
var AllTasks []string
var tasks []*Task
for _, RootPath := range ParseStorageINI(StorageINIPath) {
    xmlFilesPath = GetXMLFiles(RootPath)
    for _, xmlFilesPath_str := range xmlFilesPath {
        LoopGetXmlFilesPath = append(LoopGetXmlFilesPath, xmlFilesPath_str)
    }
}

解决

把 xmlFilesPath 放到 for range 循环里面声明初始化即可