[讨论] JVM堆外内存泄露的问题
 |
charleehu
2013-02-18
[root@TJHY251-103 ~]# java -version
java version "1.6.0_39" Java(TM) SE Runtime Environment (build 1.6.0_39-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode) jvm参数：Xmx4g -Xms4g -XX:PermSize=256m -XX:MaxPermSize=256m -Xss256K 每1秒开500用例持续压测，resin显示活跃线程大约在1000，看到java进程占用内存在以100M/3s的速度增长，持续一段时间后，top下已经占了7.1G了。 13971 root 15 0 8724m 7.1g 13m S 0.0 90.6 38:40.39 java 这是R大的工具看NIO内存 [root@TJHY251-103 ~]# java -cp .:$JAVA_HOME/lib/* DirectMemorySize 13971 Attaching to process ID 13971, please wait... Debugger attached successfully. Server compiler detected. JVM version is 20.14-b01 NIO direct memory: (in bytes) reserved size = 345.664551 MB (362455552 bytes) max size = 3925.375000 MB (4116054016 bytes) 这是挂的gperftools看的内存占用： Total: 3169.2 MB 3147.6 99.3% 99.3% 3147.6 99.3% os::malloc 12.7 0.4% 99.7% 12.7 0.4% ObjectSynchronizer::omAlloc 4.6 0.1% 99.9% 4.6 0.1% zcalloc 3.0 0.1% 100.0% 3.0 0.1% init 0.5 0.0% 100.0% 0.5 0.0% readCEN 0.3 0.0% 100.0% 0.3 0.0% _dl_allocate_tls 0.1 0.0% 100.0% 0.1 0.0% instanceKlass::add_dependent_nmethod 0.1 0.0% 100.0% 0.1 0.0% pthread_cond_wait@GLIBC_2.2.5 0.0 0.0% 100.0% 3.9 0.1% Thread::Thread 0.0 0.0% 100.0% 0.0 0.0% pthread_cond_timedwait@GLIBC_2.2.5 0.0 0.0% 100.0% 0.0 0.0% allocZip 0.0 0.0% 100.0% 0.0 0.0% _dl_new_object 0.0 0.0% 100.0% 0.0 0.0% addMetaName 0.0 0.0% 100.0% 4.6 0.1% Java_java_util_zip_Inflater_init 0.0 0.0% 100.0% 0.0 0.0% pthread_cond_signal@GLIBC_2.2.5 0.0 0.0% 100.0% 0.0 0.0% newEntry 0.0 0.0% 100.0% 0.0 0.0% strdup 0.0 0.0% 100.0% 0.0 0.0% _nl_intern_locale_data 0.0 0.0% 100.0% 0.0 0.0% growMetaNames 0.0 0.0% 100.0% 0.0 0.0% JLI_MemAlloc 0.0 0.0% 100.0% 0.0 0.0% _dl_check_map_versions 0.0 0.0% 100.0% 0.0 0.0% _dl_map_object_deps 0.0 0.0% 100.0% 0.0 0.0% getpwuid 0.0 0.0% 100.0% 0.0 0.0% __fopen_internal 0.0 0.0% 100.0% 0.0 0.0% _dl_scope_free 0.0 0.0% 100.0% 0.0 0.0% dl_open_worker 0.0 0.0% 100.0% 0.0 0.0% initLoopbackRoutes 0.0 0.0% 100.0% 0.0 0.0% os::format_boot_path 0.0 0.0% 100.0% 0.0 0.0% os::strdup 0.0 0.0% 100.0% 0.0 0.0% __tzfile_read 0.0 0.0% 100.0% 0.0 0.0% new_composite_name 0.0 0.0% 100.0% 0.0 0.0% add_to_global 0.0 0.0% 100.0% 0.0 0.0% os::realloc 0.0 0.0% 100.0% 0.0 0.0% _nl_load_locale_from_archive 0.0 0.0% 100.0% 0.0 0.0% expand_dynamic_string_token 0.0 0.0% 100.0% 0.0 0.0% __tzstring 0.0 0.0% 100.0% 24.9 0.8% 0x00002aaaab08d44d 0.0 0.0% 100.0% 10.8 0.3% 0x00002aaaab092859 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09289e 0.0 0.0% 100.0% 8.4 0.3% 0x00002aaaab092928 0.0 0.0% 100.0% 0.1 0.0% 0x00002aaaab09296d 0.0 0.0% 100.0% 5.1 0.2% 0x00002aaaab0929b2 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0929f7 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab092a3c 0.0 0.0% 100.0% 25.7 0.8% 0x00002aaaab092a81 0.0 0.0% 100.0% 0.2 0.0% 0x00002aaaab092d33 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab092dbd 0.0 0.0% 100.0% 0.8 0.0% 0x00002aaaab092e02 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab092e47 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab092e8c 0.0 0.0% 100.0% 19.4 0.6% 0x00002aaaab092f5b 0.0 0.0% 100.0% 0.1 0.0% 0x00002aaaab09d47d 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09d493 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09d7ad 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09d9b9 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09eb2b 0.0 0.0% 100.0% 16.7 0.5% 0x00002aaaab09ecab 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09f287 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09f29d 0.0 0.0% 100.0% 0.1 0.0% 0x00002aaaab09f771 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab09fd4d 0.0 0.0% 100.0% 0.1 0.0% 0x00002aaaab0a0783 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0a0799 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0a0907 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0a091d 0.0 0.0% 100.0% 0.1 0.0% 0x00002aaaab0a987d 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0a9893 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0a9cbd 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0aa1a1 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0aa1b7 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0aa401 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0aa417 0.0 0.0% 100.0% 0.2 0.0% 0x00002aaaab0aa581 0.0 0.0% 100.0% 2.2 0.1% 0x00002aaaab0aa597 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0aa701 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0aa717 0.0 0.0% 100.0% 0.3 0.0% 0x00002aaaab0aafb1 0.0 0.0% 100.0% 0.2 0.0% 0x00002aaaab0aafc7 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab1ba 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab34b 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab361 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab5ab 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab5c1 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab907 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ab91d 0.0 0.0% 100.0% 2.2 0.1% 0x00002aaaab0c5c00 0.0 0.0% 100.0% 0.3 0.0% 0x00002aaaab0ea7bd 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0ead3c 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0eadfa 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab0eaecd 0.0 0.0% 100.0% 0.3 0.0% 0x00002aaaab0edc3a 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab10ac1c 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab145704 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab187c64 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab36819c 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab3741b3 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab387157 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab39666f 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab4b37e7 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab4e847a 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab505ce8 0.0 0.0% 100.0% 10.9 0.3% 0x00002aaaab569ce6 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab574e97 0.0 0.0% 100.0% 3102.7 97.9% 0x00002aaaab601926 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab67fb23 0.0 0.0% 100.0% 0.0 0.0% 0x00002aaaab7d2075 0.0 0.0% 100.0% 0.0 0.0% AbstractCompiler::initialize_runtimes 0.0 0.0% 100.0% 0.0 0.0% AbstractInterpreterGenerator::generate_method_entry 0.0 0.0% 100.0% 0.0 0.0% AdapterFingerPrint::AdapterFingerPrint 0.0 0.0% 100.0% 0.0 0.0% AdapterHandlerLibrary::create_native_wrapper 看的出是0x00002aaaab601926那块占用了……不太明白那块是什么。 UPDATE 换jdk7之后问题解决。 |
 |
ray_linn
2013-03-10
gdb attach 进去看一下呗。
|
|
sanzangc_sdn
2013-03-15
jmap 打一下内存镜像 eclipse MemAnalyzer 分析下 看看是哪里
|
 |
yidianfengfan
2013-03-15
我这里压力测试也遇到了， 无招了。
|
|
jianglei3000
2013-03-29
jmap/mat是看java堆内存，楼主是进程地址空间不足了
这个就只有gdb调试来看看了 |
|
riddle_chen
2013-08-16
想问一下楼主装的是什么版本的gperftools和libunwind，怎么挂gperftools跑resin的
|
|
charleehu
2013-08-21
sanzangc_sdn 写道 jmap 打一下内存镜像 eclipse MemAnalyzer 分析下 看看是哪里
这个已经是堆外的内存溢出了，gc很正常。 |
|
charleehu
2013-08-21
yidianfengfan 写道 我这里压力测试也遇到了， 无招了。
最后没招换了jdk7之后，问题解决。 |
|
charleehu
2013-08-21
riddle_chen 写道 想问一下楼主装的是什么版本的gperftools和libunwind，怎么挂gperftools跑resin的
gperftools-2.0 libunwind-0.99 resin启动脚本加下面2行 export LD_PRELOAD=/opt/perftools/lib/libtcmalloc.so export HEAPPROFILE=/test |
|
riddle_chen
2013-09-17
charleehu 写道 riddle_chen 写道 想问一下楼主装的是什么版本的gperftools和libunwind，怎么挂gperftools跑resin的
gperftools-2.0 libunwind-0.99 resin启动脚本加下面2行 export LD_PRELOAD=/opt/perftools/lib/libtcmalloc.so export HEAPPROFILE=/test 谢谢，之前我是也这么试过的，但是一启动后就卡住没往下走，后来换了libunwind-1.1也不行，我是64位linux环境，resin原来用3.1.8pro，换了3.1.12社区版也不行。 |
