正擼著代碼汞舱,公司內的聊天工具彈出一條信息:
“狼哥,我這個機器總是頻繁FGC...”
我趕緊打開對話框宗雇,機智的回復一個表情
然后繼續(xù)默默擼碼昂芜。
隨后,小伙伴砸了一段GC日志過來
2019-09-17T20:33:57.889+0800: 4753520.554: [Full GC (Metadata GC Threshold) 4753520.554: [CMS[YG occupancy: 723220 K (1887488 K)]4753520.988: [weak refs process ing, 0.0042134 secs]4753520.992: [class unloading, 0.0987343 secs]4753521.091: [scrub symbol table, 0.0237609 secs]4753521.115: [scrub string table, 0.0025983 s ecs]: 145423K->141376K(3354624K), 0.6260023 secs] 868644K->864597K(5242112K), [Metaspace: 128179K->128179K(1234944K)], 0.6264315 secs] [Times: user=1.24 sys=0.0 0, real=0.63 secs]
4159962 Heap after GC invocations=8029 (full 50):
4159963 par new generation total 1887488K, used 723220K [0x0000000673400000, 0x00000006f3400000, 0x00000006f3400000)
4159964 eden space 1677824K, 42% used [0x0000000673400000, 0x000000069ed59090, 0x00000006d9a80000)
4159965 from space 209664K, 4% used [0x00000006d9a80000, 0x00000006da36c210, 0x00000006e6740000)
4159966 to space 209664K, 0% used [0x00000006e6740000, 0x00000006e6740000, 0x00000006f3400000)
4159967 concurrent mark-sweep generation total 3354624K, used 141376K [0x00000006f3400000, 0x00000007c0000000, 0x00000007c0000000)
4159968 Metaspace used 128145K, capacity 136860K, committed 262144K, reserved 1234944K
4159969 class space used 14443K, capacity 16168K, committed 77312K, reserved 1048576K
4159970 }
4159971 {Heap before GC invocations=8029 (full 50):
4159972 par new generation total 1887488K, used 723220K [0x0000000673400000, 0x00000006f3400000, 0x00000006f3400000)
4159973 eden space 1677824K, 42% used [0x0000000673400000, 0x000000069ed59090, 0x00000006d9a80000)
4159974 from space 209664K, 4% used [0x00000006d9a80000, 0x00000006da36c210, 0x00000006e6740000)
4159975 to space 209664K, 0% used [0x00000006e6740000, 0x00000006e6740000, 0x00000006f3400000)
4159976 concurrent mark-sweep generation total 3354624K, used 141376K [0x00000006f3400000, 0x00000007c0000000, 0x00000007c0000000)
4159977 Metaspace used 128145K, capacity 136860K, committed 262144K, reserved 1234944K
4159978 class space used 14443K, capacity 16168K, committed 77312K, reserved 1048576K
我這慧眼一瞧逾礁,看到了幾個關鍵單詞Full GC
、Metadata GC Threshold
访惜,然后很隨意的回復了
“是不是metaspace沒有設置嘹履,或者設置太小,導致了FGC” 外加一個得意的表情
然后债热,又砸過來一段JVM參數(shù)配置
CATALINA_OPTS="$CATALINA_OPTS -server -Djava.awt.headless=true -Xms5324m -Xmx5324m -Xss512k -XX:PermSize=350m -XX:MaxPermSize=350m -XX:MetaspaceSize=256m -XX:MaxMet aspaceSize=256m -XX:NewSize=2048m -XX:MaxNewSize=2048m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=9 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX :+CMSScavengeBeforeRemark -XX:+ScavengeBeforeFullGC -XX:+UseCMSCompactAtFullCollection -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:CMSInitiat ingOccupancyFraction=80 -XX:+CMSClassUnloadingEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:-ReduceInitialCardMarks -XX:+CMSPermGenSweepingEnabled -XX:CMSInitiatingPerm OccupancyFraction=80 -XX:+ExplicitGCInvokesConcurrent -Djava.nio.channels.spi.SelectorProvider=[sun.nio.ch](http://sun.nio.ch/).EPollSelectorProvider -Djava.util.logging.manager=org.apac he.juli.ClassLoaderLogManager -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:+PrintHeapAtGC -Xloggc:/data/applogs/heap_trace.txt -XX:+IgnoreUnrecognizedVMOptions -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/data/applogs/HeapDumpOnOutOfMemoryError"
“應該不是砾嫉,我們配置了-XX:MaxMetaspaceSize=256m -XX:MetaspaceSize=256m”
看到配置之后,有點懵逼窒篱,好像超出了我的認知范圍焕刮,一下子沒回復,又扔過來一堆數(shù)據(jù)墙杯。
“看cat監(jiān)控數(shù)據(jù)配并,Metaspace使用率在50%的時候就FGC了,GC 日志上的顯示也只用了142M高镐,可是我們明明設置了初始值是256M溉旋,最大值250M,這還沒達到閾值”
機智如我嫉髓,趕緊回復 “等等观腊,我空的時候再看看”
等空閑下來,我又想起了這個問題算行,決定好好研究下梧油。
既然是Metadata GC Threshold引起的FGC,那么只可能是MetadataSpace使用完了州邢,我又反復的看了下GC日志片段儡陨,盯著下面看了會
[0x00000006f3400000, 0x00000007c0000000, 0x00000007c0000000)
4159977 Metaspace used 128145K, capacity 136860K, committed 262144K, reserved 1234944K
發(fā)生FGC之前,Metaspace的committed確實達到了256M,
我們需要知道一點:當目前的committed內存+當前需要分配的內存達到Metaspace閾值迄委,就會發(fā)生Metadata GC Threshold的FGC褐筛。
看到這里,我們也能大概猜出了叙身,這次FGC是合理的渔扎。
但是,為什么used指標只有125M信轿,卻還要申請其它內存晃痴?
看來只有一個原因可以解釋了:內存碎片。
之前只聽過老年代因為CMS的標記清理會產生內存碎片導致FGC财忽,為什么Metaspace也會有這樣的問題倘核?
讓同事對有問題的機器dump了下,用mat打開之后即彪,發(fā)現(xiàn)了一個新大陸紧唱,包含了大量的類加載器。
難道這個碎片問題是大量類加載器引起的隶校?
本地驗證
有了這個疑問漏益,那就簡單了,看看能不能在本地復現(xiàn)深胳。
1绰疤、先定義一個自定義的類加載器,破壞雙親委派
public class MyClassLoader extends ClassLoader {
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
try{
String filePath = "/Users/zhanjun/Desktop/" + name.replace('.', File.separatorChar) + ".class";
//指定讀取磁盤上的某個文件夾下的.class文件:
File file = new File(filePath);
FileInputStream fis = new FileInputStream(file);
byte[] bytes = new byte[fis.available()];
fis.read(bytes);
//調用defineClass方法舞终,將字節(jié)數(shù)組轉換成Class對象
Class<?> clazz = this.defineClass(name, bytes, 0, bytes.length);
fis.close();
return clazz;
}catch (FileNotFoundException e){
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
return super.findClass(name);
}
}
2轻庆、然后在while循環(huán)中,不斷的 load 已經編譯好的class文件
public static void main(String[] args) throws Exception {
while (true) {
Class clazz0 = new MyClassLoader().loadClass("com.sankuai.discover.memory.OOM");
}
}
3敛劝、最后余爆,配置一下jvm啟動參數(shù)
-Xmx2688M -Xms2688M -Xmn960M -XX:MetaspaceSize=50M -XX:MaxMetaspaceSize=100M -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+UseConcMarkSweepGC
啟動之后,不一會兒在控制臺果然出現(xiàn)了日志
{Heap before GC invocations=0 (full 0):
par new generation total 884736K, used 330302K [0x0000000752400000, 0x000000078e400000, 0x000000078e400000)
eden space 786432K, 42% used [0x0000000752400000, 0x000000076668fae0, 0x0000000782400000)
from space 98304K, 0% used [0x0000000782400000, 0x0000000782400000, 0x0000000788400000)
to space 98304K, 0% used [0x0000000788400000, 0x0000000788400000, 0x000000078e400000)
concurrent mark-sweep generation total 1769472K, used 0K [0x000000078e400000, 0x00000007fa400000, 0x00000007fa400000)
Metaspace used 22636K, capacity 102360K, committed 102400K, reserved 1118208K
class space used 8829K, capacity 33008K, committed 33008K, reserved 1048576K
2019-09-21T16:09:28.562-0800: [Full GC (Metadata GC Threshold) 2019-09-21T16:09:28.562-0800: [CMS: 0K->5029K(1769472K), 0.0987115 secs] 330302K->5029K(2654208K), [Metaspace: 22636K->22636K(1118208K)], 0.1340367 secs] [Times: user=0.11 sys=0.03, real=0.13 secs]
Heap after GC invocations=1 (full 1):
par new generation total 884736K, used 0K [0x0000000752400000, 0x000000078e400000, 0x000000078e400000)
eden space 786432K, 0% used [0x0000000752400000, 0x0000000752400000, 0x0000000782400000)
from space 98304K, 0% used [0x0000000782400000, 0x0000000782400000, 0x0000000788400000)
to space 98304K, 0% used [0x0000000788400000, 0x0000000788400000, 0x000000078e400000)
concurrent mark-sweep generation total 1769472K, used 5029K [0x000000078e400000, 0x00000007fa400000, 0x00000007fa400000)
Metaspace used 2885K, capacity 4500K, committed 43008K, reserved 1058816K
class space used 291K, capacity 388K, committed 33008K, reserved 1048576K
}
從日志可以看出來夸盟,發(fā)生FGC之前龙屉,used大概22M,committed已經達到100M满俗,這時再加載class的時候转捕,需要申請內存,就不夠了唆垃,只能通過FGC對Metaspace的內存進行整理壓縮五芝。
到現(xiàn)在,我們已經驗證了過多的類加載器確實可以引起FGC辕万。
內存碎片是怎么產生的枢步?
其實沉删,JVM內部為了實現(xiàn)高效分配,在類加載器第一次加載類的時候醉途,會在Metaspace分配一個獨立的內存塊矾瑰,隨后該類加載加載的類信息都保存在該內存塊。但如果這個類加載器只加載了一個類或者少數(shù)類隘擎,那這塊內存就被浪費了殴穴,如果類加載器又特別多,那內存碎片就產生了货葬。