Bug 19790644 - RS-7445 [SERV MS LEAKING MEMORY]
Issue is a memory leak in the Java executable.
This bug affects systems running with JDK 7u51 or later versions (1.7.0_55-b13) which are 11.2.3.3.1 and 12.1.1.1.1
This is relevant for all versions 11.2.3.3.1 to 12.1.2.1.1 [Release 11.2 to 12.1] (excluding 11.2.3.3.0 or 12.1.1.1.0)
Systems running 11.2.3.3.0 or 12.1.1.1.0 are not affected as they use 1.7.0_25-b15
Cause:
MS process will be consuming memory (up to 2GB). Normally MS will use around 1GB of memory but because of the bug, the memory allocated can grow upt to 2GB.
Normal memory usage:ps -feal|grep java
0 S root 18585 13652 0 80 0 - 15319 pipe_w 15:21 pts/1 00:00:00 grep java
0 S root 27960 27958 0 80 0 - 292553 futex_ Jun17 ? 01:45:06 /usr/java/default/bin/java -Xms256m -Xmx512m -XX:-UseLargePages -Djava.library.path=/opt/oracle/cell/cellsrv/lib -Ddisable.checkForUpdate=true -jar /opt/oracle/cell/oc4j/ms/j2ee/home/oc4j.jar -out /opt/oracle/cell/cellsrv/deploy/log/ms.lst -err /opt/oracle/cell/cellsrv/deploy/log/ms.err
292553 * 4096 = 1142MB (1GB).
Larger values will indicate memory leak.
When using command pmap -x <MS process pid>, if memory leak is still present it will report a large number of 64 memory chunks:
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 4 4 0 r-x-- java
0000000000600000 4 4 4 rw--- java
00000000019c3000 85212 83816 83816 rw--- [ anon ]
00000000dae00000 46080 45856 45856 rw--- [ anon ]
00000000ddb00000 37888 0 0 ----- [ anon ]
00000000e0000000 175104 174900 174900 rw--- [ anon ]
00000000eab00000 174080 0 0 rw--- [ anon ]
00000000f5500000 87552 87552 87552 rw--- [ anon ]
00000000faa80000 87552 0 0 ----- [ anon ]
00007f261c000000 38384 37488 37488 rw--- [ anon ]
00007f261e57c000 27152 0 0 ----- [ anon ]
00007f2624000000 58488 56628 56628 rw--- [ anon ]
00007f262791e000 7048 0 0 ----- [ anon ]
00007f262c000000 65524 65444 65444 rw--- [ anon ]
00007f262fffd000 12 0 0 ----- [ anon ]
00007f2634000000 65528 65528 65528 rw--- [ anon ]
00007f2637ffe000 8 0 0 ----- [ anon ]
00007f263c000000 65536 65528 65528 rw--- [ anon ]
00007f2644000000 65536 65360 65360 rw--- [ anon ]
00007f264c000000 65528 65520 65520 rw--- [ anon ]
00007f264fffe000 8 0 0 ----- [ anon ]
00007f2654000000 65456 65456 65456 rw--- [ anon ]
0000000000400000 4 4 0 r-x-- java
0000000000600000 4 4 4 rw--- java
00000000019c3000 85212 83816 83816 rw--- [ anon ]
00000000dae00000 46080 45856 45856 rw--- [ anon ]
00000000ddb00000 37888 0 0 ----- [ anon ]
00000000e0000000 175104 174900 174900 rw--- [ anon ]
00000000eab00000 174080 0 0 rw--- [ anon ]
00000000f5500000 87552 87552 87552 rw--- [ anon ]
00000000faa80000 87552 0 0 ----- [ anon ]
00007f261c000000 38384 37488 37488 rw--- [ anon ]
00007f261e57c000 27152 0 0 ----- [ anon ]
00007f2624000000 58488 56628 56628 rw--- [ anon ]
00007f262791e000 7048 0 0 ----- [ anon ]
00007f262c000000 65524 65444 65444 rw--- [ anon ]
00007f262fffd000 12 0 0 ----- [ anon ]
00007f2634000000 65528 65528 65528 rw--- [ anon ]
00007f2637ffe000 8 0 0 ----- [ anon ]
00007f263c000000 65536 65528 65528 rw--- [ anon ]
00007f2644000000 65536 65360 65360 rw--- [ anon ]
00007f264c000000 65528 65520 65520 rw--- [ anon ]
00007f264fffe000 8 0 0 ----- [ anon ]
00007f2654000000 65456 65456 65456 rw--- [ anon ]
Solution:
1. Error is ignorable as MS service will be re-started automatically, which will reset the process and memory used.
2. While patching the storage cell with one-off patches is not generally recommended, if there are issues where the MS service is not automatically re-started, the JDK needs upgraded on the Storage Cell
Use Patch 20328167: TRACKING BUG FOR JDK 1.7.0.72- B33 PATCH (wrapper for 20328167: Oracle JDK 7 Update 72 b33 or later)
3. If no other issues are being seen, the recommended action is to wait for Exadata Cell Software version 12.1.2.1.2 or later.
excellent
ReplyDelete