Full GC on java server nodes

[Former Member]

Hi guys,

My portal system is on SAP NW 7.0 EHP2 and backend on ECC 6 EHP6.  Recently, on the portal system I'm getting lot of Full GCs and concurrent mode failure. I have two server nodes and heap size set to 3gb for both (It was 2gb initially). When the Full GCs and concurrent mode failure are occurring, users are not able to work on the portal application and it is timing out. It is only becoming operational again long after a heap dump occurs or I restart the instance manually.  Please suggest how I can resolve this issue.

Additional info: The portal is a Tax Officer Portal used to view, input and amend tax returns. Since the portal is java stack, my ECC system uses this java stack for ADS (z-reports displayed in PDF).

So the issue is that I'm unable to know what is causing the frequent Full GC situation and the OOM heap dump. I tried to analyze using Memory Analyzer but it's not clear enough. Please advise.

Regards.

[Reagan]

Hello

I recommend you to first do an analysis to identify where the memory leak is with the help of MAT

http://www.eclipse.org/mat/

Adding more memory to the heap size will not do help.

You need to first identify whether there is enough memory on the machine.

If the system is configured with memory which is less than the allocated memory for the applications then there is no point in increasing memory parameters.

First check the total memory allocated for the database and the J2EE stack.

You also need to consider the CPU configuration on the system.

Regards

RB

[Former Member]

Hi all,

Thanks for your replies.

1) Below are my JVM parameters (same for both server nodes):

Max Heap size in MB is 3072.

-Xms2048M

-XX:MaxNewSize=600M

-XX:NewSize=600M

-XX:MaxPermSize=512M

-XX:PermSize=512M

-XX:+DisableExplicitGC

-XX:TargetSurvivorRatio=90

-XX:SurvivorRatio=4

-XX:+PrintGCDetails

-XX:+PrintGCTimeStamps

-verbose:gc

-XX:SoftRefLRUPolicyMSPerMB=1

-XX:+HeapDumpOnOutOfMemoryError

-Djco.jarm=1

-Djava.awt.headless=true

-Djava.security.policy=./java.policy

-Djava.security.egd=file:/dev/urandom

-Dsun.io.useCanonCaches=false

-Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy

-Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy

-Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.services.ts.jts.ots.PortableInterceptor.JTSInitializer

-Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy

-Xss2M

-XX:+UseConcMarkSweepGC

-XX:HeapDumpPath=OOM.hprof

-Dcom.sap.jvm.scenario=j2ee

2) I dont have Wily Introscope now. I had tried configuring earlier but didnt proceed. Need to look at it afresh.

3) Like RB is saying, I'm also relunctant to further increase the heap memory.

On the server/architecture side, note the following details:

The Portal resides on a VM (solaris container). It shares its memory with the global zone (main server - 256 GB). On the main server, there are some other VMs also. Memory usage at OS level looks fine. Maybe I need to cap the memory of the portal VM to something like 16GB or 32GB....

4) On the database side, note the below details:

SQL> show parameter sga;

NAME                                 TYPE        VALUE

lock_sga                             boolean     FALSE

pre_page_sga                         boolean     FALSE

sga_max_size                         big integer 4512M

sga_target                           big integer 0

SQL> show parameter pga;

NAME                                 TYPE        VALUE

pga_aggregate_target                 big integer 3068217262

5) I was also thinking at the scenario where my ECC uses the ADS of the Portal server for displaying z-reports....can this be a cause??

Can you please advise based on the above info?

regards.

[Reagan]

Max Heap size in MB is 3072.
-Xms2048M

As far as I know the -Xmx and -Xms should have the same value.

With the information provided by you shows that the Max Heap size (-Xmx) in MB is 3072 and -Xms is 2048M.

Regards

RB

[Former Member]

Hi RB,

Indeed I was also thinking about that. I thought the smallest value is 2048 and the max value of the heap is 3072. In fact, the heap dumps are of size 3GB.

So what do you advise, I set same value (3072) for -Xmx and -Xms?

regards.

[Reagan]

Set them to 3072M and restart the system.

If possible increase the Perm size to 768M

-XX:MaxPermSize=768M

-XX:PermSize=768M

If there is still heavy garbage collection then check what is being executed from the ECC system.

With this configuration the system (Java + DB) will be consuming approx 12 to 13 GB of memory.

Regards

RB

[Former Member]

Ok RB,

I'll try this and update here. Hope it works out fine.

regards.

[Former Member]

Hi,

How did you find that GC is full.

Do you see that in the std_serverX.out. file.

Note 723909 - Java VM settings for J2EE 6.40/7.0

Thanks

Rishi Abrol

[Former Member]

Hi,

As mentioned the meaning of the

the initial and the maximal heap size should be equal: -Xmx1024m -Xms1024m

So you can set that to same but can be different also.

Thanks

Rishi Abrol