jstat
JavaScript Statistical Library
jStat Documentation
We recently upgraded from JDK6u20 (Linux, 32bit and 64bit) to JDK6u23. Since then, we cannot longer use the tools jstack and jstat to get monitoring information from the running process. If we switch back to JDK6u20, everything works fine.
We are running Tomcat 6. According to this forum post, others have the same problem:
http://forums.oracle.com/forums/thread.jspa?threadID=2151967&tstart=0
Running simple plain Java processes and using the tools works.
Jstack says: Unable to open socket file: target process not responding or HotSpot VM not loaded The -F option can be used when the target process is not responding.
Jstat says: 19799 not found
Using Jps does not show the running processes at all, so I guess the problem is more of general nature with JDK6u23 and also JDK6u24. It has a new Hotspot engine. Maybe something does not work in conjunction with Tomcat and that Hotspot v19.
Any idea? Help is appreciated.
P.S. Of course, we run that as the same user and we have not changed anything else. Only the JDK.
Source: (StackOverflow)
I'm trying to analyze GC behaviour for our application (running in Tomcat, under Sun's Hotspots, JVM 1.6).
So far I've Instructed the JVM to emit GC logs to a separate file using...
-Xloggc:gc.log
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintGCApplicationConcurrentTime
-XX:+PrintGC
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
...and used jstat to output logs using...
jstat -gc -t 29045 5s > jstat.gc
I am seeing interesting information, but haven't found a tool to help me analyze/visualize these logs. I was pointed to GCViewer by this question, but it only parses a few log lines from gc.log and then crashes with an exception. Is there a better or more up-to-date tool for parsing these specific logs, for the specific JVM I'm using?
Source: (StackOverflow)
I have a question about how to use the distribution functions within the jstat library. Specifically, I am focused on studentt.
I've tried this:
var alphaLevel = 0.05;
var degreesOfFreedom = 18;
// the answer I want to get is 2.100922
tStat = jStat.studentt(alphaLevel,degreesOfFreedom);
// but all that is returned is an object with
// members _a,_b,_c (_a=alphaLevel, _b=degreesOfFreedom,_c=undefined).
As explained on the jstat github site, there is a difference between static and instance functions. However, it is above my experience with javascript as to how to do this.
Can anyone explain how to properly call the studentt function and get the proper result?
Thank you!
Source: (StackOverflow)
As per link : -gccapacity shows :
NGCMX : Maximum new generation capacity (KB).
NGC : Current new generation capacity (KB).
Is NGC shows currently free new generation capacity or currently used new generation capacity.
Source: (StackOverflow)
I have installed and configured jstat in my PC with Windows XP, tested it with sample Java program. But I need to know how to use jstat with JSP or Tomcat, is there any other futher configuration needed or not?
Source: (StackOverflow)
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
245760.0 245760.0 33804.8 0.0 1966080.0 364994.1 8028160.0 138003.6 25984.0 25429.9 2944.0 2877.2 8 0.451 2 0.153 0.603
This is the output of ./jstat -gc pid 250ms 0
I want to know what is the unit YGCT or FGCT? i.e. us or ms?
Source: (StackOverflow)
Running tomcat 7.0.32 on oracle hotspot jdk 1.7.0_7 I get this peculiar output.
I was hoping to pinpoint a memory leak, so I ran jstat -gc -t 2000. At first everything looks reasonable. I get a new line every 2-ish seconds. At some point though, the timestamp column starts getting weird. It returns after 1 second, then after 5 seconds, then 2, then 7, until it reaches timestamp 518058.9 when it finally says "Screwit. You got enough data son. Go work on it. Imma call it a day."
Werid? Why does this happen? I was forced to restart the instance.
There could have been database errors and c3p0 was crying DEADLOCK, however this shouldn't stop the jvm reporting gc utilization? Right? Actually - not stop reporting it, just stop increasing the timestamp.
jstat output:
Timestamp S0C S1C S0U S1U EC EU OC OU PC PU YGC YGCT FGC FGCT GCT
259712.9 11584.0 15424.0 9560.8 0.0 285056.0 160309.1 470016.0 163917.6 142912.0 123577.1 1494 123.712 17 31.547 155.259
259715.1 11584.0 15424.0 9560.8 0.0 285056.0 160319.3 470016.0 163917.6 142912.0 123577.1 1494 123.712 17 31.547 155.259
259717.1 11584.0 15424.0 9560.8 0.0 285056.0 160334.9 470016.0 163917.6 142912.0 123577.1 1494 123.712 17 31.547 155.259
....
518016.3 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518018.0 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518023.7 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518025.8 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518026.7 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518029.9 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518031.1 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518034.2 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518036.4 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518037.3 21696.0 22080.0 0.0 15712.7 592064.0 433482.6 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518040.8 21696.0 22080.0 0.0 15712.7 592064.0 433502.5 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518043.1 21696.0 22080.0 0.0 15712.7 592064.0 433536.3 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518045.7 21696.0 22080.0 0.0 15712.7 592064.0 433631.9 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518050.1 21696.0 22080.0 0.0 15712.7 592064.0 433685.9 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518054.4 21696.0 22080.0 0.0 15712.7 592064.0 433766.8 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518056.1 21696.0 22080.0 0.0 15712.7 592064.0 433844.9 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518057.6 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
518058.9 21696.0 22080.0 0.0 15712.7 592064.0 433920.0 527296.0 495141.6 140224.0 126562.4 1811 185.571 18 50.010 235.581
....
Source: (StackOverflow)
I'm using jstat to collect jvm metrics and push them to graphite through collectd.
A few days ago i noticed that i had "spikes" in the graphs regarding Old Space, but those were not triggering any full GCs, which lead me to thinking that the metric was off.
So i started looking at jstat's output, searching for such a metric to pop and here it is :
jstat -gccause 123456 1s
S0 S1 E O M CCS YGC YGCT FGC FGCT GCT LGCC GCC
27,81 0,00 49,52 63,18 65,05 48,04 7146 703,497 20 10,564 714,061 Allocation Failure No GC
27,81 0,00 63,65 63,18 65,05 48,04 7146 703,497 20 10,564 714,061 Allocation Failure No GC
27,81 16,89 100,00 99,93 65,05 48,04 7147 703,497 20 10,564 714,061 Allocation Failure Allocation Failure
0,00 23,40 10,34 63,19 65,05 48,04 7147 703,621 20 10,564 714,185 Allocation Failure No GC
0,00 23,40 35,39 63,19 65,05 48,04 7147 703,621 20 10,564 714,185 Allocation Failure No GC
As you can see, my Old space is increasing normally and just "peaks" to 99.93%, not triggering any full GC and returns to a normal metric right after that.
So i'm wondering if something's off or if it's just an incorrect iteration.
Thanks !
Source: (StackOverflow)
I am investigating performance problems on an EE application which are probably a result of a less than optimally tuned garbage collector.
While looking into jstat logs I gathered under heavy load I stumbled upon the following finding:
S0 S1 E O P
7.39 0.00 41.81 88.89 53.93
NGCMN NGCMX NGC S0C S1C
131072.0 301056.0 301056.0 30080.0 30080.0
EC OGCMN OGCMX OGC
240896.0 1441792.0 2058240.0 2058240.0
As per the official jstat documentation here we can see that the current old generation size (OGC) has reached the maximum old generation size (OGCMX). The first line however tells us that "Old space utilization as a percentage of the space's current capacity" (O) is "only" at 88%.
My questions are:
- What mechanic is setting the old generation limit seeen as OGCMX in the jstat output?
- What is keeping old space from growing and using up the remaning ~12% of heap?
We have JVM parameters which are setting the min/max for total heap, new space and perm space (-Xms -Xmx -XX:NewSize -XX:MaxNewSize -XX:PermSize -XX:MaxPermSize).
But there are no limits set on old space (I don't know if such parameters even exist).
Edit concerning NewRatio:
It is my understadning and also mentioned here that the JVM does not enforce the ratio between new and old space, especially not if the MaxNewSize param is used. And the MaxNewSize param is accepted, as the value above (301056.0KB) is in fact exactly the 294MB we are specifying for MaxNewSize. Furthermore, the default value for NewRatio is 2 (see here), which is not at all the ratio between any of the above values listed by jstat.
Source: (StackOverflow)
Our Java-based server application is reporting an unusually high class loader time:
# jstat -class 10625 1000
Loaded Bytes Unloaded Bytes Time
4781 9165.6 114 185.2 17769.35
4781 9165.6 114 185.2 17769.85
4781 9165.6 114 185.2 17770.36
4781 9165.6 114 185.2 17771.11
4781 9165.6 114 185.2 17771.73
This is on a server that had been running ~8 hours. jstat reports it has spent 17,769s (~4h56m!) doing class loading, and about 0.5-0.6 extra seconds every second! We've been tracking a performance issue and this is our best candidate for a culprit. Just to be sure, we checked our other Java services: jstat showed a very low value in the Time column (a few seconds, even after having run for hours)
Our code doesn't do constant class loading, but we can't rule out a misbehaving 3rd party library. We enabled -verbose:gc hoping to diagnose the problem. But once our server loads all of its classes (within a minute or so of heavy traffic) the verbose class log became quiet -- we were half-expecting to see a flurry of activity given the jstat data.
My questions would be:
- Is this really indicative of a problem?
- If so, what else could be done to diagnose it?
Would really appreciate any suggestions.
Source: (StackOverflow)
I'm trying to profile remote java app, actually it is a gameserver. It works normally on my local machine (windows XP x64 with JDK1.7.0_02 x64), but behaves very wierd on the production server (CentOS with JDK1.7.0_03 i586).
I've done a lot of searching and found out that I should use VisualVM for this task. So VisualVM works great on local machine, but there is no hangs on local machine, i need profiling in production environment with real payload. I started jstatd on the remote machine with arguments
jstatd -J-Djava.security.policy=jstatd.all.policy -J-Djava.rmi.server.logCalls=false &
with the policy file
grant codebase "file:/usr/java/jdk1.7.0_02/lib/tools.jar" {
permission java.security.AllPermission;
};
then I started my java application like this
java -server -Dcom.sun.management.jmxremote\
-Dcom.sun.management.jmxremote.port=4000\
-Dcom.sun.management.jmxremote.ssl=false\
-Dcom.sun.management.jmxremote.authenticate=false\
-jar /home/pinballSocketServer/pinballSocketServer.jar
Both application and jstatd are launched with root priveledges.
and VisualVM didn't manage to connect to remote host. But on the remote host i see the following log, while VisualVM is running and remote host added:
Feb 16, 2012 7:11:52 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(3)-217.16.27.195: [217.16.27.195: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]
Feb 16, 2012 7:11:56 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(3)-217.16.27.195: [217.16.27.195: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]
Feb 16, 2012 7:12:00 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(3)-217.16.27.195: [217.16.27.195: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]
Feb 16, 2012 7:12:04 PM sun.rmi.server.UnicastServerRef logCall
FINER: RMI TCP Connection(3)-217.16.27.195: [217.16.27.195: sun.rmi.registry.RegistryImpl[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)]
After further googling, I found out that I need to use ssh tunneling. I configured putty in the following way
http://www.advancedigital.ru/show/putty_config.jpg
and VisualVM as this
http://www.advancedigital.ru/show/visualvm_config.jpg
Adter munipulations above VisualVM connects to remote host, but I can only see the threads summary chart and profiler is inactive.
I've seen some recommendations that jvms on both machines should be similar and have the same platform (x86 or x64) but i've already tried profiling from another machine (windows 7 x86 with JDK1.7.0_03 x86), and have the same result.
I've also tried this, but get the same result again.
VisualVM over ssh
How can I get this profiling to work?
Source: (StackOverflow)
I have a webapp running on tomcat and I want to look at garbage collection during runtime using jstat.
I have the command:
jstat.exe -gc <vmid> 1000
How can I find what the vmid is?
Source: (StackOverflow)
I want to find the non-heap memory code cache details of a remote JVM (JDK 8) running on windows from my windows machine. I can get the details using jconsole. But, what i need is to get the values using jstat command. GUI monitoring tools doesn't serves my purpose. Thanks
Source: (StackOverflow)
I am trying to run jstat for monitoring GC. I get the vmid using jps command:
> jps
18928 GCTest
but when i try to run jstat like this:
>jstat -gc <vmid> 18928
The system cannot find the file specified.
I get the message The system cannot find the file specified. . The syntax seems fine. Am i making any errors?
Source: (StackOverflow)