Trifork Blog

Java Profiler Comparison

March 13th, 2012 by
|

While maintaining one of our client’s applications I noticed some performance problems under higher load. It seems logical to think when increasing the load on the application, that the performance will decrease. However, in my case the performance decreased at a rate much higher than when compared to the increased load. I did wonder, maybe the performance could be increased by altering the application’s logic or maybe the application had reached its limit and needed to be scaled up. In order to figure this out in more detail, I decided to do some profiling to see where the application was using most of its resources and how to tackle this problem.

There are many Java profilers out there that could help me to figure out this problem, only the question was which one should I use?

In order to make the right choice, I set up some minimum requirements that the profiler should meet:

  • The layout of application flow should include classes and methods against time

  • Memory monitoring must be included

  • Usability must be featured

My setup is a Spring based application with a RESTful interface using the MongoDB as its data store. It also uses Solr for searching. The application is deployed on Tomcat server.

My options were:

VisualVM

VisualVM is a resource profile tool that comes with JDK starting from 6 update 7. It has default memory and CPU monitoring. It can tell you which classes and methods are hit, but it will not show the flow through the application.

JProfiler

JProfiler was easy to set up and it walked me through a wizard where you can select the application server used to run the application. I had to select the application server home directory and the wizard generated a separate start up script for the application server to use with JPofiler. Then I run the server and it did wait till JProfiler connected to it. I attached the session to the listening process and it gave couple options. I could record memory usage and CPU usage. I went for the CPU usage and could see the execution path. This gave me a nice overview of where the request took the most time. Using the plugin for my IDE, IntelliJ in my case, running JProfiler is even easier. I didn’t have to generate separate Tomcat startup. That was taken care of by the IntelliJ plugin.

YourKit

YourKit is one of the profilers that I came across on another project. The setup was very easy. The setup gave me an option to install a plugin for my IDE. After setup I run the application using the plugin and it automatically attached the process to YourKit. It showed me a nice user interface with tabs like memory monitoring and CPU profiling. I took a snapshot of the CPU time and could see the execution path for my request.

JProbe

I had some difficulties to get started with JProbe. The installation was not straight forward I did figure out the configuration eventually. It uses a similar setup like JProfiler. It generates a start up script in your Tomcat directory. I started the server through the script and attached the session to it. The interface had buttons and tables everywhere and I got lost in the UI. I did see the memory usage, but couldn’t find the execution path of my process.

Spring Insight

Hearing about the Spring Insight monitoring interface as part of the TC Server I was excited to try it out. After installing and setting up a TC Server developer edition, I deployed the application on the TC Server instance. I browsed to the Insight interface where I could monitor my application’s flow. It gave me a nice flow layout of the classes and methods. It showed me how much time the methods took to complete. I could see the values of input parameters as well as the return values. Because my application is Spring based, Spring Insight presented very useful data about my application. A TC Server plugin is available for my IDE for configuring an instance that is similar to Tomcat configuration. The SpringSource Tool Suite comes bundles with Spring Insight.

Conclusion

Having seen these five profiling tools, I can safely say that I am very optimistic about all their possibilities. If you have Spring based application the right choice for development is clearly the Spring Insight. It is free for development purposes, but your application needs to be deployed on the TC Server.

If you want more freedom in monitoring local and remote processes I would go for JProfiler or YourKit. These two can also assist the Spring Insight monitoring in your conquest in finding performance bottlenecks.

Both JProfiler and YourKit meet my requirements described above. JProfiler, YourKit and Spring Insight can trace the flow through the classes and methods of the application. JProfiler and YourKit show memory usage. Although Spring Insight cannot show memory usage, it shows the throughput trend nicely. These three profilers are very clear, not confusing and easy to use. And finally they have nice plugins for my IDE. Hope this little journey helps you make the right choices too.

11 Responses

  1. March 14, 2012 at 22:13 by Machiel Groeneveld

    For me the UI of Yourkit is good and it can connect easily to running or remote java processes where other tools need a custom start-up script.

  2. March 14, 2012 at 22:56 by William Louth

    Amazingly the most advanced one that is 100 to 1000 times faster is not listed.

    http://www.jinspired.com/solutions

  3. March 15, 2012 at 10:59 by ubersoldat

    Not very thorough because you could write a book reviewing this tools. But yes, the conclusion would be the same. I’ve worked with all except for Insight and YourKit is clearly one of the bests… but it’s not free, although it isn’t expensive either and we happily pay for it

  4. March 20, 2012 at 22:13 by William Louth

    I think one should make a distinction with a code profiler and a heap (dump) analyser. Yourkit might be good for memory analysis but its code profiling leaves a lot to be desired. In fact it has the highest event based measurement overhead of all products on the market. Unless of course you are performing call stack sampling and in which case you are not measuring latency but work (well at least trying to do it).

    http://www.jinspired.com/solutions/benchmarks

    But if you have low hanging fruit (stuff that would be caught just browsing the code or dumping a stack) then any profiling will point you in the right direction.

  5. June 1, 2012 at 10:56 by Vladimir Kondratyev

    William, please stop writing bullshit about YourKit profiler. Our overhead is one of the lowest and our feature set is the most advanced. If you cannot compete with us, please stop acting like a bullshit generator. If you have a comparison or test cases, just publish them be ready for discussion.

  6. June 19, 2012 at 10:05 by William Louth

    Vladimir now we both know that I don’t make such things up and we both know that YK does (or has) always had a much greater performance overhead than all other profilers when it comes to event based measurement and not sampling. Seriously if what I tested and measured was not correct then how come in the latest release of YK you claim:

    “Applications run with CPU tracing up to 3 times faster comparing with the previous version.”

    Being a performance engineer means I measure first before releasing and reporting. Maybe you should practice what you preach (and advise customers) by measuring your own performance so that you will not be surprised when some other vendor reports on how bad it is.

    http://www.jinspired.com/wp-content/uploads/2011/11/JXInsightOpenCore_vs_JProfiler_vs_NetBeans_vs_YourKit.pdf

    With regard to “our feature set is the most advanced” I invite you to reproduce the 10 part performance analysis series I did on Scala’s compiler with YK without actually cloning our particular feature set which is something you are not exactly shy of doing.

    http://www.jinspired.com/solutions/case-studies/scala-compiler

  7. September 13, 2012 at 15:36 by Vladimir Kondratyev

    > Being a performance engineer means I measure first before releasing and reporting.

    A-ha-ha 😉 You, guy, always release super-fast code which does not require any future improvement. Right? Just read your own release notice http://www.jinspired.com/site/jxinsight-opencore-6-3-10-released

    Your v6.3.10 is 4 time faster than v6.3.9. You’ve beaten our record!

    Again, stop mentoring and acting like bullshit generator.

  8. October 6, 2012 at 10:08 by William Louth

    The performance of the core metering engine has not changed much in the last year or more…and yet it still out performs every other product including YK.

    The improvement you have called out relates to an optional extension (we have 30 or more of these) which records all metering to a binary log file for playback offline. The throughput and scalability improvements have come by way of work we did in delivering JXInsight/Simz – the first real-time discrete event simulation engine for Java performance measurement and monitoring. This specifically involved a number record format.

    http://www.jinspired.com/products/simz

    That said it is probably time we did look to see how we could further extend our lead in the core parts.

  9. October 27, 2012 at 03:04 by William Louth

    Vladimir I see from the latest EAP build you continue to copy the inventions of others now its “adaptive profiling” with similar settings to ours but still as is typical of your work – limited, narrow, inflexible, sub-par performance and non-adaptive in itself. What amazes me is the little respect you have for your own work in not attempting to mask this blatant plagiarism of pioneering work done by others….it all started with J2EE tracing (which offered in 2000), then probes (delivered 2007-8) and now adaptive xxxx…

    Wondering when will you ever invent something of your own thinking or is that something not customary in your neck of the woods.

  10. December 10, 2012 at 20:11 by Michael

    God you two are pathetic.