Last week I had the pleasure attended Heinz Kabutz Java Specialists Master course to sharpen my Java skills. Java Champion Heinz, is a great trainer who manages to combine anecdotes, hard facts and deep Java knowledge with engaging exercises to a well done course. The scope was the whole spectrum of Java, but focusing on the details you normally do not use, or know how to use. Some of the material he already published as part of his newsletters, which are read all around the world.

Let me share my impressions on the course with you in this day by day review… read more…

I will be attending the course by Dr. Heinz Kabutz next week from 2nd-5th of March in Düsseldorf.

I have pretty high expectations, as I already read his newsletter since quite some time and also my impressions of Heinz, when he gave his talk at our meet the experts, were very positive. He really knows all the nasty details about Java, and I hope I can get some of those out of his and into my brain. The course promises “Extreme Java”. I am looking forward to very esotheric subtleties without any practical value. And of course to many caveates that you indeed would find in everyday work.

I am also looking forward to meeting other Java geeks to exchange knowledge and have a good week. Will you be there?

(note that the course is in German)

Last week I had to change a webservice-client in such a way as to be able to configure a timeout. The webservice is implemented with spring-webservices and uses the WebServiceTemplate class. read more…

One of the biggest strength of the Java Platform is the implementation of an automatic memory management in the Java Virtual Maschine. Everybody who has programmed with languages like C/C++ knows about the problems of managing memory allocation and deallocation in the code. With Java problems like deallocating memory too early (corrupted pointer) or too late (memory leak) cannot occur by specification. The question is: Why am I writing these blog entries?

The problem is that even with an implicit memory management integrated, Java cannot prevent application of being corrupt in sense of memory management, even it is not allowed to explicitly allocate memory in Java. The result of such wrongly programmed code normally is an exception of type: java.lang.OutOfMemoryError.

This part of the blog series about Java OutOfMemoryError, will introduce the Java Memory Model in detail and shows in which memory areas an java.lang.OutOfMemoryError can occur. Details about the cause of these errors and the tools and methods for analysis will be covered in later entries.

Lets start by looking at the Javadoc of java.lang.OutOfMemoryError:

Thrown when the Java Virtual Machine cannot allocate an object because it is out of memory, and no more memory could be made available by the garbage collector.

This description copied from the actual Java API Documentation (Version 6) is not only very short, but in my point of view incomplete and therefore wrong. This description does only cover the heap of the JVM – as we will learn later, OutOfMemoryError can also occur in different areas of the JVMs memory. These errors are not mentioned in the Javadoc, but you can see them every day in real world applications.

The architecture of Java’s memory management is defined for all JVM implementations in the Java Virtual Maschine Specification. Chapters 3.5 Runtime Data Areas and 3.6 Frames are the most relevant for memory architecture. For a better understanding, I’ve drawn the following picture as a summary of the chapters on memory areas in a JVM.

We can basically distinguish memory areas that are available for all threads in a JVM and those memory areas that are exclusively accessible from only one thread. The two areas that are available from all threads are the Method Area and the Heap.

The method area is responsible for storing class information. The Class-Loader will load the bytecode of a class and will pass it to the JVM. The JVM will generate an internal class representation of the bytecode and store it in the method area. The internal representation of a class will have the following data areas:

• Runtime Constant Pool
  Numeric constants of the class of types int, long, float or double, String-constants and symbolic references to all methods, attributes and types of this class.
• Method Code
  The implementation (code) of all methods of this class including constructors etc.
• Attributes
  A list of all named attributes of this class.
• Fields
  Values of all fields of this class as references to the Runtime Constant Pool.

The method area can be part of the heap and will be created at runtime. The size of the method area can be static or dynamic and it does not have to provide a Garbage Collector.

The second memory area that is available for all threads inside the JVM is the Heap. The Java heap manages instances of classes (objects) and arrays at runtime. The heap will be created at JVM startup and the size can be static or dynamic. The JVM specification mandates a Garbage Collection mechanism for reclaiming the memory of an object on the Java heap. The implementation of the Garbage Collector is not specified, but it is not allowed to provide the programmer with an explicit mechanism for deallocating the memory of an object.

Lets have a look at the Sun HotSpot implementation as an example:

The heap is devided into two generations: The Young Generation and the Tenured Generation. The details of this “generational heap” are not relevant in the context of Java OutOfMemoryError as the design is driven by optimizations of the Garbage Collection algorithm. The method area is implemented as a separated part: The Permanent Generation. All details about configuration and monitoring of these generations will be covered in the third part of this series: “JVM Monitoring  and Configuration”.

This example of the Sun HotSpot JVM memory architecure shows that the JVM specification defines how the memory inside a JVM is organized in general, but leaves enough room for implementation specific optimizations.

In addition to the heap and method area, that are available for all threads of a JVM, every thread also has exclusivly access to memory that is created for each thread:

• PC Register
  The Program Counter register. The register points to the current JVM instruction of the method the thread is executing, if the method is not a native method. If it is a native method the content of the PC register is not defined.
• Java Virtual Maschine Stack
  Each thread gets its own stack on wich so called Frames are pushed for each method the thread currently executed. This means that there can be many frames on the stack for nested method calls – but there is only one frame active at the same time for one thread. The frame contains the local variables of the method, a reference to the Runtime Constant Pool of the method’s class and an operand stack for the execution of JVM operations. (The JVM is a stack maschine!)
• Native Methode Stack
  Native methods get its own stack, the so called „C-Stack“.

Until now you should have get an overview of the Java Memory Model including its different memory areas – this is essential, because now we will take a closer look at our java.lang.OutOfMemoryError. As mentioned before the Javadoc of this exception is not very meaningful, but the Java Virtual Maschine specification defines exactly when and where Java OutOfMemoryError can occur. The difficulty is that theses errors can occur in every memory area I’ve described before. Let’s have a look at the Sun HotSpot JVM and its concrete implementation of OutOfMemoryError errors.

In the heap we get an OutOfMemoryError, if the garbage collector cannot reclaim enough memory for a new object. In such situation the Sun HotSpot JVM shows this error message:

Exception in thread "main": java.lang.OutOfMemoryError: Java heap space

A alternative for this is

Exception in thread "main": java.lang.OutOfMemoryError: Requested array size exceeds VM limit

if the application tries to create an array on the heap that is bigger than the total heap size.

If there is not enough memory in the method area for creating a new class, the Sun HotSpot implementation gets an error in the permanent generation:

Exception in thread "main": java.lang.OutOfMemoryError: PermGen space

Both kinds of OutOfMemoryError occur very often in real life and the reasons for them are very different and will be covered in later blog entries.

OutOfMemory errors in thread exclusive memory areas occur less frequently and are identified by the following error messages in the Sun HotSpot JVM:

Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread

Exception in thread "main": java.lang.OutOfMemoryError: <reason> <stacktrace> (Native method)

The first error is thrown if there are too many threads in the JVM and there is not enough memory left to create a new thread. I’ve seen this because the memory limits of a process have been reached (especially in 32bit operating systems, e.g. on Windows 32bit it is 2GB) or the maximum number of file handles for the user that executes the java process has been reached. The second error message indicates that a memory allocation error on a native stack (JNI method call) has occured.

It is also interesting that a memory allocation error on the JVM stack (too many frames on the stack) does not throw an Java OutOfMemory error but as the JVM specification mandates: java.lang.StackOverflowError.

The last variant of the OutOfMemoryError that I know of is

Exception in thread "main": java.lang.OutOfMemoryError: request <size> bytes for <reason>. Out of swap space?

This error is thrown if there is not enough memory left on the operating system level – which is normally true if other processes are using all of the available memory or the swap space is configured too small.

This first blog entry of the Java OutOfMemoryError series covered the basics of the Java Memory Model. In my point of view it is essential to know the different memory areas of the JVM and its functions if you want to understand why a java.lang.OutOfMemoryError occured in your application. I hope that I have made clear that there can be many variations of this error with totally different possible causes. There are a lot of open questions about when and why theses errors occur and how we can monitor and analyze memory problems in our applications. This is exactly what the next episodes ot this Java OutOfMemoryError series will cover.

One day every professional Java programmer will suffer from this exception thrown by his Java application: java.lang.OutOfMemoryError.

With a little luck this error will occur during development of the application – if not and it happens in a production environment, you will quickly get a lot of unintentional attention by your management, as a Java OutOfMemoryError normally means that you had an application crash! The reasons for these memory problems in the Java Virtual Maschine are versatile and in many cases not easys to analyze. In our troubleshooting services we see workarounds for these errors every day: Servers are preventevly restarted every night as the heap gets closer and closer to its limits or – even worth – the application regulary crashes with an java.lang.OutOfMemoryError indicating that the hunger of the JVM for more memory was bigger than the available heap.

As 64bit JVMs are getting more and more available in production environments, this seems as a perfect solution for theses kind of problems, as more adressable memory means bigger heaps and less frequent crashes. This is why I see that heaps in production are moving to the GB-area without thinking about the consequences: Less performance and troughput, because of more Garbage Collection overhead and pause times. Even worth if the Java OutOfMemoryError had nothing to do with the heap size…

Our performamce study has shown that memory problems are under the most critical problems with regard to stability and performance of Java applications. Almost 50% of the 250 questioned Java specialists answered, that Java Memory Management is critical for the performance and stability of their application. I think that it is time to write a comprehensive guide to the Java Memory Model inclusing tools and methods for anlyzing Java OutOfMemory errors and this is what this blog series is about.

I am calling this series “Java OutOfMemoryError – A tragedy in seven acts”, as I liked the definition of “tragic” in the German Wikipedia entry for “tragedy”, as it says that in the context of a tradegy the word “tragic” is not used in the sense of “sad” but in the sense that somebody gets blameless guilty.

In my opinion this is often true for developers that are responsible for Java OutOfMemoryError – they get blameless guilty. In my troubleshooting sessions I often see that managers are more interested in finding the person who is responsible for the problem than in finding the reason for the problem – this is sad and I hope that I will help the readers of this blog to better understand the reasons for OutOfMemory errors and give them the right knowledge to avoid them or if they have already occured, to quickly find and resolve them.

This is an overview of my Java Memory Tragedy:

1. Act- The Java Memory Model.

The first part of this series will introduce the memory architecture inside the Java Virtual Maschine in detail and will explain where and when java.lang.OutOfMemoryError can occur.

2. Act – Java Memory Leaks et al.

The second part of the series will cover the reasons for java.lang.OutOfMemoryError in detail – especially Java Memory Leaks and high, temporary memory consumption.

3. Act – JVM Monitoring and Configuration

The third part of the series will describe the parameters to configure the JVMs  memory areas and show tools to monitore these parameters at runtime.  As the parameter are different between JVM vendors and implementation, this blog entry will focus on the Sun HotSpot JVM.

4. Act – Creating and understanding Java Heapdumps.

The fourth part of the series will show how to generate Heapdumps with the Sun HotSpot JVM and how to analyze these dumps.

5. Act – Runtime Analyzis of the Java Heap.

The fifth part of the series will cover tools and methods to analyze objects on the Java Heap at runtime, especially to find high memory allocation hotspots and slow growing memory leaks.

6. Act – Typical Java Memory problems

The sixth part of the series will show you typical Java OutOfMemoryError scenarios in practise. I will show you real examples and describe how to analyze and fix these problems by applying the tools and rules of the last 5 parts of this series.

7. Act – Summary and Outlook

The last part of the series will summarize the whole series and will give you an outlook to the next tragedy I am planning to write: The Java Garbage Collector.

I am really happy to write this series in the next weeks, as I am writing a book on performance related topics together with Alois Reitbauer from dynaTrace and I hope to get a lot of feedback to these blog entries, which are partely on the same topic as in the book.

This post shall sum up the results from our fruitful discussion on friday evenig. The idea for the open space discussion was sparked by Stefan Zörner who talked about modularity and what happens when you have no control over modularity. This post will not try to repeat the discussion but merely conserve the results:
read more…