Overview

Useful JVM Flags – Part 5 (Young Generation Garbage Collection)

7 Comments

In this part of our series we focus on one of the major areas of the heap, the “young generation”. First of all, we discuss why an adequate configuration of the young generation is so important for the performance of our applications. Then we move on to learn about the relevant JVM flags.

From a purely functional perspective, a JVM does not need a young generation at all – it can do with a single heap area. The sole reason for having a young generation in the first place is to optimize the performance of garbage collection (GC). More specifically, the separation of the heap into a young generation and an old generation has two benefits: It simplifies the allocation of new objects (because allocation only affects the young generation) and it allows for a more efficient cleanup of objects not needed anymore (by using different GC algorithms in the two generations).

Extensive measurements across a wide range of object-oriented programs have shown that many applications share a common characteristic: Most objects “die” young, i.e., after their creation they are not referenced for long in the program flow. Also, it has been observed that young objects are rarely referenced by older objects. Now if we combine these two obsrvations, it becomes apparent that it is desirable for GC to have quick access to young objects – for example in a separate heap area called the “young generation”. Within this heap area, GC can then identify and collect “dead” young objects quickly without having to search them between all the old objects that will still live on the heap for a long time.

The Sun/Oracle HotSpot JVM further divides the young generation into three sub-areas: one large area named “Eden” and two smaller “survivor spaces” named “From” and “To”. As a rule, new objects are allocated in “Eden” (with the exception that if a new object is too large to fit into “Eden” space, it will be directly allocated in the old generation). During a GC, the live objects in “Eden” first move into the survivor spaces and stay there until they have reached a certain age (in terms of numbers of GCs passed since their creation), and only then they are transferred to the old generation. Thus, the role of the survivor spaces is to keep young objects in the young generation for a little longer than just their first GC, in order to be able to still collect them quickly should they die soon afterwards.

Based on the assumption that most of the young objects may be deleted during a GC, a copying strategy (“copy collection”) is being used for young generation GC. At the beginning of a GC, the survivor space “To” is empty and objects can only exist in “Eden” or “From”. Then, during the GC, all objects in “Eden” that are still being referenced are moved into “To”. Regarding “From”, the still referenced objects in this space are handled depending on their age. If they have not reached a certain age (“„tenuring threshold“”), they are also moved into “To”. Otherwise they are moved into the old generation. At the end of this copying procedure, “Eden” and “From” can be considered empty (because they only contain dead objects), and all live objects in the young generation are located in “To”. Should “To” fill up at some point during the GC, all remaining objects are moved into the old generation instead (and will never return). As a final step, “From” and “To” swap their roles (or, more precisely, their names) so that “To” is empty again for the next GC and “From” contains all remaining young objects.

Example: The young generation right before and after a garbage collection

Example showing the initial state and the result of a young generation GC. Free space is green, objects not referenced anymore are yellow, and still referenced objects are red. In this example, the survivor spaces are large enough so that no objects need to be moved into the old generation.

As a summary, an object is usually born in “Eden” and then alternates between the survivor spaces on each young generation GC. If the objects survives until a certain number of young generation GCs has passed, it will finally be moved into the old generation and stay there with all other long-lived objects. When the object eventually dies in the old generation, it has to be collected with higher effort, by one of the more heavyweight GC algorithms (a plain copy collection cannot be used here – there simply is no place to copy to).

It now becomes clear why young generation sizing is so important: If the young generation is too small, short-lived objects will quickly be moved into the old generation where they are harder to collect. Conversely, if the young generation is too large, we will have lots of unnecessary copying for long-lived objects that will later be moved to the old generation anyway. Thus we need to find a compromise somewhere between small and large young generation size. Unfortunately, finding the right compromise for a particular application can often only be done by systematic measurement and tuning. And that’s where the JVM flags come into play.

-XX:NewSize and -XX:MaxNewSize

Similar to the total heap size (with -Xms and -Xmx) it is possible to explicitly set a lower and upper bound for the size of the young generation. However, when setting -XX:MaxNewSize we need to take into account that the young generation is only one part of the heap and that the larger we choose its size the smaller the old generation will be. For stability reasons it is not allowed to choose a young generation size larger than the old generation, because in the worst case it may become necessary for a GC to move all objects from the young generation into the old generation. Thus -Xmx/2 is an upper bound for -XX:MaxNewSize.

For performance reasons we may also specify the initial size of the young generation using the flag -XX:NewSize. This is useful if we know the rate at which young objects are being allocated (for example because we measured it!) and can save some of the costs required for slowly growing the young generation to that size over time.

-XX:NewRatio

It is also possible to specify the young generation size in relation to the size of the old generation. The potential advantage of this approach is that the young generation will grow and shrink automatically when the JVM dynamically adjusts the total heap size at run time. The flag -XX:NewRatio allows us to specify the factor by which the old generation should be larger than the young generation. For example, with -XX:NewRatio=3 the old generation will be three times as large as the young generation. That is, the old generation will occupy 3/4 and the young generation will occupy 1/4 of the heap.

If we mix absolute and relative sizing of the young generation, the absolute values always have precedence. Consider the following example:

$ java -XX:NewSize=32m -XX:MaxNewSize=512m -XX:NewRatio=3 MyApp

With these settings, the JVM will try to size the young generation at one third of the old generation size, but it will never let young generation size fall below 32 MB or exceed 512 MB.

There is no general rule if absolute or relative young generation sizing is preferable. If we know the memory usage of our application well, it can be advantageous to specify a fixed size both for the total heap and the young generation, and it can also be useful to specify a ratio. If we only know a little or maybe nothing at all about our application in this respect, the correct approach is to just let the JVM do the work and not to mess around with the flags. If the application runs smoothly, we can be happy that we didn’t put in extra effort where none was needed. And should we encounter performance problems or OutOfMemoryErrors, we would still need to first perform a series of meaningful measurements to narrow down the root cause of the problem before moving on to tuning.

-XX:SurvivorRatio

The flag -XX:SurvivorRatio is similar to -XX:NewRatio but applies to the areas inside the young generation. The value of -XX:SurvivorRatio specifies how large “Eden” should be sized relative to one of the two survivor spaces. For example, with -XX:SurvivorRatio=10 we dimension “Eden” ten times as large as “To” (and at the same time ten times as large as “From”). As a result, “Eden” occupies 10/12 of the young generation while “To” and “From” each occupy 1/12. Note that the two survivor spaces are always equal in size.

What effect does survivor space sizing have? Suppose that the survivor spaces are very small compared to “Eden”. Then we have lots of space in “Eden” for newly allocated objects, which is desirable. If all these objects can be collected during the next GC, “Eden” is empty again and everything is fine. However, if some of these young objects are still being referenced, we have only little space in the survivor spaces to accommodate them. As a consequence, most of these objects will be moved to the old generation right after their first GC, which is not desirable. Now let us consider the opposite situation: Suppose that the survivor spaces are relatively large in size. Then they have lots of space to fulfill their main purpose, to accommodate objects that survive one or more GCs but still die young. However, the smaller “Eden” space will be exhausted more quickly, which increases the number of young generation GCs performed. This is undesirable.

In summary, we want to minimize the number of short-lived objects that are prematurely moved into the old generation, but we also want to minimize the number and duration of young generation GCs. Once again we need to find a compromise, which in turn depends on the characteristics of the application at hand. A good starting point for finding an adequate compromise is to learn about the age distribution of the objects in the particular application.

-XX:+PrintTenuringDistribution

With the flag -XX:+PrintTenuringDistribution we tell the JVM to print the age distribution of all objects contained in the survivor spaces on each young generation GC. Take the following example:

Desired survivor size 75497472 bytes, new threshold 15 (max 15)
- age   1:   19321624 bytes,   19321624 total
- age   2:      79376 bytes,   19401000 total
- age   3:    2904256 bytes,   22305256 total

The first line tells us that the target utilization of the “To” survivor space is about 75 MB. It also shows some information about the “tenuring threshold”, which represents the number of GCs that an object may stay in the young generation before it is moved into the old generation (i.e., the maximum age of the object before it gets promoted). In this example, we see that the current tenuring threshold is 15 and that its maximum value is 15 as well.

The next lines show, for each object age lower than the tenuring threshold, the total number of bytes of all objects that currently have that age (if no objects currently exist for a certain age, that line is omitted). In the example, about 19 MB have already survived one GC, about 79 KB have survived two GCs, and about 3 MB have survived three GCs. At the end of each line, we see the accumulated byte count of all objects up to that age. Thus, the “total” value in the last line indicates that the “To” survivor space currently contains about 22 MB of object data. As the target utilization of “To” is 75 MB and the current tenuring threshold is 15, we can conclude that no objects have to be promoted to the old generation as part of the current young generation GC. Now suppose that the next GC leads to the following output:

Desired survivor size 75497472 bytes, new threshold 2 (max 15)
- age   1:   68407384 bytes,   68407384 total
- age   2:   12494576 bytes,   80901960 total
- age   3:      79376 bytes,   80981336 total
- age   4:    2904256 bytes,   83885592 total

Let us compare the output to the previous tenuring distribution. Apparently, all the objects of age 2 and 3 from the previous output are still located in “To”, because here we see exactly the same number of bytes printed for age 3 and 4. We can also conclude that some of the objects in “To” have been successfully collected by the GC, because now we only have 12 MB of objects of age 2 while in the previous output we had 19 MB listed for age 1. Finally, we see that about 68 MB of new objects, shown at age 1, have been moved from “Eden” into “To” during the last GC.

Note that the total number of bytes in “To” – in this case almost 84 MB – is now larger than the desired number of 75 MB. As a consequence, the JVM has reduced the tenuring threshold from 15 to 2, so that with the next GC some of the objects will be forced to leave “To”. These objects will then either be collected (if they have died in the meantime) or moved to the old generation (if they are still referenced).

-XX:InitialTenuringThreshold, -XX:MaxTenuringThreshold and -XX:TargetSurvivorRatio

The tuning knobs shown in the output of -XX:+PrintTenuringDistribution can be adjusted by various flags. With -XX:InitialTenuringThreshold and -XX:MaxTenuringThreshold we can set the initial and maximum value of the tenuring threshold, respectively. Additionally, we can use -XX:TargetSurvivorRatio to specify the target utilization (in percent) of “To” at the end of a young generation GC. For example, the combination -XX:MaxTenuringThreshold=10 -XX:TargetSurvivorRatio=90 sets an upper bound of 10 for the tenuring threshold and a target utilization of 90 percent for the “To” survivor space.

While there are different approaches to use these flags to tune young generation behavior, no general guideline is available. We restrict ourselves to two cases that are pretty clear:

  • If the tenuring distribution shows that many objects just grow older and older before finally reaching the maximum tenuring threshold, this indicates that the value of -XX:MaxTenuringThreshold may be too large.
  • If the value of -XX:MaxTenuringThreshold is larger than 1 but most objects never reach an age larger than 1, we should take a look at the target utilization of “To”. Should the target utilization never be reached, then we know that all young objects get collected by the GC, which is exactly what we want. However, if the target utilization is frequently reached, then at least some of the objects beyond age 1 have been moved into the old generation, and maybe prematurely so. In this case, we can try to tune the survivor spaces by increasing their size or target utilization.

-XX:+NeverTenure and -XX:+AlwaysTenure
Finally, I would like to quickly mention two rather exotic flags which we can use to test two extremes of young generation GC behavior. If -XX:+NeverTenure is set, objects are never promoted to the old generation. This behavior makes sense when we are sure that we don’t need an old generation at all. However, as such, the flag is apparently very risky and also wastes at least half of the reserved heap memory. The inverse behavior can be triggered with -XX:+AlwaysTenure, i.e., no survivor spaces are used so that all young objects are immediately promoted to the old generation on their first GC. Again, it is difficult to find a valid use case for this flag – it can be fun to see what happens in a testing environment, but apart from that I would not recommend using either flag.

Conclusion

It is important to run an application with an adequate configuration for the young generation, and there are quite a few flags to tune it. However, tuning the young generation without considering the old generation as well rarely leads to success. When tuning the heap or the GC settings, we should always take the interplay between the young and old generation into account.

In the next two parts of this series we will learn about two fundamental old generation GC strategies offered by the HotSpot JVM. We will get to know the “Throughput Collector” and the “Concurrent Low Pause Collector” and take a look at their basic principles, algorithms, and tuning flags.

Kommentare

  • Prajeesh Kumar

    7. August 2012 von Prajeesh Kumar

    It is a great article and the visualization helped me to understand a lot. Thank you for publishing this article and time

  • Hoi Le

    Greate post, the series would help me alots in tuning my apps, thanks for sharing.

  • James Watson

    13. May 2014 von James Watson

    Your comments on -XX:+NeverTenure are contradicted by some experimentation that I am doing with the 1.7 Hotspot JRE (Win). You can set the NewSize to be as large as 1K smaller than the total heap size. Interestingly enough, the JVM will allocate more than 1K to old in this scenario. The only time it is supposed to do this is when it cannot find space in the survivor area but that doesn’t add up because one half of the survivor space (500MB+) seems to be clear fully after every minor collection and there is 3.5G+ of free Eden after every collection. The old space only contained 7K after running my test for 4 hours.

    The reason I am experimenting with this approach is that I am trying to optimize for a distributed cache. In this scenario, 9.99999% of the objects will live for at least 2 minute and for no longer than 6. This violates the main assumption that default GC settings are based on: that most objects die young and if they don’t they will live for a long time. The optimal situation as I see it is for none of the cache objects to ever get promoted to the old generation as it is much more expensive when objects die in the old generation than when they die in the young generation. Ideally, I wan’t only the root of the cache to end up in old and for everything else to die in the young generation.

    I’ve had fairly good results and been able to sustain GC times of around 0.11 seconds every 90-110 seconds for days while collecting around 3.5GB on every minor collection. I’m struggling with how the JVM will seem ignore my settings in some situations when there seems to be no reason for it. I wish there was more discussion of these ‘exotic’ settings and acknowledgement of when the might make sense.

    -James

  • radha krishna

    16. January 2015 von radha krishna

    Simple and clear explanation of garbage collection and how to tune, all that was missing was some real use cases showing when each sizes are important, for example, larger new gen for stateless read intensive applications and larger old gen for cache intensive, long stateful applications, and so on. Similar use cases for survivor, tenuring ratios would have just completed this article.

    Do you have a single link for all the chapters, I want to save it as PDF reference and share with everyone in my company.

    • radha krishna

      16. January 2015 von radha krishna

      Could you please also compare/contrast this with G1GC and some example tuning for popular Java application stacks like Tomcat server hosting REST APIs with caching, Cassandra database for write/read intensive and messaging event driven architectures like AKKA/APAMA. You could consider writing these as a book may be.

    • radha krishna

      16. January 2015 von radha krishna

      In our write/read intensive cassandra with some caching, we noticed our 12 GB heap, new ratio=2, tenuredRatio=10 and survivor ratio=8 or about 250 MB is working fine because most read data generated garbage remains in survivor stays for a long time to reach 250 MB at >80% of the time, but is this a good idea to keep such read related data and cache in survivor spaces as very little amount 40 MB alone is filled up in survivor space? In this case there is no point in increasing survivor space as we don’t fill up but what about increasing tenured ratio to > 15, is it ok to live with 20-40ms par new GCs every half a min or reduce this size and tenuring ratio as most of the time only about 1-10 MB is surviving and flushed to old gen.
      Desired survivor size 250511360 bytes, new threshold 10 (max 10)
      – age 1: 21011760 bytes, 21011760 total
      – age 2: 4261448 bytes, 25273208 total
      – age 3: 2583752 bytes, 27856960 total
      – age 4: 1939944 bytes, 29796904 total
      – age 5: 1933296 bytes, 31730200 total
      – age 6: 1876272 bytes, 33606472 total
      – age 7: 1900928 bytes, 35507400 total
      – age 8: 1756536 bytes, 37263936 total
      – age 9: 2162784 bytes, 39426720 total
      – age 10: 1401480 bytes, 40828200 total

      • radha krishna

        17. January 2015 von radha krishna

        Forgot to include our fixed size configuration. Wonder if variable ranges to let JVM decide the sizes would make more sense?

        -XX:+UseThreadPriorities
        -XX:ThreadPriorityPolicy=42
        -Xms14G -Xmx14G
        -XX:+HeapDumpOnOutOfMemoryError
        -XX:NewRatio=2 -XX:MaxTenuringThreshold=10 -XX:+DisableExplicitGC -Xss250k
        -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
        -XX:SurvivorRatio=8 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB -XX:+UseCondCardMark -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -XX:PrintFLSStatistics=1

Comment

Your email address will not be published. Required fields are marked *