PPM Cache Tuning

Tuning PPM Cache performance should be done at the same time as tuning PPM JVM heap size. This means finding the right balance between the following two things:

  • JVM heap size

    Before PPM 9.20, only 32-bit JVM was supported, which limited the JVM heap size to 1.3 GB. Since PPM 9.20 and the adoption of 64-bit JVM, the size of the JVM heap memory is only limited by the installed physical memory. However, too large heap size can result in long full garbage collection times, during which the application is unresponsive. It is common to see PPM JVM heap sizes of up to 4 GB, and sometimes larger on Service nodes.

  • Cache size

    A larger cache size means more cacheable objects, a better hit rate, and fewer objects to reload from the database, which result in better application performance. However, if cached objects end up taking too much memory, it will impact the performance of the application and might even cause out-of-memory problems. Note that only legacy cache is prone to cause such memory problems, as the JVM will automatically discard objects from the new cache whenever the available free memory runs too low.

To Summarize

  • If JVM heap size is too large, full garbage collection periods will be too long and the application might become unresponsive for seconds, degrading users experience.

  • If caches are too large and end up taking too much memory, the performance of the application will degrade and in the case of legacy cache it might even cause out-of-memory issues.

Following these steps may end up conflicting with each other. If that happens, use your best judgment.

  • If you see a large number of swaps (in legacy cache) or “max cache size reached” flushes (in new cache), increase the cache size.
  • If you see some “soft reference reclaimed” flushes (in new cache), increase the JVM heap or reduce the cache size for caches take large amounts of space.
  • If you see a high miss rate (above 20% in legacy cache) or a low hit rate (below 80% in new cache) even after prolonged PPM usage (at least one day of heavy usage), increase the cache size.
  • If you notice long full garbage collection time (many seconds) during which the system is unresponsive, reduce the heap size or better tune the JVM garbage collection. Note that you will need to use a JVM monitoring tool in order to ensure that JVM pauses are caused by full garbage collection.
  • There is no “standard” cache configuration. Measure, tune, flush, and repeat until you reach satisfactory numbers. All PPM usages are different, and as a result cache configurations should be tuned accordingly.
  • When tuning the PPM cache, try to do so after capturing statistics during the highest peak load time (usually happening on Friday afternoon or Monday morning). Tuning cache only makes sense if it is done to properly handle peak load usage.
  • Try not to flush the caches (using kRunCacheManager or ksc_flush_cache) in an automated way unless you really have to. If you use kRunCacheManager, NEVER use the “A” option to flush all caches in an automated script. You should not flush more caches than necessary.
  • If memory limit does not allow you to set the proper max cache size for all the entities, you can rank the caches by the value “Average Load time” x “Misses”, and first increase the cache size of the cache(s) with the highest value. They are the caches most likely to have a measurable performance impact.
  • You might want to tune differently the PPM nodes in your cluster depending on whether they are Service nodes or Web User only nodes. The entities loaded (and thus the optimal cache settings) are different. For example, a pure Service node will never load portlets or menus, but might need a larger cache size for fiscal periods.

  • Don’t forget that when you use kRunCacheManager.sh, it will always flush selected caches on all nodes of your PPM cluster, but it will reset caches statistics and force garbage collection only on the node it is connected to.

In the environments where more than one language is installed on PPM, if a same object is loaded by different users using different languages, the object will be stored in the cache once for each of the different languages used. Your cache should be set accordingly.

For example, on a system with 3 languages installed for PPM, if you have 20 request types, you should set the maxSize value for the request types cache to 60, i.e. 20 (request types) times 3 (languages). However, if you have 1000 users, you might still set the users cache maxSize value to 1000, since a given user will usually always log in PPM using the same language; so each user object will be loaded in the cache most often once, using the language of the user.