Tuning eiPlatform Memory and CPU Usage

The Problem

The default eiPlatform configuration is designed to work well in most typical installations.  However, this default configuration may not be optimal when processing large documents or a large number of small documents.

Solutions

The following list contains configuration changes that can be made to maximize performance and throughput.

1. Polling Cycles

Most of the listeners operate via polling, where at a set interval they will attempt to retrieve messages from whatever they are listening too. Polling cycle intervals should be set based on two things: how frequently you expect to have new messages to process, and how much of a strain processing each message will place on the system.

2. Max Transactions

Many (although not all) of the listeners have a maximum Transactions setting. This is effectively a message limit, it ensures that in a given polling cycle only that many messages will be picked up and sent into the system via transactions, regardless of how many messages are actually available to process. This is another great way to control how much of a strain you place on the system in any given Route.

3. Cache Type

While processing a transaction, the message data is stored in an internal cache. These caches come in several types based on your needs and can be configured in the eipServer.conf via the “com.pilotfish.eip.transact.defaultCache” property. Here is a quick summary again of the available types:

• memory – This is an in-memory cache that is always initialized at 1 MB and expands to larger sizes as-needed. It’s better if you’re primarily dealing with a few larger files rather than many smaller ones.
• segment – This is another in-memory cache, but it is initialized at a much, much tinier size and expands to larger sizes as-needed. It’s better if you’re primarily dealing with lots of smaller files, rather than a few large ones.
• file – This is a file-based cache. The data is not stored in memory, but in a temporary file, for the duration of the transaction. While performance is obviously reduced due to the need for I/O, it trades the memory burden from being on the RAM to being on the disk.

4. Thread Pool Settings

The most important settings when optimizing transaction concurrency are related to thread pool configuration. The eiPlatform runs transactions in parallel, and each needs a thread for execution. These thread pools can be organized in two different ways, each for a different use case, and controlled by the com.pilotfish.eip.ntm.PoolPerRoute=true or false option in the eipServer.conf:

• Pool Per Route: In this setup, each route has a pool of execution threads that each stage shares when they need to run. This is the default thread pool mode and is ideal for most cases, especially when dealing with a smaller volume of large files where heavy throughput is less of a factor. The two settings that control the per-Route pool size are:
  • com.pilotfish.eip.transact.baseThreadCount=4
  • com.pilotfish.eip.transaction.maxThreadCount=8
• Pool Per Stage: In this mode, each stage has its own pool of threads for execution. There is a larger memory footprint since each Route will contain at a minimum 7 Stages, but the overall throughput is higher in almost all cases. For example, if a Route has an intensive XSLT or database Stage, in per-Route threading this would cause the whole Route to bottleneck if all the threads were stuck at that stage. In per-Stage threading, the other stages would still have threads to be able to run alongside the intensive one, reducing overall bottlenecks. The settings that control per-Stage pool size are below, also note they are lower than the per-Route defaults since each stage will have its own pool rather than the whole Route:
  • com.pilotfish.eip.ntm.stage.baseThreadCount=2
  • com.pilotfish.eip.ntm.stage.maxThreadCount=4

5. Optimize Thread Pool Settings

In most cases, just choosing the right threading model – per-Route or per-Stage – and using the pool size defaults will be all that’s needed to optimize execution to a better place. If more granular control is desired, there are some things to keep in mind:

• More threads don’t necessarily mean things will go faster. Since at a certain point every thread will need to use the CPU to run, setting the maximum threads to a value wildly higher than the number of CPU cores will most likely hurt performance as all these threads fight for CPU time. Each thread also incurs a memory cost, so there will be less memory for the running parts of the system to use when idle threads are sitting around waiting to run. Typically, don’t set the max thread count to any higher than 2x the number of cores in your CPU, and try to stick to multiples of the number of CPU cores (4 core CPU, 8 threads max).
• The idle timeout can be helpful if you know the system will be spending lots of time idle with a few periods of burst activity. By setting the com.pilotfish.eip.ntm.allowCoreTimeout and idleTimeout settings, when a thread has been sitting idle in the pool for too long it will be destroyed and then recreated when it is needed again. This will help keep memory usage low during periods of low activity, though setting the idle timeout too low (less than 10 seconds) can hurt performance as threads are being destroyed and remade unnecessarily.

6. Increase eiPlatform Max Memory

In the eiPlatform Service.vmoptions file, the max memory setting defaults to 1 GB.  This means that the eiPlatform will not use more than 1 GB of RAM regardless of how much RAM is available on the server.

You can modify the –Xmx property you’ll find in there to a larger amount of megabytes, let’s say 4 GB for now, but it can be whatever you want it to be. To do 4 GB, it would look like –Xmx4096m.