Performance & Optimization – New Threading Model
New Threading Model (NTM) is a new approach in dealing with transactions in eiPlatform. In the original model we have single thread pool to one transaction. In the NTM we can use any number of thread pools configured for any number of stages. By default, we have one thread pool by one stage. There, thread pools are very configurable which allows to tune eiP very accurately.
The NTM is implemented as a separated module. It allows to switch eiP to use the old model, just set the parameter ‘com.pilotfish.eip.useNewThreadModel’ to false.
One can use NTM without providing any additional configuration of the thread pools. All that is necessary for it is to have ‘com.pilotfish.eip.useNewThreadModel’ property set to true.
In that case the system works with default values of the settings from ‘eipServer.conf’ file. See ‘Default configuration’ section for more details.
To configure NTM explicitly one needs to create a ‘pools.xml‘ file next to the ‘route.xml‘ file in the appropriate Route folder. See next sections for more details.
Each thread pool has several parameters to set:
1) Base Thread Count – the number of threads to keep in the pool, even if they are idle.
2) Max Thread Count – the maximum number of threads to allow in the pool.
3) Idle Timeout – when the number of threads is greater than the core, this is the maximum time that excess idle threads will wait for new tasks before terminating.
4) Queue Size – the size of the queue to use for holding tasks before they are executed. If queue size is set to unbounded, system uses LinkedBlockingQueue. Otherwise – ArrayBlockingQueue with queue size as constructor parameter.
5) Expiration Timeout. If the thread in the pool takes more time than defined by this parameter – system pushes it out of the pool and terminates.
When NTM is not configured explicitly, it uses configuration settings from ‘eipServer.conf’ file.
To configure default behavior of the NTM you may need to define the following properties:
1) ‘com.pilotfish.eip.ntm.stage.baseThreadCount’ – Base Thread Count
2) ‘com.pilotfish.eip.ntm.stage.maxThreadCount’ – Max Thread Count
3) ‘com.pilotfish.eip.ntm.stage.idleTimeout’ – Idle Timeout
4) ‘com.pilotfish.eip.ntm.stage.stageQueueSize’ – Queue Size
5) ‘com.pilotfish.eip.ntm.stage.expirationTimeout’ – Expiration Timeout
There’s one more very important property – ‘com.pilotfish.eip.ntm.PoolPerRoute’. If this property set to ‘true’, the system initializes one thread pool for one route, not stage. In this case default thread pools options are used, such as:
Thread Pools Configuration
As was mentioned before, one can configure thread pools using ‘pools.xml’ file in the appropriate ‘route’ folder. The root element of this file is ‘PoolsConfig’ element.
So far, three different child elements are acceptable:
In this chapter we are interested in the first one. Each ‘Pool’ element defines separated thread pool and its parameters. It has 6 mandatory attributes (letter case is not important):
1) ‘name’ – the name of the certain thread pool. Try to use obvious names in order to be able to find necessary log lines easily.
2) ‘BaseThreadCount’ – base thread count of the thread pool.
3) ‘MaxThreadCount’ – max thread count of the thread pool.
4) ‘IdleTimeout’ – idle timeout of the thread pool (in milliseconds).
5) ‘QueueSize’ – the size of the queue for this thread pool.
6) ‘ExpirationTimeout’ – expiration timeout of the tasks executed in this thread pool (in milliseconds).
Only one type of child is allowed for ‘Pool’ element. It is ‘Stage’ element. Each stage element defines a certain stage or set of stages that will use this thread pool. ‘Stage’ element has two attributes:
1) ‘type’ – the type of the stage. This is mandatory attribute and it has a set of allowed values:
? – “Listener”
? – “Transport”
? – “XSLT”
? – “Transformation”
? – “Processor”
? – “Routing”
? – “Infrastructure”
? – “Forking”
? – “Joining”
? – “Any” – any not explicitly configured stage will use one common thread pool. Only “Any” or “Every” is acceptable, not both.
? – “Every” – any, not explicitly configured, stage will use its own thread pool.
2) ‘name’ – name of the stage. E.g. ‘test1.Directory Listener’. This field might be empty or missing all together depending on the ‘type’ attribute.
‘Stage’ element can’t contain any children.
To show you how it works, sample pools.xml is provided below:
<PoolsConfig> <Pool name="MainTransportPool" BaseThreadCount="4" MaxThreadCount="10" IdleTimeout="500" ExpirationTimeout="10000" QueueSize="10"> <Stage type="Transport" name="test1.Directory Transport"/> </Pool> <Pool name="Pool_Others" BaseThreadCount="4" MaxThreadCount="10" ExpirationTimeout="10000" IdleTimeout="500" QueueSize="10"> <Stage type="Every"/> </Pool> </PoolsConfig>
This configuration XML defines three thread pools:
1) ‘MainTransportPool’ – as we can see in the XML, this thread pool will work with only one stage – ‘test1.Directory Transport’. Such approach is necessary when we need to tune the thread pool configuration for some specific stage or stages.
2) ‘Pool_Others’ – this thread pool is necessary to service all other stages. Considering that we chose ‘Every’ stage type, every not explicitly configured stage will use its own thread pool.
It is possible not to configure thread pool for other stages. In this case the system will create one pool per stage with default thread pool parameters.
Listeners’ volume configuration
There’s a possibility to control listener volumes. One can specify how many transactions should be transferred through a specific listener in a given period of time (or during some period). If listener doesn’t process expected amount, the system decides that it is hung/deadlocked and restarts listener.
One should be very careful configuring this feature. Incorrect configuration may cause unnecessary listener restarts.
To configure listener’s volume checker you need to open appropriate route’s ‘pools.xml’ file and add ‘MinimumListenerVolume’ element(s) as a child of ‘PoolsConfig’ element. This element should contain two mandatory attributes:
1) ‘ListenerName’ – the name of the listener to control.
2) ‘CronCleanupAt’ – cron expression that defines the time, when system should clean-up transaction statistics. For more details see https://docs.oracle.com/cd/E12058_01/doc/doc.1014/e12030/cron_expressions.htm
‘MinimumListenerVolume’ element should contain one or more ‘DurationVolume’ element. Each of such elements configures one specific checking rule. Depending on rule type, this element might have two different attribute sets:
1) Time scheduled rule. This type controls listener’s volume for defined period of time. For this type one should define three attributes:
1. ‘Start’ – start time of the control period. Format is “hh:mm”, e.g. “18:50”.
2. ‘End’ – end time of the control period. Format is “hh:mm”, e.g. “18:56”.
3. ‘TransactionsCount’ – count of the transactions that should be processed through this listener during defined period of time. If listener processes less than defined number of transactions during this period of time – system restarts it.
2) Duration rule. This type controls listener’s volume for duration of time. For this type one should define two attributes:
1. ‘DurationMS’ – duration of the checking period in miliseconds.
2. ‘TransactionsCount’ – count of the transactions that should be processed through this listener during defined duration of time. If listener processes less than defined number of transactions during this duration of time – system restarts it.
You should pay attention to ‘CronCleanupAt’ attribute. The issue is that to control listener’s volume system should collect minimal data about the processed transactions. Thus, the system needs to clean-up this information from time to time. You need to avoid clean-up time crossing with the control times.
You shouldn’t forget to configure listeners’ volume check period in ‘eipServer.conf’ file. Property ‘com.pilotfish.eip.ntm.listenerVolumeCheckTimeoutMs’ is responsible for that.
The example of the listener’s volume checker configuration is below:
<PoolsConfig> ... <MinimumListenerVolume ListenerName="test1.Directory Listener" CronCleanupAt="0 15 10 ? * *"> <DurationVolume Start="18:32" End="19:33" TransactionsCount="200" /> <DurationVolume DurationMS="10000" TransactionsCount="5" /> </MinimumListenerVolume> <MinimumListenerVolume ListenerName="test1.FTP Listener" CronCleanupAt="0 0 12 * * ?"> <DurationVolume Start="10:00" End="11:00" TransactionsCount="20" /> </MinimumListenerVolume> </PoolsConfig>
In this example we define volume checkers for two listeners. For ‘test1.Directory Listener’ we define two rules. In such combination of rules this listener will have to process more than 200 transactions from 18:32 till 19:33 every day and more than 5 transactions in 10 seconds. The ‘test1.FTP Listener’ will have to process 20 transactions from 10:00 till 11:00. For first volume controller clean-up will be performed at 15:10 every day. For second one will fire clean-up at 12pm (noon) every day.
Transaction saving configuration
There’s an ability to enable saving transaction before certain stage execution. This is necessary in order not to lose transaction data after any error on this stage, and have an ability to restart it. Thus, NTM removes saved transaction from disc after successful stage execution.
NTM keeps transaction data file in the ‘saved-transactions’ directory of the main interface folder (e.g. eip-root). The name of the file with transaction data consists of route name, stage name, stage type and transaction ID. It has “stx” extension.
One can enable any stage to save transactions before execution. To do it, you need to edit appropriate ‘pools.xml’ file. Add ‘SaveTransactionsBeforeStageExecution’ element as a child of the ‘PoolsConfig’ element. This element shouldn’t have any attributes but must have one of more child elements or ‘Stage’ elements, that are the same as ones described in section ‘Thread Pools Configuration’. An important thing to remember is: this stage can’t be type ‘Any’. The only allowed generic type is ‘Every’, which activates transaction saving for every stage in the route.
There could be any number of the ‘Stage’ elements, but there’s no need to define any other stage elements if there’s one “Every” stage type.
Here’s an example of the configuration:
... <PoolsConfig> <SaveTransactionsBeforeStageExecution> <Stage type="Processor" name="ROOT.test.Processor"/> <Stage type="Transport" name="ROOT.test.Transport"/> </SaveTransactionsBeforeStageExecution> </PoolsConfig>
This configuration will force processor and transport to save transactions before execution.
NTM allows collection of statistics data about transaction data processing and other technical details. There are five properties in eiP server configuration that are responsible for statistics collecting:
1) ‘com.pilotfish.eip.ntm.heartbeat.period’ – heartbeat period in milliseconds. It defines timeout for saving statistics data about each thread pool.
2) ‘com.pilotfish.eip.ntm.collectStatistic’ – if “true”, system collect
3) ‘com.pilotfish.eip.ntm.memoryCheckPeriod’ – period in milliseconds when system stores information about memory and threads in runtime.
4) ‘com.pilotfish.eip.ntm.dumpStatisticPeriodSeconds’ – period of time (in milliseconds) when system saves statistical data to disk. Before that, NTM keeps statistical data in the memory.
5) ‘com.pilotfish.eip.ntm.dumpStatisticFolder’ – the folder to store statistical data.
Be careful with statistics collector configuration. You should define heartbeat period considering quantity of the transactions in certain route. E.g. there’s no sense to set heartbeat period to 100 milliseconds if there is only couple transactions per minute. Also, there’s no need to set memory check period to very small number. You can easily notice memory changing tendency if you set it to 1000 milliseconds or more. Recommended dump statistics period value depends on certain aims. If you want to decrease hard drive IO operations and you’re sure about the availability of the memory (or there is not so much statistics collected), you can set it to a larger value.
System collects the following statistical information:
1. Transaction statistic – transaction ID, start time, end time, execution status (success/failure) and error message/exception if transaction processing failed.
2. Stage statistic – for each stage and each processed transaction it contains information about its start time, end time, stage status (success/failure), error message and flag that indicates if stage has timed out.
3. Thread pool statistic – time of the snapshot of the thread pool state, number of the active tasks and approximate total number of tasks that have ever been scheduled for execution.
4. Runtime memory statistic – time of the memory snapshot, free memory, total memory, max memory.
5. Runtime threads statistic – thread count, peak thread count and total started thread count.
As was mentioned above, system stores statistical information in files. These file are not readable using standard text viewing applications, because they contain serialized objects. These files have ‘eipstat’ extension. Their name consists of start and end time of the statistical data they contain. For example, here’s typical statistical dump file name: ‘102009_145534-102009_155534.eipstat’.
Statistics review tool
There’s a UI statistics review tool implemented. You can access it either via eiConsole menu (Tools -> EIP Statistics Review…) or by running ‘com.pilotfish.eip.gui.console.stat.StatisticsReview’ class. You can see how it looks below:
In the top of the window you can specify the folder to look for statistics files. If you start the utility from eiConsole it opens appropriate ‘statistics’ folder automatically. In the appropriate start and end fields you can specify start and end times to review the statistical data. In the text area below you can see statistic summary for given period.
eipServer.conf properties index
1) ‘com.pilotfish.eip.ntm.stage.baseThreadCount’ – Base Thread Count for thread pool in pool-per-stage mode.
2) ‘com.pilotfish.eip.ntm.stage.maxThreadCount’ – Max Thread Count for thread pool in pool-per-stage mode.
3) ‘com.pilotfish.eip.ntm.stage.idleTimeout’ – Idle Timeout for thread pool in pool-per-stage mode.
4) ‘com.pilotfish.eip.ntm.stage.stageQueueSize’ – Queue Size for thread pool in pool-per-stage mode.
5) ‘com.pilotfish.eip.ntm.stage.expirationTimeout’ – Expiration Timeout for thread pool in pool-per-stage mode.
6) ‘com.pilotfish.eip.ntm.PoolPerRoute’. If this property set to ‘true’, system initializes one thread pool for one route (pool-per-route mode), not stage (stage-per-route mode).
7) ‘com.pilotfish.eip.ntm.heartbeat.period’ – There’s such a notion as ‘heartbeat’ in the NTM. It means that every X milliseconds system calls necessary listeners and allows them to handle anything in the runtime at this precise moment. So, this property defines this heartbeat period in milliseconds.
8) ‘com.pilotfish.eip.ntm.collectStatistic’ – defines if system should collect statistical data. If “true” – system collects statistics.
9) ‘com.pilotfish.eip.ntm.dumpStatisticPeriodSeconds’ – period of time (in milliseconds) when system saves statistical data to disk. Before that, NTM keeps statistical data in the memory.
10) ‘com.pilotfish.eip.ntm.dumpStatisticFolder’ – the folder to store statistical data.
11) ‘com.pilotfish.eip.ntm.memoryCheckPeriod’ – period in milliseconds when system stores information about memory and threads in runtime.
12) ‘com.pilotfish.eip.ntm.allowCoreTimeout’ – If false (default) core threads stay alive even when idle. If true, core threads use keepAliveTime to time out waiting for work.
13) ‘com.pilotfish.eip.ntm.listenerVolumeCheckTimeoutMs’ – Timeout of the checking listeners’ volumes (milliseconds).
14) ‘com.pilotfish.eip.ntm.saveDataToFileOnPoolPause’. This property defines whether system should save transaction data to file on thread pool pause. Note, that if your interface will presumably process big transactions, you should set this property to ‘true’, in order not to cause memory issues. On the other hand, if the transactions are going to be small, it’s more desirable to set it to ‘false’.