eiConsole v.24R1
Documentation
Process Orchestration
Training KitThe “Process Orchestration” components are made up of the Process Orchestration Listener and the Process Orchestration Transport. These disparate modules communicate via a “Process Instance,” which is a system-wide, unique identifier for an internal process (which may span multiple topologies, timeframes, etc.). The primary use of these two components is to perform an asynchronous join.
This example includes two basic Routes:
1 – Directory to PO Transport
2 – PO Listener to Directory
The first Route, “1 – Directory to PO Transport,” polls an arbitrary directory for files, storing the usual Transaction Attributes provided by a Directory / File Listener and passing each such Transaction (one per file polled) un-transformed to a Process Orchestration Transport. This Transport is configured like so:
The “Event Type” item provides a message identifier to separate messages for the provided Process Instance. If there are multiple Process Orchestration components in use and the keying mechanism isn’t necessarily guaranteed to be unique, then the Event Type can be used to add another key to prevent such conflicts. This value will need to match a similarly-named item within the Process Orchestration Listener configuration, so it’s advised to simply name this after what the join operation is. Here, we’ve left it as the default.
The “Store transactions in memory” allows storing the queue of the waiting transactions in memory, not on the hard drive.
The “Process Instance” item is the unique identifier for the Process Instance. In practical terms, this is the “key” to a group of Transactions. The value we’ve provided here is the Transaction Attribute “com.pilotfish.PollingCycleID,” which is a value provided by a Directory Listener starting at 1 and incrementing for each polling cycle within an eiPlatform start-up. This means that if the Directory Listener is on its third polling and picked up seven files, each of the seven created Transactions will have this Attribute matching (probably as “3”). The Process Instance is the unique key for the joining context.
The “Process Component” item is an identifier (preferably unique to make it easy for us to distinguish each component) for this Transaction within the Process Instance context. We may wish to have multiple components feeding into the same join operation, each identified by what it represents. You may opt to uniquely identify this value or just name it according to what it represents. Here we’ve specified the name of the File meaning each Transaction we receive will have Process Component name as the name of the file. This value is most typically used to specify names for particular joining criteria or to simply make the resulting join more visible with respect to what components represent what data source.
The second Route, “2 – PO Listener to Directory,” specifies a Process Orchestration Listener which performs joining and then passed the joined result, un-transformed, to a Directory Transport to deliver to an arbitrary file system directory. The Listener is configured like so:
The “Event Type” item corresponds to the Process Orchestration Transport Event Type and is largely used here to specify what types of events/messages this Listener “listens” for. This value is what keys this particular Listener to other Transports.
The “Firing Conditions” tab provides three items that specify available conditions on which this Listener will actually create a Transaction:
“Process Component Names Required” is a comma-separated list of component names (see the Transport’s “Process Component” field) that must have been “received” (generated a message for) by this Listener before it will trigger. If the Listener is intended to join three disparate components, those should be listed here.
“Transaction Count Required” is a value for the number of messages (Transactions) this Listener must receive before it will trigger. Here we’ve specified the Transaction Attribute “com.pilotfish.PollingCycleSize,” which is available because this expression is evaluated against each Transaction this Listener “receives.” This value is provided by the Directory Listener with each polling cycle and gives the number of files (Transactions created) polled by the Listener.
“Conditional Statement” is an expression that must evaluate as “true” for the Listener to trigger. This could be any literal or dynamic value and is evaluated against each Transaction received by this Listener.
The “Type of Transaction to Create” item identifies what the Listener does with the aggregate of Transactions. There are several options here, but the one of primary interest (and the one used most often, if not exclusively) is “Aggregate Transactions,” meaning all received Transactions should be joined into a single Transaction.
The “Remove Process Instance” item specifies when the Process Instance this Listener is handling is removed from memory. We’ve specified “When Transactions Fired” here to ensure that this is only cleared once we manage to successfully trigger a Transaction.
The “Timeouts” tab allows you to set the “Timeout interval” – the time (in seconds) before a process handled by this Listener times out.
To summarize this example:
1. We poll some directory, X, for files and assign each Transaction created a PollingCycleSize and PollingCycleID. Each polling cycle will increment the latter by one, and the number of files polled will provide the size. If the third time the Directory Listener runs picks up six files, then these will be, respectively, 6 and 3.
2. Each Transaction is passed to a Process Orchestration Transport, specifying the PollingCycleID as a unique key for this join. This means that once joined, each set of files picked up will be joined (so, our third run of six files will be joined, our fourth run will be joined, etc).
3. Our Process Orchestration Listener is set to join once we’ve received the number of Transactions specified by our PollingCycleSize.
4. The joined result is delivered to a directory, Y.
Finally, with respect to a test run, we’ve provided two source files, both with this content:
person.csv 1,Tom 2,Dick 3,Harry
address.csv 1,1 Some Street,Someplace,CT,06441 2,2 Some Street,Someplace,CT,06441 3,3 Some Street,Someplace,CT,06441
The joined result is:
<XCSData> <ProcessComponent name="address"> 1,1 Some Street,Someplace,CT,06441 2,2 Some Street,Someplace,CT,06441 3,3 Some Street,Someplace,CT,06441 </ProcessComponent> <ProcessComponent name="person"> 1,Tom 2,Dick 3,Harry </ProcessComponent> </XCSData>