Socrata Open Data Listener

In PilotFish’s Integration Platform, the Socrata Open Data Listener allows querying Socrata Open Data services with the custom SoQL queries. This Listener has 4 tabs: Basic, Advanced, Transaction Logging, Inactivity, Throttling, and SoQL Query.

Listener (Adapter) Configuration Drop-Down List

Basic Socrata Open Data Listener Configuration Options

On the Basic tab configuration you can specify:

• Polling interval – how often the Input File query is executed
• Service URL – URL of the Socrata Open Data service. Sandbox service by default
• Email – username used for authentication
• Password – password used for authentication
• Application Token – application (API) token of the user
• Resource ID – ID of the resource to query
• Return Data Type – what type of data will be returned

Socrata Open Data Listener Basic Configuration Options

Advanced Socrata Open Data Listener Configuration Options

The Advanced tab allows you to specify whether or not you should only run the Listener when it is triggered externally, and how many elements should be serialized.

Also, you can set the following options:

• Allow Command-Line Invocation – if enabled, the Listener can be invoked using the CLI client application
• Restart on Listening Error – if enabled, the Listener will be restarted after an error occurs
• FIFO Queue Name – the FIFO options enable a “First In, First Out” queuing mechanism between Listeners and Transports. If a FIFO Queue Name is provided, it will be used as a key for a transaction queue. Transactions will be written to this queue before they reach a Transport. The transactions in this queue will be ordered according to when they were created by the Listener.
• FIFO Queue Delay – this is the interval between updates or checks against that queue. Providing a queue name guarantees that a given Transport sends transactions in the same order that the Listener created them in.

Socrata Open Data Listener Advanced Configuration Options

Transaction Logging Socrata Open Data Listener Configuration Options

The Transaction Logging tab allows you to specify:

• Transaction Logging Enabled – if enabled, allows transaction events originating from this Listener to be logged by a TransactionEventListener
• Log Transaction Data – if enabled, logs transaction data body
• Log Transaction Data Base64 – if enabled, logs transaction data body as Base64
• Log Transaction Attributes – if enabled, logs transaction attributes
• Log All Attributes – if enabled, no attributes will be filtered
• Allowed Attributes –  attributes that are allowed to be logged

Socrata Open Data Listener Transaction Logging Configuration Options

Inactivity Socrata Open Data Listener Configuration Options

The Inactivity tab allows you to specify:

• Enable Inactivity Monitor – check this box to enable inactivity monitoring. This will throw a non-transaction exception if the specified number of transactions haven’t been processed in the specified time interval.
• Min. Transactions to Expect – the number of transactions to expect to be completed per monitoring interval
• Monitoring Interval – how often to check the specified number of transactions that have been processed
• Times to Monitor – if set, monitoring will be done during the defined times of the day. To ignore, set start and end time equally.
• Days to Exclude from Monitoring – inactivity monitoring will not occur on the days specified
• Include Errors in Transaction Count – if checked, transactions that attempted to start, but failed at the Listener stage, will also be counted

Socrata Open Data Listener Inactivity Configuration Options

Throttling Socrata Open Data Listener Configuration Options

The Throttling tab allows you to specify:

• Throttling Mode – the throttling mode to use for limiting the number of transactions or messages emitted by this Listener. “Timed” will limit transactions based on time intervals, while “Concurrent” will limit based on a concurrent number of transactions. “Concurrent” mode requires a Throttling Response Processor step later in your interface workflow to acknowledge completion.

Socrata Open Data Listener Throttling Mode

• Throttling Mechanism  – the mechanism to use for throttling messages. “Blocking” prevents the Listener from continuing to process and emit messages altogether, while “queued” pushes received messages into the interface queue or a default, in-memory queue.
• Max Concurrent Messages – how many messages can be concurrently processed, either by time-based limits (allow X per second) or synchronous (allow X at any time)
• Timed Emission Interval – the interval for time-based limits (allow X per X timed emission interval)
• Synchronous Timeout Interval – the interval to wait for a synchronous response before failing

Socrata Open Data Listener Throttling Configuration Options

SoQL Query Socrata Open Data Listener Configuration Options

On the SoQL Query tab, you can specify:

• Select All – if selected, all the data will be selected from the data set
• SoQl Query – Socrata Query Language expression

Socrata Open Data Listener SoQL Query Configuration Options