Upcoming Release: Process data pipeline

We are soon releasing the first version of a long-anticipated feature - the process data pipeline. Think about it as a database for your robots, but which is also convenient for business users to interact

Most automated processes revolve around processing external data or files (e.g., Excel sheets, PDFs) as input or provide them as an output. Also, data flows in processes can be asynchronous, so data storage can be used as an intermediary between process phases. Process data pipeline addresses these problems by allowing storing of data used in processes outside of the execution time. No need to host and maintain your own MongoDB or SQL!

Work items are the entities used in Robocorp Cloud to store any kind of data meant to be processed by robots. By using the data pipeline to handle every transaction in the process by work item basis, the state of processing is always up-to-date. For example, splitting handling of 1000 customers to 1000 work items, even if the robot crashes in an unexpected way, the processing state of each item stays known. This also allows splitting the work to multiple execution environments in one-to-many style of processing. Think about it as “work queues”, but with much more added functionality.

Old processes in Cloud are backward compatible with the data pipeline, and changes are mostly visual in status indicators.In the first version, we are rolling out some basic functionality:

  • Improved process dashboard
  • Processing of multiple work items in one process run
  • Tracking of work items in Input, In-progress, and Output storages
  • Monitoring of work item success rate
  • Process run status indicator showing if processing or work items have succeeded

During the H1 of 2021, we will be extending functionality by adding more features like:

  • Processing multiple items from the queue one-by-one (looping through the queue)
  • Processing work items with environment groups (parallel processing with self-hosted environments)
  • Extensive API:s to interact with the inputs and outputs

More information coming next week - stay tuned!!

3 Likes