Scaling Workflows with Multiple Items

Overview

When using workflows to process large numbers of items, performance can start to become a limiting factor. To give you more control over how workflows are performed, you can now choose whether workflows operate in Single Item or Multiple Item mode.

Typically, a workflow that outputs a single item is easier to work with. However, when there are thousands of items to process, running thousands of corresponding workflows can get a bit slow!

In such situations, processing multiple items in a single workflow provides a huge performance boost, which is typically worth the price of having the workflow output all the items at once.

The current behaviour is as follows:

  • Event workflows – a workflow is performed for each item.

  • Manual workflows – a single workflow performs on all items passed to it (10,000 items max).

The upcoming changes essentially give you the option of swapping these behaviours if desired.

Who will be affected?

The changes will affect any users that create and edit workflows via the Alloy API. The new properties described below will eventually surface in Alloy Web.

Details

Event workflows

A new optional property outputMode will be added to the EventTrigger on the Workflow Create (ReDoc) and Workflow Edit (ReDoc) endpoints. The default value is OneItem.

When set to ManyItems, the event workflow will run with as many items as possible on its trigger node (depending on the item collection/event type). The performance will be orders of magnitude faster then processing the equivalent workload in OneItem mode.

Expected Release Date – 29th September 2022 as part of the Alloy v2.40 release.

Manual workflows

A new optional property WorkflowManualTriggerOutputMode will be added to the ManualTrigger on the Workflow Create (ReDoc) and Workflow Edit (ReDoc) endpoints. The default value is ManyItems.

When set to OneItem, the manual workflow will run once for each item passed to it. For example, if a workflow creates an invoice when supplied with an account's Item Id, the workflow will run once for each account, thereby generating an invoice for each one. It wouldn't be possible to do this in ManyItem mode.

Expected Release Date – 27th October 2022 as part of the Alloy v2.41 release.