Understanding Tasks and Concurrency in Workflows

Understand how task concurrency affects high-volume Switch API workflows before you configure request queues, batching, and ramp-up behavior.

For implementation settings and a pre-run checklist, see Rate Limit Best Practices.

Define a task

In Switch, a task is a unit of work your workflow needs to perform. Most tasks map to one or more API calls, such as:

Creating customer accounts
Fetching tariff information
Setting usage profiles
Running Savings Analyses

When your integration runs many tasks, the number of tasks running at the same time affects response times, error rates, and overall workflow reliability.

Understand concurrency in a workflow

Concurrency is the number of tasks running at the same time. In an API workflow, this usually means the number of simultaneous API calls your application is making.

Concurrency can help your integration process work faster, but high concurrency can also create sudden request spikes. Use a fixed concurrency limit, gradual ramp-up, batching, and monitoring to keep high-volume workflows predictable.

Key concepts

Concurrency limit: The maximum number of simultaneous API calls your application can make.
Gradual ramp-up: Increasing concurrency gradually over 8–10 minutes helps avoid sudden spikes in system load.
Batching: Processing a large workflow in smaller groups instead of launching every task at once.
Monitoring: Tracking response times, HTTP status codes, retry counts, and error rates while the workflow runs.

Example workflow for acceptable use

Scenario

A customer runs 1,000 API calls while adhering to:

A concurrency limit of 4
A gradual ramp-up of request rates over 8–10 minutes

Steps

Initial setup:
- Create a few accounts, set tariffs, and create initial usage profiles.
- Monitor system response times and error rates.
Gradual ramp-up:
- Start with a concurrency of 1 and gradually increase it to 4 over 8–10 minutes (e.g., add one concurrent request every 2–3 minutes).
Batch processing:
- Process profiles and Savings Analyses in small batches while respecting the concurrency limit.
- Introduce short delays between batches.
Monitoring and adjustments:
- Continuously monitor response times and error rates.
- Adjust the rate of API calls or introduce longer delays as needed.

Example timeline

Time range	Concurrency	Behavior
Minutes 0–2	1 concurrent API call	Begin processing the first batch of 50 profiles.
Minutes 2–4	2 concurrent API calls	Increase gradually while monitoring response times and errors.
Minutes 4–6	3 concurrent API calls	Continue processing the batch and watching for elevated latency or errors.
Minutes 6–8	4 concurrent API calls	Hold at the concurrency limit until the batch is processed.
After batch completion	Pause briefly	Wait before starting the next batch of 50 profiles and monitor system performance during the delay.

Example workflow for unacceptable use

Scenario

A customer runs 1,000 API calls without following concurrency and ramp-up guidelines.

Issues

Immediate high concurrency:
- Starts with a high concurrency of 50, exceeding the recommended limit.
- No gradual ramp-up, causing a sudden spike in system load.
No batch processing:
- Attempts to process all profiles and run savings analyses simultaneously.
No monitoring:
- Ignores system performance metrics, leading to prolonged system strain.

Example timeline

Minute 0:
- Starts with 50 concurrent requests.
- Attempts to process all 1,000 profiles simultaneously.
- Ignores increasing response times and error rates.

Summary

Use these practices when you design high-volume Switch workflows:

Adhere to concurrency limits to prevent sudden request spikes.
Gradually ramp up request rates over 8–10 minutes.
Process work in batches instead of starting every task at once.
Monitor response times and errors continuously and adjust settings as needed.

Failure to follow these guidelines can result in:

Increased response times
Higher error rates
Retries or failed workflow steps
Temporary instability in high-volume workflows