Rate Limit Best Practices

What Defines a Task

In the context of our product, a task refers to a unit of work that your workflow needs to perform. Specifically, this includes operations such as:

• API Calls: Making requests to our APIs and processing their responses. Examples include:
  • Creating customer accounts
  • Fetching tariff information
  • Setting usage profiles
  • Running savings analyses

Concurrency in a Workflow

Concurrency refers to the system managing and scheduling multiple tasks simultaneously to improve efficiency. Here's how it fits into your specific needs:

• Multiple Tasks: Handle several tasks at the same time, such as creating customer accounts, fetching tariffs, and setting usage profiles.
• Overlapping Tasks: Allow tasks to overlap in time, for example, creating usage profiles while simultaneously generating solar profiles.
• Efficiency: Improves efficiency by utilizing CPU time and other system resources effectively.

Key Concepts

1. Concurrency Limit: The maximum number of simultaneous API calls to avoid overloading the system.
2. Gradual Ramp-Up: Increasing concurrency gradually over 8–10 minutes helps avoid sudden spikes in system load.
3. Monitoring: Continuously monitor system performance to dynamically adjust concurrency settings.

Example Workflow for Acceptable Use

Scenario

A customer runs 1,000 API calls while adhering to:

• A concurrency limit of 4
• A gradual ramp-up of request rates over 8–10 minutes

Steps

1. Initial Setup:

   • Create a few accounts, set tariffs, and create initial usage profiles.
   • Monitor system response times and error rates.

2. Gradual Ramp-Up:

   • Start with a concurrency of 1 and gradually increase it to 4 over 8–10 minutes (e.g., add one concurrent request every 2–3 minutes).

3. Batch Processing:

   • Process profiles and savings analyses in small batches while respecting the concurrency limit.
   • Introduce short delays between batches.

4. Monitoring and Adjustments:

   • Continuously monitor response times and error rates.
   • Adjust the rate of API calls or introduce longer delays as needed.

Example Timeline

• Minute 0–2:

  • Start with 1 concurrent API call.
  • Begin processing the first batch of 50 profiles.

• Minute 2–4:

  • Increase to 2 concurrent API calls.
  • Two API calls are made simultaneously to process the batch.

• Minute 4–6:

  • Increase to 3 concurrent API calls.
  • Three API calls are made simultaneously.

• Minute 6–8:

  • Increase to 4 concurrent API calls.
  • Four API calls are made simultaneously until the batch is processed.

• After Batch Completion:

  • Introduce a short delay before starting the next batch of 50 profiles.
  • Monitor system performance during the delay.

Example Workflow for Unacceptable Use

Scenario

A customer runs 1,000 API calls without following concurrency and ramp-up guidelines.

Issues

1. Immediate High Concurrency:

   • Starts with a high concurrency of 50, exceeding the recommended limit.
   • No gradual ramp-up, causing a sudden spike in system load.

2. No Batch Processing:

   • Attempts to process all profiles and run savings analyses simultaneously.

3. No Monitoring:

   • Ignores system performance metrics, leading to prolonged system strain.

Example Timeline

• Minute 0:
  • Starts with 50 concurrent requests.
  • Attempts to process all 1,000 profiles simultaneously.
  • Ignores increasing response times and error rates.

Summary

By following these best practices:

• Adhere to concurrency limits to prevent system overload.
• Gradually ramp up request rates over 8–10 minutes.
• Monitor system performance continuously and adjust settings as needed.

Failure to follow these guidelines can result in:

• Increased response times
• Higher error rates
• Potential system crashes