How-To: Control concurrency and rate limit applications
Typically, in distributed computing, you may only want to allow for a given number of requests to execute concurrently. Using Dapr’s app-max-concurrency, you can control how many requests and events can invoke your application simultaneously.
Default app-max-concurrency is set to -1, meaning no concurrency limit is enforced.
Different approaches
Dapr provides several approaches to concurrency and rate limiting. It’s important to understand the differences:
| Approach | What it controls | Scope |
|---|---|---|
app-max-concurrency | Max concurrent requests and events to an app | Per-sidecar |
middleware.http.ratelimit | HTTP requests per second by remote IP | Per-sidecar |
| Workflow concurrency limits | Workflow and activity executions, with per-name granularity | Per-sidecar or global (across all replicas) |
This guide focuses on app-max-concurrency. See Rate limit middleware and Workflow Concurrency Limits for the other approaches.
Demo
Watch this video on how to control concurrency and rate limiting.
Configure app-max-concurrency
Without using Dapr, you would need to create some sort of a semaphore in the application and take care of acquiring and releasing it.
Using Dapr, you don’t need to make any code changes to your application.
Select how you’d like to configure app-max-concurrency.
To set concurrency limits with the Dapr CLI for running on your local dev machine, add the app-max-concurrency flag:
dapr run --app-max-concurrency 1 --app-port 5000 python ./app.py
The above example effectively turns your app into a sequential processing service.
To configure concurrency limits in Kubernetes, add the following annotation to your pod:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nodesubscriber
namespace: default
labels:
app: nodesubscriber
spec:
replicas: 1
selector:
matchLabels:
app: nodesubscriber
template:
metadata:
labels:
app: nodesubscriber
annotations:
dapr.io/enabled: "true"
dapr.io/app-id: "nodesubscriber"
dapr.io/app-port: "3000"
dapr.io/app-max-concurrency: "1"
#...
Limitations
Controlling concurrency on external requests
Rate limiting is guaranteed for every event coming from Dapr, including pub/sub events, direct invocation from other services, bindings events, etc. However, Dapr can’t enforce the concurrency policy on requests that are coming to your app externally.