Rate limiting for AI agents

What it is

A rate limit caps requests per window: per second, minute, hour or day. RFC 6585 defines status 429, Too Many Requests, for a client that has sent too many in a given time, optionally with a Retry-After header saying how long to wait. For agents there are two limits that matter: how fast the agent may call the platform it uses, and how fast that platform calls each vendor.

Why it matters

Agents work at machine speed, in parallel and sometimes in loops. Vendor quotas are usually per account, not per integration, so an eager agent can spend the quota your billing sync and ticket integration also need. The MCP tools specification says servers must rate limit tool invocations. OWASP lists rate limiting as a way to reduce the damage of excessive agency, though not to prevent it.

How it works

• Declare each vendor's published limits and what they are counted against (account, user, database).
• Count atomically across every instance, so ten workers do not get ten budgets.
• Leave headroom below the published limit for the customer's other integrations.
• Queue briefly, then refuse with 429 and Retry-After when over budget.
• Cap concurrency per connection, so one run cannot open hundreds of sockets.
• Limit inbound traffic per address, per credential and per tenant.

Example

A vendor allows 10,000 requests an hour per account. Two agents and a nightly sync share that account. Without coordination, the agents spend the hour's budget by 9:20, and the sync fails. With pacing at 92% of the limit, counted per account, the agents slow down and the sync keeps its share.

Common pitfalls

• Agents retrying 429s immediately, turning one refusal into a burst.
• Counting per connection when the vendor counts per account.
• Ignoring Retry-After.
• Using rate limits as a budget. Slow and steady can still spend a month's allowance. See spending caps.

How actionlake handles it

Each service file declares the vendor's limits. The gateway counts them per vendor account per window, across all instances, and spends to 92% of the published limit, leaving 8% for your other systems. Over budget, it holds the request briefly, then answers 429 with Retry-After; callers should not retry a 429 themselves, because we are pacing that vendor for everyone. A per-connection concurrency cap stops one plan opening hundreds of sockets. Vendor 429s are not billed. Inbound, separate buckets apply per address, per credential and per tenant, set well above what a busy agent sends. Read more in your rate limits are safe with us.