API rate limits cap how many requests you can send in a given window. When you hit one, the server usually answers with HTTP 429 Too Many Requests, and the safest move is to obey the Retry-After header. If that header is missing, fall back to exponential backoff with jitter, cap your retries, and log what the server told you.
Table of Contents
- Why do API rate limits exist, and what do the terms mean?
- How do leaky bucket, token bucket and sliding window limits behave?
- Which HTTP signals and headers tell you that you are being throttled?
- How should your client code handle a 429?
- How do you detect and monitor quota consumption?
- How can you prevent hitting rate limits in the first place?
- How should you plan capacity and test rate-limit behaviour?
- What can you learn from GitHub, Stripe, OpenAI and Shopify?
- Where should you spend your effort when building resilient integrations?
- Need resilient API integrations built for you?
- FAQ
- Sources
Why do API rate limits exist, and what do the terms mean?
Providers set limits to help keep services stable, share capacity between clients, and blunt abuse. The Government of Canada's standards on APIs recommend defined request limits for any API exposed to external parties, and treat throttling as traffic management that stops an API from failing under load. OWASP lists unrestricted resource consumption as an API security risk and points to per-key and per-IP limits as the remedy.
Before you write any client code, get the vocabulary straight, because providers use these words loosely:
- Quota: the total number of requests or tokens you may use, often per minute, day or billing period.
- Window: the span over which usage is counted, either fixed or rolling.
- Restore rate: how quickly spent capacity comes back.
- Concurrency: how many requests may be in flight at the same moment, a separate ceiling from throughput.
- Burst capacity: how far you can briefly exceed the steady rate before you are cut off.
A quota says how much, a window says over what period, and the enforcement algorithm decides how bursts are treated. That last piece is where most surprises come from, so it gets its own section next.
How do leaky bucket, token bucket and sliding window limits behave?
The same published number, say 1,000 requests per minute, can behave quite differently depending on the algorithm behind it.
- Leaky bucket: each request adds to a bucket that drains at a fixed restore rate. A small burst fits if there is room, but the long-run average cannot exceed the drain rate. The leaky bucket model explains why a quiet period buys you headroom and a sustained flood does not.
- Token bucket: tokens refill at a steady rate, and each request spends one or more. A full bucket permits a burst, and an expensive call can cost several tokens. See the token bucket definition for how it differs from the leaky variant.
- Sliding window: the server counts requests across a rolling period. With a sliding window, requests are counted across a rolling period, so bursts may still trip the limit.
- Concurrency limits: these cap simultaneous in-flight requests, so slow responses can exhaust them even when your request rate looks modest.
Pro Tip: Measure both requests per second and in-flight requests in your client. A throttle you cannot explain from throughput alone is usually a concurrency limit.
Which HTTP signals and headers tell you that you are being throttled?
The status code is the easy part. RFC 6585 defines 429 Too Many Requests, recommends an explanatory response body, and allows a Retry-After header to say how long to wait. Servers are not obliged to use 429, though, and may simply drop connections when under attack, so treat dropped connections during heavy load as a possible throttle too.
Retry-After can carry either a number of seconds or an HTTP-date. Microsoft's Retry-After explainer covers both formats; parse both, and read the value as a minimum wait, not a target. If it is absent, use exponential backoff with jitter and cap the retries.
Beyond that, providers vary. Many send de facto X-RateLimit-* headers, while the IETF's RateLimit header draft proposes structured RateLimit and RateLimit-Policy fields that advertise the policy and the remaining quota. They help clients slow down before a 429, but they are not universally implemented.
| Signal | What it tells you | How to use it |
|---|---|---|
| 429 status | You were throttled | Stop and wait, then retry |
| Retry-After | Minimum wait, in seconds or as an HTTP-date | Parse both formats and honour it |
| X-RateLimit-* | De facto remaining or reset values | Use as a hint, not a contract |
| RateLimit, RateLimit-Policy | Draft structured quota fields | Read where offered |
How should your client code handle a 429?
Retries can fix throttling or make it worse. A thundering herd, where many clients retry at the same instant, is exactly how a brief limit becomes an outage, and jitter spreads those retries out. The thundering herd problem is worth reading before you design a retry loop.
A sound handler follows this order:
- Detect the 429 and read the full response body and headers before doing anything else.
- If Retry-After is present, wait at least that long and add a small random jitter.
- If it is missing, use exponential backoff with jitter and a hard ceiling on the delay.
- Cap total attempts, then fail with an error that names the limit, the request ID and the time.
- Check whether your SDK already retries, and disable one layer if so.
That last step matters more than it looks. Microsoft's guidance on OpenAI rate limit handling warns that custom retry logic stacked on top of SDK retries can multiply requests and deepen the throttle. It also says to inspect the error details first: a 429 can mean a temporary rate limit, an exhausted prepaid balance or a hard spending limit, and only the first of those is worth retrying.
Pro Tip: Prefer Retry-After over any reset time you infer from non-standard headers, since header formats differ between providers.
How do you detect and monitor quota consumption?
You want to see a limit approaching before customers see an error. When the provider sends RateLimit headers, read and store them per key so you always know the last reported remaining quota. Where they are not sent, count your own requests per key and window.
Track a small set of metrics and alert on them:
- 429_rate: the share of responses that are throttled, alerting on sudden spikes.
- quota_utilization_p95: how close the busiest periods run to the ceiling.
- throttled_clients_unique: how many distinct keys or tenants are being limited at once.
- Error payloads: log error.code, request ID, timestamp and the stated limit for every throttle, so provider support can trace it.
API Commons recommends documenting 429 responses and RateLimit headers in OpenAPI, exposing quota counters in headers, and instrumenting exactly these kinds of metrics.
Structured error bodies make the logging easier. The GC Notify documentation shows a 429 response with a body of the form {"error":"RateLimitError","message":"Exceeded rate limit for key type TEAM/TEST/LIVE of 1000 requests per 60 seconds"}, which names both the key type and the breached limit.
A clear 429 body is half the fix: the message names the key type and the exact limit that was breached.
How can you prevent hitting rate limits in the first place?
Most throttling is avoidable with a few design habits on the client and a few on the server.
On the client:
- Cache responses that do not change often, so repeat reads never reach the API.
- Coalesce and batch identical or related requests into one call where the API allows it.
- Throttle yourself with a client-side token bucket set a little under the published limit.
- Ramp traffic up gradually after a deploy or a restart instead of releasing a backlog all at once.
- Move heavy work into background jobs and use async or batch endpoints where they exist.
On the server, if you run the API:
- Put limits at the gateway. Canada.ca's API gateway guidance says externally exposed APIs should use gateways and throttling, and notes that common reverse proxies such as NGINX support it.
- Limit per key, per IP and per tenant, so one noisy caller cannot starve the rest.
- Degrade gracefully. Return a clear 429 with Retry-After instead of silently dropping connections.
The API scoping primer adds that throttling should not replace effective scaling; it protects availability while you scale.
How should you plan capacity and test rate-limit behaviour?
Limits only help if they are set from evidence. The same primer recommends load testing to at least double your anticipated peak usage, and the Government of Canada standards advise building automated performance testing into your CI/CD pipeline so regressions show up before release.
A workable routine:
- Record baseline traffic and latency so you know what normal looks like.
- Load test to at least 2x the peak you expect, and watch where behaviour changes.
- Test throughput and concurrency separately, since each has its own breaking point.
- Use staging keys and simulated Retry-After responses to prove your backoff logic works.
- Ask the provider for a limit increase well ahead of any planned high-traffic event.
If you self-host any part of the stack, headroom matters at the infrastructure level too. A practical piece on scaling VPS resources is useful background when you are sizing servers for a load test.
What can you learn from GitHub, Stripe, OpenAI and Shopify?
Each major provider documents its limits a little differently, and their patterns are worth copying into your own client library.
- GitHub: check the provider’s documentation for how its rate limits and response headers work.
- Stripe: per-account global limits apply, for example 100 requests per second in live mode, and individual endpoints carry their own separate limits. Concurrency is documented separately, and a Stripe-Rate-Limited-Reason header explains why a request was limited. See Stripe's rate limit documentation for the full detail, including advice on exponential backoff with jitter and batching.
- OpenAI: a 429 can signal billing or spend problems, so read the error details before retrying. Its SDKs may retry automatically.
- Shopify: check the provider’s documentation for how its rate limits work.
The common thread is that the answer is in the headers and the error body, not in guesswork.
Where should you spend your effort when building resilient integrations?
My view is that two habits do most of the work: obey Retry-After every time, and monitor 429 rates well enough that a spike pages someone before a customer notices. Load testing in CI catches the rest. Teams often over-invest in clever retry maths and under-invest in visibility. If your engineering bandwidth is thin, centralizing throttling and retries in one layer, built with a partner, is usually cheaper than patching every service separately.
— Harry Gill
Need resilient API integrations built for you?
If you would rather not hand-roll retries, throttling and monitoring for every service, we build custom integrations that centralize API traffic control in one place, with retries, throttling and monitoring built in. That helps most when you are orchestrating several APIs and your team is short on time, since one well-built layer lowers operational risk across all of them.
Our AI Workflow Automation work covers exactly this kind of resilient orchestration, and you can see how consolidation pays off in our automation roadmap. Tell us what you are integrating and we will talk through a fixed-price approach.
FAQ
How can I fix an "API rate limit exceeded" error?
Stop sending requests, then wait at least as long as the Retry-After header says. If the header is missing, retry with exponential backoff and jitter, and cap the number of attempts. Check the error body too, since some 429s signal billing limits, not temporary throttling.
How do I avoid hitting an API rate limit?
Cache repeat reads, batch related calls, and throttle your own client below the published limit. Ramp traffic up slowly after deploys. The Government of Canada API standards recommend defined request limits and throttling as traffic management.
How do I check my current API rate limit?
Read the rate limit headers on each response, such as X-RateLimit-* or the draft RateLimit fields, where the provider sends them. Otherwise consult the provider's limits page or your account dashboard. Store the values per key so you can alert before you run out.
What do "API rates" mean?
The term usually refers to the allowed number of requests, or tokens, in a set period, such as per minute. It can also mean the pace at which a client actually calls the API. Providers may set separate limits for throughput and for concurrent requests.
Sources
- Canada
- The Retry-After header: how long to wait before you retry - Dev Proxy | Microsoft Learn
- Rfc-editor