Rate limit
In shortA rate limit caps how many requests or tokens you may send to an AI API in a set time; going over returns an error, usually HTTP 429.
In plain words
Providers protect their servers and share capacity fairly. They do this with rate limits on API use. A limit counts something per minute or per day. Common measures are requests per minute (RPM), tokens per minute (TPM) and requests per day (RPD). Anthropic splits the token limit into input and output tokens per minute. OpenAI names five measures, including images per minute, and says whichever limit you hit first applies.
When you go over, the API returns an error. Both Anthropic and OpenAI use HTTP status 429 for this. Anthropic's answer may include a retry-after header that tells you how many seconds to wait.
Limits grow with your usage tier. At OpenAI, an organisation moves up tiers as its credit purchases grow. At Anthropic, tiers depend on usage history and account standing. Google applies limits per project, not per API key, and says limits differ by model, with tighter ones for experimental and preview models.
Why it matters
- Failed requests: Without handling, a burst of traffic produces errors for your users.
- Design: Use retries with growing pauses (backoff), spread jobs over time, or use the batch API for work that can wait.
- Tokens count too: Long prompts use up a token-per-minute limit fast.
- Caching helps: For most Claude models, Anthropic counts only uncached input tokens toward the input limit, so prompt caching raises your effective throughput.
- Not the same as a spend limit: A rate limit caps speed. A spend limit caps money. Anthropic treats both, and reaching a spend cap pauses use until the next month or until you raise it.
For chat subscriptions, the matching idea is the usage limit.
Example
Your limit is 60 requests per minute. A script sends 200 summary requests in ten seconds. The API accepts the first 60, then returns 429 errors. The fix: send them with a short pause, or retry after the wait the API names. A script that retries instantly in a loop makes things worse.
Common misconceptions
- "A 429 means my key is broken." It means you are going too fast, or a cap was reached. Read the error message.
- "Buying more credit removes limits." Some providers raise limits as spend grows, but not at once.
- "Limits are the same for every model." They differ by model and tier.
Questions and answers
What does HTTP 429 mean?
Too many requests. You went over a rate limit. Wait, as the response header or message says, and retry with growing pauses.
How do I avoid rate limits?
Send fewer or smaller requests per minute, spread the work out, cache repeated prompts, use batches for jobs that can wait, or ask the provider for a higher tier.
What is the difference between a rate limit and a usage limit?
A rate limit caps API speed per minute or day. A usage limit caps how much a chat subscription may be used in a period.
Read next
- Usage limit
A usage limit caps how much you can use an AI chat plan in a set period, such as a session or week, before you must wait or pay more.
- Token
A token is a small chunk of text, often a word piece, that an AI model reads, writes and bills.
- Prompt caching
Prompt caching lets an AI provider reuse the already processed start of your prompt, so repeated parts are cheaper and faster.
- API key
An API key is a secret code that lets a program call an AI service on your account, and the usage is billed to you.
- Batch API
A batch API takes many AI requests at once, processes them later and returns the results at a lower price than live calls.