API Rate Limiting Explained: Strategies and Examples

TL;DR: API rate limiting caps how many requests a client can make in a given window, protecting your service from overload, abuse, and runaway costs. The four algorithms worth knowing are fixed window, sliding window, token bucket, and leaky bucket. For most products, a token bucket keyed per API key or user, enforced at the gateway with clear 429 responses and Retry-After headers, is the pragmatic default.

What is API rate limiting and why does it matter?

API rate limiting is the practice of restricting how many requests a client can send to your API within a defined time window, for example 100 requests per minute per user. When a client exceeds the limit, the server rejects further requests, usually with an HTTP 429 Too Many Requests status, until the window resets.

It matters for four concrete reasons:

• Stability: One misbehaving client or a retry storm can exhaust database connections and take down the service for everyone. Rate limiting contains the blast radius.
• Fairness: On a shared or multi-tenant system, limits stop a single heavy user from starving everyone else. This pairs closely with the isolation concerns in multi-tenant SaaS architecture.
• Cost control: If each request triggers a paid third-party call or expensive compute, uncapped traffic maps directly to an uncapped bill.
• Security: Limits blunt credential-stuffing, scraping, and brute-force attempts by making high-volume abuse impractical.

How do the main rate limiting algorithms work?

Four algorithms cover the vast majority of real-world needs. They trade off accuracy, memory, and burst tolerance differently.

Fixed window

Count requests in a fixed clock interval, for example the current minute. Reset the counter when the next minute starts. It is trivial to implement with a single counter per client, but it has a well-known flaw: a client can send the full quota at the end of one window and again at the start of the next, effectively doubling the intended rate for a short spike.

Sliding window

A sliding window smooths out that edge problem by considering a rolling time range instead of a fixed clock boundary. A common approximation weights the previous window's count by how much of it still overlaps the current time. It is more accurate than a fixed window and only slightly more expensive to compute.

Token bucket

Each client has a bucket that refills with tokens at a steady rate up to a maximum capacity. Every request consumes one token; if the bucket is empty, the request is rejected. Because the bucket can hold a reserve, it naturally allows short bursts while still enforcing a sustained average rate. This is the model most public APIs and gateways use.

Leaky bucket

Requests enter a queue (the bucket) and are processed at a fixed, constant rate, like water leaking from a hole at the bottom. It produces a very smooth, predictable output rate and is useful when a downstream system needs even pacing, but it can add latency and drops requests once the queue is full.

Which rate limiting strategy should you choose?

There is no single winner; the right choice depends on how much burst you tolerate and how precise you need to be. Here is a practical comparison.

Algorithm	Burst handling	Accuracy	Complexity	Best for
Fixed window	Poor (boundary spikes)	Low	Very low	Simple internal limits, quick wins
Sliding window	Good	High	Medium	Fair per-user limits on public APIs
Token bucket	Allows controlled bursts	High	Medium	General-purpose default, gateways
Leaky bucket	Smooths bursts away	High	Medium-high	Protecting a slow downstream at constant rate

For most SaaS and marketplace products, we recommend starting with a token bucket keyed per API key or authenticated user. It handles legitimate bursts (a dashboard loading ten widgets at once) without penalizing users, while still capping sustained abuse. Reach for a leaky bucket only when you specifically need to protect a downstream service that cannot absorb spikes.

Where should you enforce rate limits?

Rate limiting can live at several layers, and mature systems often combine them:

1. Edge / CDN: Coarse limits on raw traffic and obvious abuse, close to the user, before requests hit your infrastructure.
2. API gateway / reverse proxy: The most common place for per-key and per-endpoint limits. Gateways like Nginx, Kong, or a cloud API gateway centralize the logic so your application code stays clean.
3. Application layer: Fine-grained, business-aware limits, for example a lower limit on an expensive report-generation endpoint than on a read endpoint.

A critical detail for anything running more than one server: the counter must be shared across instances. In-memory counters per process silently multiply your intended limit by the number of running instances. Most teams use a central store such as Redis, using atomic operations so concurrent requests cannot bypass the check.

What does a good rate limit response look like?

How you reject requests is as important as when. A well-behaved API tells clients exactly what happened and when to retry:

• Return HTTP status 429 Too Many Requests, not a generic 400 or 500.
• Include a Retry-After header with the number of seconds to wait.
• Expose the limit state with headers such as X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset so clients can throttle themselves proactively.

A minimal example of the response contract:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1719840000

{ "error": "rate_limit_exceeded", "message": "Too many requests. Retry in 30s." }

On the client side, respect Retry-After and use exponential backoff with jitter rather than retrying in a tight loop, which only makes the overload worse. This becomes especially important for automated integrations. If your clients call each other's systems through webhooks, well-designed rate limits and backoff prevent one slow consumer from cascading failures across the pipeline.

What are common mistakes to avoid?

In practice, the failures we see are rarely about the algorithm itself:

• Keying by IP alone: Users behind shared NAT or corporate proxies get lumped together and throttled unfairly. Prefer per-API-key or per-user keys, with IP as a secondary signal.
• Per-instance counters: As noted, local counters do not enforce the global limit once you scale horizontally.
• No burst allowance: Overly rigid limits break legitimate UI patterns that fire several requests at once.
• Silent or confusing rejections: Returning the wrong status code or omitting Retry-After forces client developers to guess.
• Ignoring idempotency: When clients retry after a 429, requests can be duplicated. Pairing rate limiting with idempotent API design ensures a retried write does not create a second charge or duplicate record.

Getting these boundary behaviors right is exactly the kind of detail that separates a robust API from a fragile one. At CodeVix Labs, our QA-first approach means these edge cases, such as retry storms, multi-instance counters, and 429 handling, are tested deliberately rather than discovered in production. You can see how we approach backend engineering across our services, and if you are weighing a build, our team at this software development company in Bangladesh is happy to walk through your architecture. Get in touch to discuss specifics.

Frequently asked questions

What is the difference between rate limiting and throttling?

Rate limiting sets a hard cap and rejects requests over the limit, typically with a 429 response. Throttling is softer: instead of rejecting, it delays or slows requests to keep them under the ceiling. Many systems use the terms loosely, but the practical distinction is reject versus slow down.

What is a good default rate limit for an API?

There is no universal number; it depends on your endpoint cost and typical usage. A common starting point for public read endpoints is in the range of dozens to a few hundred requests per minute per key, with tighter limits on expensive write or compute endpoints. Set an initial value from real traffic estimates, then tune it using observed usage rather than guessing.

Should rate limits differ by pricing tier?

Yes, for most commercial APIs. Tiered limits (higher quotas for paid plans) are a standard way to align cost with value and are straightforward to implement by keying the limit configuration to the account's plan. Keep the enforcement logic identical and only vary the configured numbers.

Does rate limiting replace other abuse protection?

No. Rate limiting is one layer. It should sit alongside authentication, input validation, bot detection, and, for write operations, idempotency keys. Treat it as part of a defense-in-depth strategy, not a complete solution on its own.