Rate limiting controls how systems allocate finite capacity across users, services, and regions. Each rate limiting algorithm makes a different trade-off between time, fairness, memory cost, and burst tolerance.
Choosing the wrong algorithm can create boundary exploits, uneven load, or distributed consistency failures. This guide explains how the major algorithms work, where they fail in practice, and how to choose one for an API.
Rate limiting algorithms at a glance
| Algorithm | Fairness | Burst tolerance | Memory cost | Best for |
|---|---|---|---|---|
| Fixed window | Low | High at window boundaries | Very low | Simple internal limits |
| Sliding window log | High | Low | High | Strict fairness enforcement |
| Sliding window counter | Medium to high | Low to medium | Moderate | Scalable public APIs |
| Token bucket | Medium to high | Controlled bursts | Low | Developer-facing APIs |
| Leaky bucket | High output smoothing | None in the queue-based model | Low | Traffic shaping |
Key conclusions:
- Fixed window is simple but allows boundary burst amplification.
- Sliding log is the most accurate but can become memory intensive.
- Sliding counter is a practical compromise for scale.
- Token bucket is a useful starting point when an API should allow controlled bursts.
- Leaky bucket is best for shaping outbound or downstream traffic.
A useful mental model is that token bucket models capacity accumulation, sliding window models recent history, and fixed window models discrete accounting periods.
What is rate limiting?
Rate limiting constrains how frequently an identity can perform an action within a defined time interval. The identity may be an IP address, API key, user ID, tenant ID, or service.
At a systems level, every rate limiter defines:
- how time is measured;
- how usage is recorded; and
- what happens when capacity is exhausted.
Rate limiting does more than block traffic. It encodes a policy for fairness, abuse resistance, and infrastructure protection.
What is the fixed window algorithm?
The fixed window algorithm divides time into discrete intervals such as 60 seconds. For each identity, a counter tracks how many requests occur within the current interval. When the counter exceeds the configured limit, additional requests are rejected until the next window begins.
Fixed window example
A limit of 100 requests per minute means:
- Requests between 12:00:00 and 12:00:59 increment one counter.
- At 12:01:00, the counter resets to zero.
This is computationally efficient and requires only a counter and a window identifier. However, it introduces boundary amplification: a client can send 100 requests at 12:00:59 and another 100 at 12:01:00. The system technically enforces 100 requests per minute, but 200 requests arrive within two seconds.
When fixed window fails in practice
- A login endpoint can be brute-forced at window boundaries.
- Public APIs can experience load spikes at reset times.
- Multi-tenant systems can see unfair traffic distribution near window edges.
Fixed window is appropriate when implementation simplicity matters and boundary bursts are acceptable. Avoid it where those bursts could overload a resource or weaken an abuse control.
What is the sliding window algorithm?
The sliding window algorithm evaluates requests against a rolling time interval instead of discrete blocks. There are two primary implementations: sliding window log and sliding window counter.
Sliding window log
The system stores a timestamp for every request. When a new request arrives, timestamps older than the window duration are removed and the remaining entries are counted. At any moment, the system evaluates exactly the last N seconds of traffic, which provides strong fairness.
Memory usage grows with request volume, however. During traffic spikes, log-based implementations can create memory pressure and increased CPU overhead for pruning. A high-throughput endpoint using sliding logs can experience memory amplification during an attack spike.
Sliding window counter
Instead of storing every timestamp, a sliding window counter blends counters from the current and previous window based on elapsed time. It approximates sliding behavior with far lower memory cost. Accuracy is slightly reduced, but boundary amplification is dramatically minimized.
For large-scale APIs, sliding counter is often the scalable alternative to sliding log.
What is the token bucket algorithm?
The token bucket algorithm models capacity as tokens accumulating over time. Each identity has:
- a maximum bucket capacity;
- a refill rate; and
- a current token count.
Requests consume tokens. If no tokens remain, requests are rejected. If a client is idle, tokens accumulate up to the bucket's maximum capacity. This allows controlled bursts without violating the long-term average limit.
For many developer-facing APIs, token bucket is a useful starting point. It enforces a steady average rate while allowing short bursts, which accommodates clients that send requests in batches.
Where token bucket fails
- Extremely strict fairness requirements where recent history must be exact.
- Situations where output must be perfectly smoothed rather than burst-tolerant.
What is the leaky bucket algorithm?
The queue-based leaky-bucket model enforces a constant output rate. Incoming requests enter a queue that drains at a steady pace. If the queue is full, new requests are rejected. Unlike token bucket, this model does not accumulate burst capacity; it smooths traffic strictly. Some systems use “leaky bucket” for a meter-based variant with different burst behavior, so verify the implementation rather than relying on the name alone.
This is useful for:
- protecting downstream services;
- traffic shaping; and
- preventing retry storms from overwhelming dependencies.
Distributed rate limiting and global coordination
Distributed rate limiting introduces coordination complexity. If multiple instances serve traffic for the same identity, they must share state. Without coordination, each node may independently allow traffic, multiplying the effective limit.
Strong vs eventual consistency
Strong consistency ensures accurate limits but increases latency and reduces availability. Eventual consistency improves resilience but allows temporary overages. In multi-region deployments, global rate limiting across regions requires explicit tradeoffs. During a regional partition, systems may double-allow traffic if coordination fails.
Common failure modes
- Redis hot keys under high traffic.
- Network latency increasing enforcement delays.
- Clock skew affecting sliding window accuracy.
- Region isolation allowing temporary limit bypass.
Distributed rate limiting is fundamentally a consistency problem. Algorithm choice matters, but state coordination dominates complexity at scale.
Rate limiting in microservices architectures
In microservices architectures, rate limiting must be placed deliberately. Limits at the edge or API gateway protect external exposure and enforce per-identity fairness. Internal limits serve a different purpose: they protect downstream dependencies and reduce the risk of cascading failures.
Problems arise when retries are not coordinated with enforcement. If a downstream service begins failing and upstream services retry aggressively, total traffic can exceed the original request volume. In fan-out systems, where one request triggers multiple internal calls, this effect multiplies quickly.
Effective rate limiting in microservices requires aligning enforcement with retry behavior and service topology. Otherwise, it can amplify instability instead of containing it.
What is the best rate limiting algorithm for APIs?
For many public APIs, the token bucket algorithm is a useful starting point. It enforces a predictable long-term rate while allowing controlled bursts, which makes it well suited to developer-facing platforms whose clients send requests in batches.
If strict fairness is the primary requirement, particularly in security-sensitive environments, a sliding window log provides the highest accuracy because it evaluates exact recent request history. That precision comes with higher memory and operational cost.
For large-scale distributed systems where memory usage and coordination overhead matter, a sliding window counter is often the most practical compromise. It reduces boundary burst effects while remaining efficient enough to operate at high throughput.
Fixed window is suitable when simplicity matters and boundary bursts cannot exhaust the protected resource. Leaky bucket is better suited to traffic shaping and steady output control than to user-facing API fairness.
Choose token bucket for controlled bursts, sliding window for recent-history fairness, fixed window for simple accounting where boundary bursts are safe, and leaky bucket when downstream output must remain steady.
Choose the right identifier
The identifier determines who shares a limit. IP addresses work before authentication, but carrier networks and offices can place many users behind one address. Attackers can also rotate proxies. Authenticated APIs should usually add limits for user, account, API key, or tenant identifiers.
Apply more than one limit when risks differ. A service might enforce a generous tenant-wide quota, a smaller per-user limit, and a strict per-IP limit for failed authentication. Do not use raw email addresses or other personal data as identifiers when an opaque internal ID is available.
Trust client IP headers only when they were added by a known proxy. Otherwise, an attacker can spoof a new address on every request.
Rate limiting is one layer of a broader API security strategy, and identity-aware limits are especially important when defending against distributed API abuse.
Return useful 429 responses
When a request exceeds an enforced limit, return HTTP 429 Too Many Requests, as defined by RFC 6585. Keep the response body machine-readable and consistent. Tell the client when it may retry using Retry-After where practical. The proposed RateLimit and RateLimit-Policy fields can communicate current capacity and quota policy, but they remain an IETF Internet-Draft and may change before publication as an RFC.
Clients should use exponential backoff with jitter and avoid retrying an operation that is no longer useful. Servers should not return 429 for unrelated authorization or validation failures; accurate status codes make client behavior and monitoring more reliable.
Rate limiting implementation checklist
- Identify the constrained resource and the consequence of exhaustion.
- Select stable characteristics that attackers cannot freely choose.
- Set independent limits for operations with very different costs.
- Account for retries, idempotency, background jobs, and trusted automation.
- Start in dry-run mode and measure which users would be limited.
- Return 429 with actionable retry information.
- Log the policy, characteristic, remaining capacity, and reset time.
- Alert on sudden changes in limited traffic and on limiter failures.
- Test exact boundaries, concurrent requests, clock behavior, and store outages.
- Review limits as traffic patterns and product plans change.
Frequently asked questions
Which rate limiting algorithm should I use?
Use a fixed window for simple quotas, a sliding window when boundary bursts would be harmful, and a token bucket when controlled bursts or variable request costs are useful. The correct choice depends on the resource and user experience you need to protect.
Should APIs rate limit by IP address?
IP-based limits are a useful anonymous fallback, but should not be the only identifier. Shared networks can put many users behind one IP, while attackers can rotate addresses. Prefer stable user, account, API key, or tenant identifiers when available.
What HTTP status code should a rate-limited API return?
Return HTTP 429 Too Many Requests. Include enough information for a client to know when to retry, and keep the error body stable and machine-readable.
Application security in your code
Protect your application with Arcjet
Arcjet runs inside your application, where it can use request and identity context to enforce rate limits, detect bots, and block common attacks.