fix(api): evict only non-rate-limited entries when rate limiter map exceeds capacity

[MinitJain]

Summary

• requestCounts.clear() was called when the in-memory rate limiter map exceeded 10,000 entries after pruning expired ones
• This reset rate limit state for all clients, including those already being blocked — allowing them to immediately bypass the limit
• Fix: instead of clearing everything, evict only entries with count < RATE_LIMIT_MAX_REQUESTS (clients not yet at the limit) until the map is back within bounds
• Clients at or above the rate limit threshold keep their counters

Test plan

• Verify API rate limiting still blocks clients that exceed 60 req/min
• Confirm memory doesn't grow unbounded under normal load (expired entry pruning every 100 requests handles this)

🤖 Generated with Claude Code

Greptile Summary

This PR replaces the unsafe requestCounts.clear() call with targeted eviction that preserves rate-limited client state, correctly preventing rate-limited clients from bypassing the limit after an eviction cycle.

• The new eviction loop only removes entries whose count < RATE_LIMIT_MAX_REQUESTS and stops once the map is back within RATE_LIMIT_MAX_ENTRIES. If all remaining entries are at or above the limit, nothing is evicted and the map stays over capacity — there is no fallback path, so memory can grow unbounded within a single 60-second window under adversarial load.

Confidence Score: 4/5

Safe to merge with awareness of the edge-case memory growth when all entries are rate-limited.

The core security fix is correct and well-reasoned. The one remaining P2 concern — no fallback when targeted eviction leaves the map over capacity — is an edge case that only materialises under adversarial load with >10,000 unique IPs, and entries self-expire after 60 s anyway. Scored 4 rather than 5 to surface that the capacity bound is no longer hard-guaranteed.

apps/web/app/api/utils.ts — eviction fallback path

Important Files Changed

Filename	Overview
apps/web/app/api/utils.ts	Replaces requestCounts.clear() with targeted eviction of non-rate-limited entries; fixes the security bypass but lacks a fallback when all entries are at the rate limit.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Every 100th request] --> B[Prune expired entries\nnow > v.resetAt]
    B --> C{requestCounts.size\n> RATE_LIMIT_MAX_ENTRIES?}
    C -- No --> G[Continue to request handling]
    C -- Yes --> D[Iterate map entries in insertion order]
    D --> E{size <=\nRATE_LIMIT_MAX_ENTRIES?}
    E -- Yes --> G
    E -- No --> F{v.count <\nRATE_LIMIT_MAX_REQUESTS?}
    F -- Yes --> H[Delete entry\nsize--]
    F -- No\nrate-limited client --> I[Skip entry\nsize unchanged]
    H --> E
    I --> J{More entries?}
    J -- Yes --> E
    J -- No\nall entries rate-limited --> K[Loop ends — map may\nstill exceed capacity]
    K --> G

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: apps/web/app/api/utils.ts
Line: 36-41

Comment:
**Eviction may not reduce map when all entries are rate-limited**

If every entry in `requestCounts` has `count >= RATE_LIMIT_MAX_REQUESTS` at the time of eviction (e.g., a burst from many distinct IPs all hitting their limit within the same 60 s window), the loop iterates through the entire map without deleting anything — leaving the map above `RATE_LIMIT_MAX_ENTRIES` indefinitely until the window resets. The old `requestCounts.clear()` was a hard safety valve; the new targeted eviction has no fallback. Consider evicting the entry whose `resetAt` is soonest (i.e., closest to expiry) when the targeted pass fails to bring the map within bounds.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "fix(api): evict only non-rate-limited en..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.