Skip to content

[Backend] Comprehensive Multi-Tier Rate Limiting & Abuse Prevention with Redis Token Bucket #1447

Description

@blurbeast

Overview & Background

Currently, the backend's rate limiting implementation is basic and relies on in-memory counters or fragmented express-rate-limit instances. In a production multi-replica deployment, in-memory counters fail because traffic is distributed across multiple pods. A centralized, Redis-backed sliding-window token bucket rate limiter is needed to prevent abuse, scraping, and brute force attacks across public and authenticated routes.


Detailed Problem Statement

  1. Ineffective Multi-Pod Limiting:
    • Memory-based rate limits reset per-instance, allowing an attacker to multiply their allowed requests by the number of running backend replicas.
  2. Missing Rate Tiering:
    • Unauthenticated IP addresses and verified wallet users share the same quota limits.
  3. Missing Standard Headers:
    • Clients do not receive standard Retry-After, X-RateLimit-Limit, and X-RateLimit-Reset response headers to manage backoff properly.

Technical Specification & Architecture

1. Redis Sliding-Window Rate Limiter

Implement backend/src/middleware/rate-limiter.ts using Redis atomic Lua scripts:

  • Sliding-window timestamp sorting: tracks exact timestamps in a Redis sorted set (ZSET).
  • Automatically prunes expired request timestamps on each check.

2. Multi-Tier Configuration

  • Tier 1 (Public / Anonymous): 60 requests/minute per client IP.
  • Tier 2 (Wallet Authenticated): 300 requests/minute per Stellar public key.
  • Tier 3 (Simulation / Write Actions): 20 requests/minute per wallet (protects Soroban RPC from exhaustion).
  • Tier 4 (Admin Endpoints): 100 requests/minute per admin user.

3. Response Standard

When a limit is exceeded:

  • Return HTTP 429 Too Many Requests.
  • Include headers: Retry-After: <seconds>, X-RateLimit-Remaining: 0.
  • Structured JSON response:
{
  "success": false,
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Too many requests. Please retry after 24 seconds.",
    "retryAfter": 24
  }
}

Target Files

  • backend/src/middleware/rate-limiter.ts
  • backend/src/lib/redis.ts
  • backend/src/app.ts
  • backend/tests/rate-limiter.test.ts

Acceptance Criteria

  • Rate limits persist and aggregate across multiple backend processes via Redis.
  • Authenticated wallet requests correctly receive higher tiered quotas than anonymous IPs.
  • Returns HTTP 429 and correct Retry-After header when limits are breached.
  • Unit and load tests verify sliding window accuracy under concurrent bursts.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions