QueryShield 🛡️

Adaptive Rate Limiting & Intelligent Caching Middleware for High-Performance APIs

A high-performance FastAPI middleware powered by Redis that provides endpoint-aware rate limiting, adaptive threshold-based caching, and dynamic TTL scaling to protect backend databases and dramatically improve API latency under heavy load.

---

🎯 Problem Statement

Modern APIs face several critical challenges:

• 🚨 Traffic spikes and burst attacks can overload databases
• 💰 Expensive queries get re-executed repeatedly — same result, full DB cost every time
• 🐢 Every request hitting the DB increases latency — even for identical queries
• 🔄 Cold endpoints pollute cache — wasting memory on low-traffic paths
• 📉 No adaptive mechanism to dynamically identify and prioritize "hot" endpoints

QueryShield solves all of these by acting as an intelligent middleware layer between your clients and the database — caching smart, throttling hard, and doing both adaptively.

---

✨ Key Features

🚦 1. Endpoint-Aware Rate Limiting

• Token bucket algorithm implemented via Redis + Lua script (atomic, no race conditions)
• Configurable per-endpoint capacity — heavy analytics endpoints get tighter limits than lightweight reads
• IP-based isolation — each client IP gets its own token bucket
• Burst handling with configurable refill rates
• Protection against DDoS-like single-source floods

🧠 2. Adaptive Threshold-Based Caching

• Cache only activates after an endpoint becomes "hot" (crosses a request threshold)
• Prevents cache pollution from cold or rarely-used endpoints
• Automatic promotion of popular query patterns to cached status

📈 3. Sliding Window Popularity Tracking

• True rolling 1-hour window (not fixed-interval — no cliff effects)
• Implemented using Redis sorted sets for accuracy and efficiency
• Real-time hot endpoint detection without clock skew issues

⏱️ 4. Dynamic TTL Scaling

TTL is not a fixed constant — it is computed from:

• Query execution cost (measured DB time)
• Endpoint popularity (hit frequency from sliding window)

This means expensive + popular endpoints stay cached longer, while cheap or rarely-hit endpoints expire quickly — maximizing cache value per memory byte.

---

📊 Performance Highlights

All results from Locust load testing under controlled conditions.

---

🔬 Test 1 — DB Baseline vs. Redis Cache (Latency Comparison)

Direct measurement of DB round-trip vs. Redis cache retrieval per endpoint:

Endpoint	DB Avg (ms)	Redis Avg (ms)	Speedup
Active Users	372	10.4	35.7×
Orders by Date	410	11.2	36.6×
Category Stats	298	9.3	32.0×
Products by Category	212	7.8	27.1×
Above Avg Products	284	8.6	33.0×
User Orders	350	10.1	34.6×

⚡ Average speedup: 33× — Redis responses are 30–36× faster than hitting the database directly.

---

🔥 Test 2 — Single-IP DDoS Simulation (86% Traffic Blocked)

Simulates an aggressive single-source attacker hammering all endpoints simultaneously.

Test Config: 200 concurrent users · 50 spawn rate · 2 min · Single IP

Metric	Value
Total Requests	45,050
429 Rate Limited	38,979 (86.52%)
DB Hits	175 (0.39%)
Cache Hits	5,896 (13.09%)
Cache Hit Ratio	97.12% (of successful requests)
Median Latency	11ms
95th Percentile	130ms
Throughput	250.71 req/s

Per-Endpoint Results:

Endpoint	Requests	429 Rate	Notes
Active Users (30d)	14,782	96.20%	Most aggressive — most blocked
Orders by Date (7d)	10,139	88.90%	Heavy query, high throttle
Category Stats	5,100	89.02%	Analytics endpoint, protected
Above Avg Products	5,036	88.86%	Analytical query, blocked
User Orders	4,982	88.72%	Joins-heavy, protected
Products by Category	5,011	46.10%	Lighter query, less throttled

Key Observations:

• 🛡️ 86% of malicious traffic blocked — rate limiter correctly identified the flood
• 🧠 Database almost completely untouched — only 175 DB hits out of 45,050 requests
• ⚡ Cached responses served in ~11ms median — lightning fast even under attack
• 🚫 Zero 500 errors — system remained stable throughout the DDoS simulation
• 🎯 Endpoint-aware throttling — lighter endpoints (Products by Category) were throttled less aggressively than expensive analytics

---

🌐 Test 3 — Multi-IP Distributed Traffic (Real-World Simulation)

Simulates 10 distributed IPs — represents legitimate distributed users or a coordinated but dispersed traffic pattern.

Test Config: 200 concurrent users · 50 spawn rate · 2 min · 10 IP Pool

Metric	Value
Total Requests	30,047
429 Rate Limited	4,253 (14.15%)
DB Hits	361 (1.20%)
Cache Hits	25,433 (84.65%)
Cache Hit Ratio	98.60% (of successful requests)
Median Latency	120ms
95th Percentile	770ms
Throughput	167.25 req/s

Per-Endpoint Breakdown:

Endpoint	DB Hits	Cache Hits	429 Throttled	Notes
Products by Category	84	3,204	0	Fully cached, no throttle
Orders by Date (7d)	32	6,728	0	Heavy query, well-cached
Category Stats	13	3,242	0	Nearly all from cache
Active Users (30d)	48	5,857	4,253	Only endpoint triggering 429s
User Orders	88	3,195	0	Cache absorbing load
Above Avg Products	96	3,207	0	Threshold-based cache working

Key Observations:

• ✅ 98.8% of requests never touched the database — cache doing its job under real load
• ✅ Only one endpoint triggered 429s — active-users hit its per-endpoint rate cap, others stayed under
• ✅ Zero errors on 5 out of 6 endpoints — clean operation for legitimate distributed traffic
• ✅ System stable at 167 RPS sustained — no degradation over the full test window

---

📉 Single-IP vs. Multi-IP: Side-by-Side

Metric	Single-IP (DDoS)	Multi-IP (Distributed)
Total Requests	45,050	30,047
429 Throttled	86.52%	14.15%
DB Hits	0.39%	1.20%
Cache Hit Ratio	97.12%	98.60%
Median Latency	11ms	120ms
95th Percentile	130ms	770ms

The contrast between tests is exactly what you want to see: a single abusive source is crushed at 86% throttle, while 10 legitimate distributed IPs experience only 14% throttling with near-perfect cache performance.

---

🏗️ Architecture

                ┌───────────────┐
                │    Client     │
                └───────┬───────┘
                        │
                        ▼
        ┌─────────────────────────────────┐
        │       QueryShield Middleware    │
        │                                 │
        │  ┌───────────────────────────┐  │
        │  │  Rate Limiter             │  │
        │  │  - Token bucket (Lua)     │  │
        │  │  - Per-endpoint policy    │  │
        │  │  - IP-based isolation     │  │
        │  └──────────────┬────────────┘  │
        │                 │               │
        │           429?  ▼  Pass         │
        │  ┌───────────────────────────┐  │
        │  │  Threshold Cache Engine   │  │
        │  │  - Sliding window (1hr)   │  │
        │  │  - Adaptive activation    │  │
        │  │  - Dynamic TTL scaling    │  │
        │  └──────────────┬────────────┘  │
        │                 │               │
        │         Cache?  ▼  Miss         │
        │  ┌───────────────────────────┐  │
        │  │  Redis Layer              │  │
        │  │  - Rate limit tokens      │  │
        │  │  - Cached responses       │  │
        │  │  - Popularity sorted sets │  │
        │  └──────────────┬────────────┘  │
        └─────────────────┼───────────────┘
                          │ (cache miss only)
                          ▼
                   ┌─────────────┐
                   │  PostgreSQL │
                   └─────────────┘

Request flow:

1. Request arrives → Rate limiter checks IP + endpoint token bucket
2. If tokens exhausted → return 429 immediately (no DB touch)
3. If tokens available → check sliding window popularity
4. If endpoint is "hot" and cache key exists → return cached response from Redis
5. If cache miss or cold endpoint → query PostgreSQL, store result if hot, return response

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• Redis 7.x running locally or remotely
• PostgreSQL 15+
• pip / virtualenv

Installation

# Clone the repository
git clone https://github.com/yourusername/queryshield.git
cd queryshield

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Configuration

Create a .env file in the project root:

# Database
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/queryshield_db

# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0

# Rate Limiting
RATE_LIMIT_WINDOW_SECONDS=60
RATE_LIMIT_DEFAULT_CAPACITY=10

# Caching
CACHE_HOT_THRESHOLD=5        # requests before endpoint is considered "hot"
CACHE_DEFAULT_TTL_SECONDS=60

# Dynamic TTL Scaling
TTL_COST_WEIGHT=0.6          # weight of query cost in TTL calculation
TTL_POPULARITY_WEIGHT=0.4    # weight of endpoint popularity in TTL calculation
TTL_MAX_SECONDS=300
TTL_MIN_SECONDS=10

Run the Application

uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

The API will be available at http://localhost:8000. Interactive docs at http://localhost:8000/docs.

---

📖 Usage

Making Requests

QueryShield transparently intercepts all requests to configured endpoints:

# Standard request — middleware handles rate limiting + caching automatically
curl http://localhost:8000/endpoints/analytics/active-users?days=30

# Response headers reveal middleware decision
# X-Cache: HIT       → served from Redis cache
# X-Cache: MISS      → forwarded to PostgreSQL
# X-Cache: COLD      → endpoint not yet "hot", no caching
# X-RateLimit-Remaining: 8
# X-RateLimit-Reset: 1708200060

Rate Limit Response

When the rate limit is exceeded:

{
  "status": 429,
  "error": "Rate limit exceeded",
  "message": "Too many requests from this IP for this endpoint.",
  "retry_after_seconds": 45
}

Monitoring Endpoint

curl http://localhost:8000/metrics

{
  "total_requests": 30047,
  "cache_hits": 25433,
  "db_hits": 361,
  "throttled_requests": 4253,
  "cache_hit_ratio": 0.986,
  "hot_endpoints": [
    "/endpoints/orders/by-date",
    "/endpoints/analytics/active-users",
    "/endpoints/analytics/category-stats"
  ]
}

---

🔬 Load Testing

Tests were run with Locust. Test files are in the /load_tests directory.

Single-IP DDoS Simulation

locust -f load_tests/locust_single_ip.py \
  --headless \
  -u 200 \
  -r 50 \
  --run-time 2m \
  --host http://localhost:8000

Multi-IP Distributed Test (10 IP Pool)

locust -f load_tests/locust_multi_ip_10_pool.py \
  --headless \
  -u 200 \
  -r 50 \
  --run-time 2m \
  --host http://localhost:8000

All tests were run on a Dell laptop (local environment). Results are reproducible — test files and DB seed scripts are included in the repo.

---

⚙️ Caching Strategy Deep Dive

Cold Endpoint Behavior

1. First N requests (below threshold) → always hit PostgreSQL
2. No cache entry written
3. Prevents cache memory waste on unpopular paths

Hot Endpoint Promotion

1. Request count crosses threshold within the 1-hour sliding window
2. Endpoint is promoted to "hot" status in Redis
3. Next DB response is cached with dynamically computed TTL
4. Subsequent requests served from Redis until TTL expires

Dynamic TTL Formula

normalized_cost       = query_execution_ms / max_observed_ms
normalized_popularity = endpoint_hits / max_endpoint_hits

ttl = TTL_MIN + (TTL_MAX - TTL_MIN) × (
    cost_weight × normalized_cost +
    popularity_weight × normalized_popularity
)

An endpoint that takes 400ms DB execution time and receives heavy traffic could be cached for up to 5 minutes, while a fast, rarely-hit endpoint might only cache for 10 seconds.

---

🛠️ Tech Stack

Layer	Technology
API Framework	FastAPI
Database	PostgreSQL 15 + async SQLAlchemy
Cache / RL	Redis 7 (sorted sets + Lua scripts)
Load Testing	Locust
Runtime	Python 3.11+ with asyncio

---

🎯 Use Cases

• Public API protection — rate limit by IP before any DB work happens
• SaaS backend shielding — prevent one tenant from starving others
• Analytics-heavy endpoints — cache expensive aggregation queries adaptively
• High-read workloads — serve the majority of reads from Redis at 10ms
• Cost reduction — dramatically cut DB compute costs on cloud-hosted databases

---

🚀 Why QueryShield Is Different

Feature	Traditional Approach	QueryShield
Caching strategy	Cache everything	Cache only hot endpoints
TTL	Fixed constant	Dynamic: cost × popularity
Rate limiting	Global request count	Endpoint-aware token bucket
Popularity tracking	None	Sliding 1-hour Redis sorted set
Cache pollution	Common problem	Threshold-based activation prevents it
Attack vs. legit traffic	Same treatment	Single-IP floods blocked at 86%

---

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

👨‍💻 Author

Your Name

• GitHub: pseudorex
• LinkedIn: asmitgpt

---

📚 References

• Token Bucket Algorithm
• FastAPI Performance Guide
• Locust Load Testing Docs