Skip to content

Commit 183ed90

Browse files
committed
feat: add Google Gemini provider adapter and configuration
1 parent 2319b63 commit 183ed90

9 files changed

Lines changed: 234 additions & 14 deletions

File tree

‎.env.example‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ RATE_LIMIT_REDIS_FAILURE_POLICY=fail_open
3434

3535
# Upstream LLM Provider API Keys
3636
# At least one provider key is required for handling generation requests
37+
GEMINI_API_KEY=
38+
GEMINI_API_BASE=https://generativelanguage.googleapis.com/v1beta
39+
3740
OPENAI_API_KEY=
3841
OPENAI_API_BASE=https://api.openai.com/v1
3942

‎README.md‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
[![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/Jemade/API-WRAPPER)
44

5-
A production-oriented API gateway and reverse proxy built in Python with FastAPI, PostgreSQL, and Redis that controls, observes, and standardizes traffic between client applications and upstream Large Language Model providers (OpenAI and Anthropic). It provides API-key authentication with hashed credentials, distributed sliding-window rate limiting, exponential backoff retries for transient upstream failures, normalized response structures, HMAC-SHA256-signed webhook delivery, and structured JSON logging with correlation IDs.
5+
A production-oriented API gateway and reverse proxy built in Python with FastAPI, PostgreSQL, and Redis that controls, observes, and standardizes traffic between client applications and upstream Large Language Model providers (Google Gemini, OpenAI, and Anthropic). It provides API-key authentication with hashed credentials, distributed sliding-window rate limiting, exponential backoff retries for transient upstream failures, normalized response structures, HMAC-SHA256-signed webhook delivery, and structured JSON logging with correlation IDs.
66

77
---
88

@@ -11,7 +11,7 @@ A production-oriented API gateway and reverse proxy built in Python with FastAPI
1111
Connecting client applications directly to third-party LLM providers creates serious production risks: client code becomes tightly coupled to vendor-specific SDK formats, rate limits cannot be enforced across multiple consumers, upstream network hiccups or temporary 5xx errors cause user-facing failures, and auditing token usage across teams requires combing through third-party billing dashboards.
1212

1313
I built this gateway as a centralized control plane to solve these problems:
14-
1. **Vendor Decoupling**: Clients consume a single, normalized API format regardless of whether the underlying model is hosted by OpenAI or Anthropic.
14+
1. **Vendor Decoupling**: Clients consume a single, normalized API format regardless of whether the underlying model is hosted by Google Gemini, OpenAI, or Anthropic.
1515
2. **Resiliency**: Transient network drops and upstream 5xx outages are automatically absorbed using exponential backoff retries before reaching the user.
1616
3. **Usage Governance & Security**: Distributed sliding-window rate limiting prevents noisy neighbors or runaway loops from blowing through API budgets, and credentials are stored securely as cryptographic hashes.
1717
4. **End-to-End Observability**: Correlation IDs propagate through every HTTP hop, database audit log, and structured JSON log event.
@@ -34,8 +34,10 @@ flowchart TD
3434
subgraph "Service Layer"
3535
RL --> Val[Pydantic v2 Request Validation]
3636
Val --> Router[Provider Factory Router]
37+
Router -->|gemini-*, gemma-*| GEM[Gemini Adapter]
3738
Router -->|gpt-*, o1-*| OAI[OpenAI Adapter]
3839
Router -->|claude-*| ANTH[Anthropic Adapter]
40+
GEM -->|Tenacity Retry| UpstreamGem[Google Gemini API]
3941
OAI -->|Tenacity Retry| UpstreamOAI[OpenAI API]
4042
ANTH -->|Tenacity Retry| UpstreamAnth[Anthropic API]
4143
end
@@ -52,7 +54,7 @@ flowchart TD
5254

5355
## Features
5456

55-
- **Provider Abstraction**: Unified adapter interface supporting OpenAI (`gpt-4o`, `o1`, etc.) and Anthropic (`claude-3-5-sonnet`, etc.) with normalized response shapes.
57+
- **Provider Abstraction**: Unified adapter interface supporting Google Gemini (`gemini-3.6-flash`, `gemini-flash-latest`, etc.), OpenAI (`gpt-4o`, `o1`, etc.), and Anthropic (`claude-3-5-sonnet`, etc.) with normalized response shapes.
5658
- **API Key Authentication**: Constant-time verification against SHA-256 hashed keys stored in PostgreSQL. Raw keys are never stored.
5759
- **Distributed Rate Limiting**: Atomic Redis sliding-window counter per API key with configurable quotas and documented failure policies (`fail_open` vs `fail_closed`).
5860
- **Resilient Retries**: `tenacity`-powered exponential backoff for transient failures (timeouts, connection resets, 500, 502, 503, 504) while immediately failing on client errors (400, 422).
@@ -91,7 +93,7 @@ curl -i -X POST http://localhost:8080/v1/generate \
9193
-H "X-Request-ID: req_custom_trace_123" \
9294
-H "Content-Type: application/json" \
9395
-d '{
94-
"model": "gpt-4o",
96+
"model": "gemini-3.6-flash",
9597
"messages": [
9698
{"role": "system", "content": "You are a concise engineering tutor."},
9799
{"role": "user", "content": "What is an idempotency key?"}
@@ -110,14 +112,14 @@ curl -i -X POST http://localhost:8080/v1/generate \
110112
```json
111113
{
112114
"request_id": "req_custom_trace_123",
113-
"provider": "openai",
114-
"model": "gpt-4o",
115+
"provider": "gemini",
116+
"model": "gemini-3.6-flash",
115117
"content": "An idempotency key is a unique token generated by a client to ensure that an API request can be safely retried without performing the same operation multiple times, preventing duplicate transactions.",
116118
"input_tokens": 28,
117119
"output_tokens": 36,
118120
"total_tokens": 64,
119121
"finish_reason": "stop",
120-
"latency_ms": 521.43
122+
"latency_ms": 312.43
121123
}
122124
```
123125

@@ -265,10 +267,12 @@ uvicorn app.main:app --host 0.0.0.0 --port 8080 --reload
265267
| Variable | Default | Description |
266268
|---|---|---|
267269
| `APP_ENV` | `development` | Environment: `development`, `staging`, `production` |
268-
| `DATABASE_URL` | `postgresql+asyncpg://...` | SQLAlchemy async connection string |
270+
| `DATABASE_URL` | `sqlite+aiosqlite:///...` | SQLAlchemy async connection string (Postgres or SQLite fallback) |
269271
| `REDIS_URL` | `redis://localhost:6379/0` | Redis connection URL |
270272
| `RATE_LIMIT_PER_MINUTE` | `60` | Default requests per minute per key |
271273
| `RATE_LIMIT_REDIS_FAILURE_POLICY` | `fail_open` | `fail_open` (allow traffic) or `fail_closed` (reject with 429) |
274+
| `GEMINI_API_KEY` | None | Google Gemini API secret key |
275+
| `GEMINI_API_BASE` | `https://generativelanguage.googleapis.com/v1beta` | Google Gemini API base endpoint |
272276
| `OPENAI_API_KEY` | None | OpenAI API secret key |
273277
| `OPENAI_API_BASE` | `https://api.openai.com/v1` | OpenAI API base endpoint |
274278
| `ANTHROPIC_API_KEY` | None | Anthropic API secret key |
@@ -334,7 +338,7 @@ The repository includes a ready-to-use Render Blueprint (`render.yaml`) that pro
334338
`https://render.com/deploy?repo=https://github.com/Jemade/API-WRAPPER`
335339
2. Connect your GitHub account if prompted.
336340
3. In the Blueprint configuration screen:
337-
- Provide your `OPENAI_API_KEY` and/or `ANTHROPIC_API_KEY`.
341+
- Provide your `GEMINI_API_KEY` (and optionally `OPENAI_API_KEY` / `ANTHROPIC_API_KEY`).
338342
- The PostgreSQL database (`gateway-postgres`) and Redis service (`gateway-redis`) are provisioned automatically.
339343
- `WEBHOOK_SECRET` is automatically generated.
340344
4. Click **Apply**. Render builds the Docker container, executes database migrations (`alembic upgrade head`), and starts the gateway service.

‎app/core/config.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,8 @@ class Settings(BaseSettings):
3737

3838
# Database
3939
database_url: str = Field(
40-
default="postgresql+asyncpg://postgres:postgres@localhost:5432/gateway",
41-
description="SQLAlchemy async connection string",
40+
default="sqlite+aiosqlite:///./gateway.db",
41+
description="SQLAlchemy async connection string (PostgreSQL or SQLite fallback)",
4242
)
4343

4444
# Redis
@@ -61,6 +61,9 @@ class Settings(BaseSettings):
6161
)
6262

6363
# Providers
64+
gemini_api_key: str | None = None
65+
gemini_api_base: str = "https://generativelanguage.googleapis.com/v1beta"
66+
6467
openai_api_key: str | None = None
6568
openai_api_base: str = "https://api.openai.com/v1"
6669

‎app/providers/factory.py‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,21 @@
55
from app.core.exceptions import InvalidRequestError
66
from app.providers.anthropic import AnthropicProvider
77
from app.providers.base import LLMProvider
8+
from app.providers.gemini import GeminiProvider
89
from app.providers.openai import OpenAIProvider
910

1011

1112
class ProviderFactory:
1213
"""Resolves and instantiates the appropriate LLM provider adapter based on model name."""
1314

15+
_GEMINI_PREFIXES = ("gemini-", "gemma-")
1416
_OPENAI_PREFIXES = ("gpt-", "o1-", "o3-", "chatgpt-", "text-embedding-")
1517
_ANTHROPIC_PREFIXES = ("claude-",)
1618

1719
def __init__(self, http_client: httpx.AsyncClient | None = None) -> None:
1820
self._http_client = http_client
1921
self._providers: dict[str, LLMProvider] = {
22+
"gemini": GeminiProvider(http_client=self._http_client),
2023
"openai": OpenAIProvider(http_client=self._http_client),
2124
"anthropic": AnthropicProvider(http_client=self._http_client),
2225
}
@@ -25,6 +28,11 @@ def resolve(self, model: str) -> LLMProvider:
2528
"""Inspect model identifier and return the corresponding provider adapter."""
2629
normalized_model = model.lower().strip()
2730

31+
if normalized_model in ("gemini", "default") or any(
32+
normalized_model.startswith(prefix) for prefix in self._GEMINI_PREFIXES
33+
):
34+
return self._providers["gemini"]
35+
2836
if any(normalized_model.startswith(prefix) for prefix in self._OPENAI_PREFIXES):
2937
return self._providers["openai"]
3038

@@ -33,7 +41,7 @@ def resolve(self, model: str) -> LLMProvider:
3341

3442
raise InvalidRequestError(
3543
f"Unsupported model '{model}'. Supported prefixes: "
36-
f"{list(self._OPENAI_PREFIXES + self._ANTHROPIC_PREFIXES)}."
44+
f"{list(self._GEMINI_PREFIXES + self._OPENAI_PREFIXES + self._ANTHROPIC_PREFIXES)}."
3745
)
3846

3947
def get_provider(self, name: str) -> LLMProvider:

‎app/providers/gemini.py‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
"""Google Gemini provider adapter."""
2+
3+
from typing import Any
4+
5+
import httpx
6+
7+
from app.core.exceptions import ProviderConfigurationError
8+
from app.core.logging import get_logger
9+
from app.providers.base import LLMProvider
10+
from app.schemas.generate import ChatRequest
11+
from app.schemas.response import LLMResponse
12+
13+
logger = get_logger("providers.gemini")
14+
15+
16+
class GeminiProvider(LLMProvider):
17+
"""Adapter for Google Gemini generateContent API."""
18+
19+
provider_name: str = "gemini"
20+
21+
def __init__(
22+
self,
23+
http_client: httpx.AsyncClient | None = None,
24+
settings: Any | None = None,
25+
) -> None:
26+
super().__init__(http_client=http_client, settings=settings)
27+
28+
async def _call_provider(self, request: ChatRequest, request_id: str) -> LLMResponse:
29+
api_key = self.settings.gemini_api_key
30+
if not api_key:
31+
raise ProviderConfigurationError("Gemini API key is not configured.")
32+
33+
model_name = request.model.strip()
34+
if model_name == "gemini":
35+
model_name = "gemini-3.6-flash"
36+
if model_name.startswith("models/"):
37+
model_name = model_name[len("models/") :]
38+
39+
url = f"{self.settings.gemini_api_base.rstrip('/')}/models/{model_name}:generateContent"
40+
headers = {
41+
"x-goog-api-key": api_key,
42+
"Content-Type": "application/json",
43+
"User-Agent": "Production-LLM-Gateway/0.1.0",
44+
}
45+
46+
system_instruction: dict[str, Any] | None = None
47+
contents: list[dict[str, Any]] = []
48+
49+
for m in request.messages:
50+
if m.role == "system":
51+
system_instruction = {"parts": [{"text": m.content}]}
52+
elif m.role == "assistant":
53+
contents.append({"role": "model", "parts": [{"text": m.content}]})
54+
else:
55+
contents.append({"role": "user", "parts": [{"text": m.content}]})
56+
57+
generation_config: dict[str, Any] = {}
58+
if request.temperature is not None:
59+
generation_config["temperature"] = request.temperature
60+
if request.max_tokens is not None:
61+
generation_config["maxOutputTokens"] = request.max_tokens
62+
63+
payload: dict[str, Any] = {"contents": contents}
64+
if system_instruction is not None:
65+
payload["systemInstruction"] = system_instruction
66+
if generation_config:
67+
payload["generationConfig"] = generation_config
68+
69+
client = await self.get_client()
70+
response = await client.post(url, json=payload, headers=headers)
71+
72+
if response.status_code != 200:
73+
self._handle_http_error(response.status_code, response.text, request_id)
74+
75+
data = response.json()
76+
candidates = data.get("candidates", [])
77+
if not candidates:
78+
content = ""
79+
finish_reason = "stop"
80+
else:
81+
first_candidate = candidates[0]
82+
content_obj = first_candidate.get("content", {})
83+
parts = content_obj.get("parts", [])
84+
content = "".join(
85+
part.get("text", "") for part in parts if isinstance(part, dict) and "text" in part
86+
)
87+
raw_finish = first_candidate.get("finishReason", "STOP")
88+
if raw_finish in ("STOP", None):
89+
finish_reason = "stop"
90+
elif raw_finish == "MAX_TOKENS":
91+
finish_reason = "length"
92+
elif raw_finish in ("SAFETY", "RECITATION"):
93+
finish_reason = "content_filter"
94+
else:
95+
finish_reason = str(raw_finish).lower()
96+
97+
usage = data.get("usageMetadata", {})
98+
input_tokens = int(usage.get("promptTokenCount", 0))
99+
output_tokens = int(usage.get("candidatesTokenCount", 0))
100+
total_tokens = int(usage.get("totalTokenCount", input_tokens + output_tokens))
101+
102+
return LLMResponse(
103+
request_id=request_id,
104+
provider=self.provider_name,
105+
model=request.model,
106+
content=content,
107+
input_tokens=input_tokens,
108+
output_tokens=output_tokens,
109+
total_tokens=total_tokens,
110+
finish_reason=finish_reason,
111+
latency_ms=0.0,
112+
)

‎app/schemas/generate.py‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,13 @@ class ChatRequest(BaseModel):
2525
model: str = Field(
2626
...,
2727
min_length=1,
28-
description="Target model identifier (e.g. 'gpt-4o', 'claude-3-5-sonnet-20241022').",
29-
examples=["gpt-4o", "claude-3-5-sonnet-20241022"],
28+
description="Target model identifier (e.g. 'gemini-3.6-flash', 'gpt-4o').",
29+
examples=[
30+
"gemini-3.6-flash",
31+
"gemini-flash-latest",
32+
"gpt-4o",
33+
"claude-3-5-sonnet-20241022",
34+
],
3035
)
3136
messages: list[ChatMessage] = Field(
3237
...,

‎docker-compose.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ services:
6161
RATE_LIMIT_PER_MINUTE: "60"
6262
RATE_LIMIT_REDIS_FAILURE_POLICY: fail_open
6363
WEBHOOK_SECRET: change-this-secret-to-a-secure-random-string-in-production
64+
GEMINI_API_KEY: ${GEMINI_API_KEY:-}
6465
OPENAI_API_KEY: ${OPENAI_API_KEY:-}
6566
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
6667
LOG_LEVEL: INFO

‎render.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ services:
2929
type: keyvalue
3030
name: gateway-redis
3131
property: connectionString
32+
- key: GEMINI_API_KEY
33+
sync: false
3234
- key: OPENAI_API_KEY
3335
sync: false
3436
- key: ANTHROPIC_API_KEY

0 commit comments

Comments
 (0)