diff --git a/.gitignore b/.gitignore index 76f863b..7415c77 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,11 @@ yarn-debug.log* yarn-error.log* .pnpm-debug.log* +# Python +.venv/ +__pycache__/ +*.pyc + # Runtime / logs *.log logs/ diff --git a/README.md b/README.md index 68b99de..9e927a3 100644 --- a/README.md +++ b/README.md @@ -16,9 +16,10 @@ playground's README states what it needs. | Playground | Language / Stack | What it shows | | --------------------------------- | ---------------------- | ------------------------------------------------------------------------- | | [mongoose](playgrounds/mongoose/) | Node.js — Mongoose ODM | Express REST API + a CRUD/compatibility test suite using the Mongoose ODM. | +| [beanie](playgrounds/beanie/) | Python — Beanie ODM | FastAPI REST API + a CRUD/compatibility test suite using the Beanie ODM. | -More playgrounds are planned (for example **PyMongo**, **Beanie**, and other -MongoDB drivers). Contributions are welcome. +More playgrounds are planned (for example **PyMongo** and other MongoDB +drivers). Contributions are welcome. ## Getting Started @@ -36,6 +37,14 @@ cd playgrounds/mongoose ./scripts/run-app.sh # or run the demo REST API ``` +To try the Beanie playground: + +```bash +cd playgrounds/beanie +./scripts/run-test.sh # start DocumentDB locally and run the compatibility suite +./scripts/run-app.sh # or run the demo REST API +``` + ## Repository Layout ``` @@ -43,7 +52,8 @@ documentdb-playground/ ├── README.md ├── LICENSE └── playgrounds/ - └── mongoose/ # Node.js + Mongoose ODM + ├── mongoose/ # Node.js + Mongoose ODM + └── beanie/ # Python + Beanie ODM ``` ## License diff --git a/playgrounds/beanie/README.md b/playgrounds/beanie/README.md new file mode 100644 index 0000000..023eba3 --- /dev/null +++ b/playgrounds/beanie/README.md @@ -0,0 +1,302 @@ +# Beanie with DocumentDB (local) + +This playground shows how to use [Beanie](https://beanie-odm.dev/), a popular +**asynchronous Python ODM** built on [Motor](https://motor.readthedocs.io/) +(async PyMongo) and [Pydantic](https://docs.pydantic.dev/), against DocumentDB — +running **entirely on your machine**. It includes: + +- a small **FastAPI + Beanie REST API** (`app/`), and +- a standalone **Beanie CRUD/compatibility test suite** + (`app/beanie_crud_test.py`) that exercises connect, index creation, insert, + query, update, aggregation, unique-index enforcement, and delete. + +There is **no Kubernetes and no cloud**. DocumentDB runs as the +[`documentdb-local`](https://github.com/documentdb/documentdb) emulator in a +single Docker container, and the app/test run as local Python processes that +connect straight to it. + +> **What is Beanie?** Beanie is a **Python library** (an async ODM, Object +> Document Mapper), not a CLI tool or a server. Your application imports it to +> define `Document` models (Pydantic classes) and talk to a MongoDB-compatible +> database over Motor. Here it is used by the demo **app** +> ([`app/main.py`](app/main.py)) and the standalone **test script** +> ([`app/beanie_crud_test.py`](app/beanie_crud_test.py)). + +## Architecture + +Everything is local. The emulator container exposes the MongoDB wire protocol on +`localhost:10260`; the Python processes connect to it directly. + +``` + Your machine (WSL / Linux / macOS) +┌──────────────────────────────────────────────────────────────────┐ +│ ┌────────────────────┐ ┌──────────────────────────────┐ │ +│ │ beanie app / │ TLS, │ documentdb-local (Docker) │ │ +│ │ test script │ wire │ ┌────────────┐ ┌─────────┐ │ │ +│ │ (Python + Beanie) │────────▶│ │ Gateway │▶│Postgres │ │ │ +│ │ │ :10260 │ │ (10260) │ │ (engine)│ │ │ +│ └────────────────────┘ │ └────────────┘ └─────────┘ │ │ +│ └──────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────┘ +``` + +Beanie talks to the emulator through Motor exactly as it would to a standalone +`mongod`, with a few required options (see [Connecting Beanie to +DocumentDB](#connecting-beanie-to-documentdb)). + +## Prerequisites + +- **Docker** (to run the `documentdb-local` emulator) +- **Python 3.10+** with `venv` (to run the app and test suite) + +The scripts create a Python virtualenv and install dependencies for you. On +Windows, run these from a **WSL** shell. + +## Quick Start + +From this directory (`playgrounds/beanie/`). `run-test.sh` and `run-app.sh` +are **two independent operations** — each starts DocumentDB on its own if it +isn't already running. + +### Option A — run the test suite + +```bash +# Run the full CRUD/compatibility suite end-to-end. +# Starts DocumentDB in Docker (first run pulls the image), then runs the tests. +./scripts/run-test.sh +``` + +### Option B — run the demo REST API + +```bash +# Starts DocumentDB (if not already running) and serves the API on :3000. +# This stays in the foreground until you press Ctrl-C. +./scripts/run-app.sh +``` + +Stop the database when you are done: + +```bash +./scripts/stop-documentdb.sh +``` + +The suite should end with `Passed: 13 Failed: 0`. + +## Trying the API + +With `./scripts/run-app.sh` running, the API is on `http://localhost:3000` +(interactive docs at `http://localhost:3000/docs`): + +```bash +# Health +curl -s http://localhost:3000/health +# {"status":"healthy","db":"connected"} + +# Create a book +curl -s -X POST http://localhost:3000/books \ + -H 'Content-Type: application/json' \ + -d '{"title":"Dune","author":"Herbert","genres":["sci-fi"],"pages":412,"rating":5}' + +# List books +curl -s http://localhost:3000/books | jq . + +# Count books per genre (aggregation) +curl -s http://localhost:3000/stats/genres | jq . +``` + +## Connecting Beanie to DocumentDB + +The DocumentDB gateway speaks the MongoDB wire protocol but advertises itself as +a **standalone** server over **TLS** (with a self-signed cert). Beanie talks to +it through Motor, which therefore needs these options (see [`app/db.py`](app/db.py)): + +```python +from motor.motor_asyncio import AsyncIOMotorClient +from beanie import init_beanie + +client = AsyncIOMotorClient( + uri, + directConnection=True, # gateway is standalone, not a replica set + tls=True, # gateway only accepts TLS + tlsAllowInvalidCertificates=True, # emulator uses a self-signed cert +) +await init_beanie(database=client[db_name], document_models=[Book]) +``` + +The connection string built by the scripts is: + +``` +mongodb://:@localhost:10260/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true +``` + +For production against a real (non-emulator) deployment, set `TLS_INSECURE=false` +and pass a CA bundle via `tlsCAFile` instead of `tlsAllowInvalidCertificates`. + +## Configuration Reference + +All settings are passed via environment variables; there is no config file. + +### Emulator + scripts (`scripts/`) + +Read by [`lib.sh`](scripts/lib.sh) and the `start`/`stop`/`run` scripts. + +| Variable | Default | Description | +| ---------------------- | -------------------------------------------------------- | -------------------------------------------------------- | +| `DOCUMENTDB_IMAGE` | `ghcr.io/documentdb/documentdb/documentdb-local:latest` | Emulator image to pull/run. | +| `DOCUMENTDB_CONTAINER` | `documentdb-local` | Docker container name. | +| `DOCUMENTDB_HOST` | `localhost` | Host the app/test connect to. | +| `DOCUMENTDB_PORT` | `10260` | Host port mapped to the gateway. | +| `DOCUMENTDB_USERNAME` | `docdbadmin` | Emulator admin username. **Do not use `documentdb`** (reserved — the gateway rejects it as "Username is invalid"). | +| `DOCUMENTDB_PASSWORD` | `Documentdb!Local1` | Emulator admin password. If you use special characters, URL-encode them in the connection string. | +| `PORT` | `3000` | Local port the FastAPI app listens on (`run-app.sh`). | + +### App + test script (`app/`) + +Read by [`app/db.py`](app/db.py), [`app/main.py`](app/main.py), and +[`app/beanie_crud_test.py`](app/beanie_crud_test.py). The scripts set `MONGO_URI` +for you from the variables above. + +| Variable | Default | Description | +| ----------------------------- | ---------------- | -------------------------------------------------------------------------------------------- | +| `MONGO_URI` | _(set by scripts)_ | DocumentDB connection string. The test script also accepts it as the first CLI argument. | +| `MONGO_DB` | `beanie_demo` (app), `beanie_test` (test) | Database name Beanie connects to. | +| `TLS_INSECURE` | `true` | When `true`, accepts the self-signed cert. Set `false` for CA-verified TLS. | +| `SERVER_SELECTION_TIMEOUT_MS` | `10000` | How long Motor waits to select a server before erroring. | +| `PORT` | `3000` | Port the FastAPI/Uvicorn API listens on. | + +## DocumentDB Compatibility Notes + +Verified against `documentdb-local:latest` (release `0.114`): + +| Beanie feature | Status | Notes | +| --------------------------------------- | ------------- | --------------------------------------------------------------------- | +| CRUD (`insert`/`find`/`save`/`delete`) | ✅ Supported | Standard document operations work as expected. | +| `get(id)` / `_id` point lookups | ✅ Supported | Works on `0.114`. (Older gateway `0.109.0` failed these with "trying to open a pruned relation".) | +| Index creation via `Settings.indexes` | ✅ Supported | Built asynchronously by the engine; `createIndexes` returns in ~2s. Avoid `collation`. | +| Unique indexes | ✅ Supported | Duplicate keys raise `DuplicateKeyError` (code `11000`). | +| Aggregation pipelines | ✅ Common stages | `$match`, `$group`, `$unwind`, `$sort`, etc. Atlas-only stages differ. | +| `$vectorSearch` | ❌ Not supported | Atlas-only operator. | +| Index `collation` | ❌ Not supported | `createIndex.collation is not implemented yet`; omit it. | +| Change streams / transactions | ⚠️ Check version | Verify against your DocumentDB version before relying on them. | + +The CRUD test suite ([`app/beanie_crud_test.py`](app/beanie_crud_test.py)) +covers the supported rows above and prints a pass/fail summary. + +## Running the Test Suite Manually + +`scripts/run-test.sh` sets `MONGO_URI` and runs the suite for you. To run it +directly against any reachable connection string: + +```bash +cd app +python3 -m venv .venv +.venv/bin/pip install -r requirements.txt +MONGO_URI="mongodb://docdbadmin:Documentdb!Local1@localhost:10260/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true" \ + .venv/bin/python beanie_crud_test.py +``` + +Expected output: + +``` +Beanie DocumentDB compatibility test +==================================== + ✅ connect + ✅ create indexes + ✅ insert_one (Document.insert) + ✅ insert_many + ✅ get by _id (Document.get) + ✅ find with filter + sort + limit + ✅ count_documents + ✅ update_one ($set) + ✅ find_one_and_update (returns new) + ✅ aggregation ($unwind/$group) + ✅ unique index enforcement (duplicate sku rejected) + ✅ delete_one + ✅ cleanup (drop collection) +==================================== +Passed: 13 Failed: 0 +``` + +## What the Scripts Do + +| Script | Purpose | +| ----------------------------- | ---------------------------------------------------------------------------------------- | +| `scripts/start-documentdb.sh` | Start the local emulator container and wait until the gateway is ready. | +| `scripts/run-test.sh` | Start DocumentDB (if needed), set up the venv, and run the Beanie CRUD/compatibility suite. | +| `scripts/run-app.sh` | Start DocumentDB (if needed), set up the venv, and run the FastAPI + Beanie demo app. | +| `scripts/stop-documentdb.sh` | Stop and remove the emulator container (full reset of its data). | +| `scripts/lib.sh` | Shared helpers: container lifecycle, readiness wait, connection-string builder, and venv setup. | + +## Verification + +- `./scripts/run-test.sh` ends with `Passed: 13 Failed: 0`. +- With `./scripts/run-app.sh` running, `curl http://localhost:3000/health` + returns `{"status":"healthy","db":"connected"}`, `POST /books` returns `201` + with the created document, and `GET /stats/genres` returns per-genre counts. + +## Cleanup + +- **App / test:** press `Ctrl-C` to stop the app; the test exits on its own. + Optionally remove the virtualenv: `rm -rf app/.venv`. +- **Emulator:** `./scripts/stop-documentdb.sh` removes the container and all its + data. + +## Troubleshooting + +### `AuthenticationFailed: Username is invalid.` + +The emulator rejects certain reserved usernames — notably `documentdb`. Use a +different admin username (the default here is `docdbadmin`). If you changed +`DOCUMENTDB_USERNAME`, recreate the container so the new credentials take effect: + +```bash +./scripts/stop-documentdb.sh && ./scripts/start-documentdb.sh +``` + +### `ServerSelectionTimeoutError` / TLS handshake failures + +The gateway requires TLS. Confirm the connection string includes `tls=true` and +`tlsAllowInvalidCertificates=true` (the scripts add these). Make sure the +emulator is running: `docker ps` should list `documentdb-local`, and +`docker logs documentdb-local` should show the gateway accepting connections. + +### Port `10260` already in use + +Another process (or a previous emulator) holds the port. Stop it, or run on a +different port: + +```bash +DOCUMENTDB_PORT=10261 ./scripts/start-documentdb.sh +DOCUMENTDB_PORT=10261 ./scripts/run-test.sh +``` + +### Credentials changed but auth still fails + +The username/password are baked into the container at creation time. Changing +`DOCUMENTDB_USERNAME`/`DOCUMENTDB_PASSWORD` only takes effect after you recreate +the container (`stop-documentdb.sh` then `start-documentdb.sh`). + +### `createIndex.collation is not implemented yet` + +A model index uses `collation`. Remove it; DocumentDB does not support collation +indexes. The models in this playground intentionally avoid it. + +## Directory Layout + +``` +beanie/ +├── README.md +├── app/ +│ ├── requirements.txt +│ ├── db.py # Motor connection + init_beanie (DocumentDB options) +│ ├── main.py # FastAPI REST API (/books, /health, /stats) +│ ├── models/ +│ │ └── book.py # Example Beanie Document model +│ └── beanie_crud_test.py # Standalone CRUD/compatibility test suite +└── scripts/ + ├── lib.sh # Connection-string builder + readiness wait + venv setup + ├── start-documentdb.sh # Start the local emulator (Docker) + ├── run-app.sh # Run the demo app locally + ├── run-test.sh # Run the test suite locally + └── stop-documentdb.sh # Stop + remove the emulator +``` diff --git a/playgrounds/beanie/app/beanie_crud_test.py b/playgrounds/beanie/app/beanie_crud_test.py new file mode 100644 index 0000000..4fb1683 --- /dev/null +++ b/playgrounds/beanie/app/beanie_crud_test.py @@ -0,0 +1,243 @@ +"""Beanie CRUD/compatibility test against DocumentDB. + +Usage: + MONGO_URI="mongodb://user:pass@host:10260/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true" \ + python beanie_crud_test.py + +Or pass the URI as the first argument: + python beanie_crud_test.py "mongodb://user:pass@host:10260/?..." + +Exercises connect, index creation, insert, find, update, aggregation, +unique-index enforcement, and delete using Beanie/Motor against the DocumentDB +gateway. Exits non-zero on the first real failure. +""" + +from __future__ import annotations + +import asyncio +import os +import re +import sys +import time +from datetime import datetime, timezone +from typing import List, Optional + +from beanie import Document, init_beanie +from motor.motor_asyncio import AsyncIOMotorClient +from pydantic import Field +from pymongo import ASCENDING, DESCENDING, IndexModel, ReturnDocument +from pymongo.errors import DuplicateKeyError + +URI = sys.argv[1] if len(sys.argv) > 1 else os.environ.get("MONGO_URI") +DB_NAME = os.environ.get("MONGO_DB", "beanie_test") +TLS_INSECURE = os.environ.get("TLS_INSECURE", "true").lower() != "false" + +passed = 0 +failed = 0 + + +def _ok(name: str) -> None: + global passed + passed += 1 + print(f" \u2705 {name}") + + +def _fail(name: str, err: object) -> None: + global failed + failed += 1 + print(f" \u274C {name}: {err}", file=sys.stderr) + + +async def step(name: str, fn) -> None: + try: + await fn() + _ok(name) + except Exception as err: # noqa: BLE001 - report any failure and continue + _fail(name, err) + + +def sanitize_uri(uri: str) -> str: + return re.sub(r"[?&]replicaSet=[^&]*", "", uri) + + +class Widget(Document): + sku: str + name: str + tags: List[str] = Field(default_factory=list) + price: Optional[float] = None + active: bool = True + created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) + + class Settings: + name = "widgets" + indexes = [ + IndexModel([("sku", ASCENDING)], unique=True), + IndexModel([("name", ASCENDING), ("price", DESCENDING)]), + ] + + +async def run() -> int: + if not URI: + print("MONGO_URI not provided. Pass it as $1 or set MONGO_URI.", file=sys.stderr) + return 2 + + print("Beanie DocumentDB compatibility test") + print("====================================") + + # Fresh collection per run keeps the test idempotent. + Widget.Settings.name = f"widgets_{int(time.time() * 1000)}" + + client: Optional[AsyncIOMotorClient] = None + connected = False + + async def _connect() -> None: + nonlocal client, connected + client = AsyncIOMotorClient( + sanitize_uri(URI), + directConnection=True, + tls=True, + tlsAllowInvalidCertificates=TLS_INSECURE, + serverSelectionTimeoutMS=15000, + ) + await init_beanie(database=client[DB_NAME], document_models=[Widget]) + ping = await client.admin.command("ping") + if ping.get("ok") != 1: + raise RuntimeError("ping did not return ok:1") + connected = True + + await step("connect", _connect) + + if not connected: + print("\nConnection failed; aborting remaining steps.", file=sys.stderr) + return 1 + + coll = Widget.get_motor_collection() + + async def _create_indexes() -> None: + await coll.create_indexes( + [ + IndexModel([("sku", ASCENDING)], unique=True), + IndexModel([("name", ASCENDING), ("price", DESCENDING)]), + ] + ) + + await step("create indexes", _create_indexes) + + created_id = None + + async def _insert_one() -> None: + nonlocal created_id + widget = Widget(sku="SKU-001", name="Gizmo", tags=["alpha", "beta"], price=9.99) + await widget.insert() + created_id = widget.id + if created_id is None: + raise RuntimeError("no _id returned") + + await step("insert_one (Document.insert)", _insert_one) + + async def _insert_many() -> None: + res = await Widget.insert_many( + [ + Widget(sku="SKU-002", name="Gadget", tags=["beta"], price=19.5), + Widget(sku="SKU-003", name="Widget", tags=["alpha", "gamma"], price=4.25), + ] + ) + if len(res.inserted_ids) != 2: + raise RuntimeError(f"expected 2 inserted, got {len(res.inserted_ids)}") + + await step("insert_many", _insert_many) + + async def _get_by_id() -> None: + doc = await Widget.get(created_id) + if doc is None or doc.sku != "SKU-001": + raise RuntimeError("document not found or mismatched") + + await step("get by _id (Document.get)", _get_by_id) + + async def _find_filter_sort_limit() -> None: + docs = await Widget.find({"price": {"$gte": 5}}).sort("-price").limit(10).to_list() + if len(docs) != 2: + raise RuntimeError(f"expected 2 docs, got {len(docs)}") + if docs[0].price < docs[1].price: + raise RuntimeError("sort order incorrect") + + await step("find with filter + sort + limit", _find_filter_sort_limit) + + async def _count() -> None: + n = await Widget.find({}).count() + if n != 3: + raise RuntimeError(f"expected 3 docs, got {n}") + + await step("count_documents", _count) + + async def _update_one() -> None: + res = await coll.update_one({"sku": "SKU-002"}, {"$set": {"price": 21}}) + if res.modified_count != 1: + raise RuntimeError(f"expected 1 modified, got {res.modified_count}") + + await step("update_one ($set)", _update_one) + + async def _find_one_and_update() -> None: + doc = await coll.find_one_and_update( + {"sku": "SKU-003"}, + {"$push": {"tags": "delta"}}, + return_document=ReturnDocument.AFTER, + ) + if not doc or "delta" not in doc.get("tags", []): + raise RuntimeError("update not applied") + + await step("find_one_and_update (returns new)", _find_one_and_update) + + async def _aggregate() -> None: + stats = await Widget.aggregate( + [ + {"$unwind": "$tags"}, + {"$group": {"_id": "$tags", "count": {"$sum": 1}}}, + {"$sort": {"count": -1}}, + ] + ).to_list() + if not stats: + raise RuntimeError("aggregation returned no results") + + await step("aggregation ($unwind/$group)", _aggregate) + + async def _unique_index() -> None: + try: + await Widget(sku="SKU-001", name="Duplicate").insert() + except DuplicateKeyError: + return + raise RuntimeError("duplicate insert was not rejected") + + await step("unique index enforcement (duplicate sku rejected)", _unique_index) + + async def _delete_one() -> None: + res = await coll.delete_one({"sku": "SKU-002"}) + if res.deleted_count != 1: + raise RuntimeError(f"expected 1 deleted, got {res.deleted_count}") + + await step("delete_one", _delete_one) + + async def _drop() -> None: + await coll.drop() + + await step("cleanup (drop collection)", _drop) + + if client is not None: + client.close() + + print("\n====================================") + print(f"Passed: {passed} Failed: {failed}") + return 0 if failed == 0 else 1 + + +def main() -> None: + try: + code = asyncio.run(run()) + except Exception as err: # noqa: BLE001 + print(f"Unexpected error: {err}", file=sys.stderr) + sys.exit(1) + sys.exit(code) + + +if __name__ == "__main__": + main() diff --git a/playgrounds/beanie/app/db.py b/playgrounds/beanie/app/db.py new file mode 100644 index 0000000..5712c74 --- /dev/null +++ b/playgrounds/beanie/app/db.py @@ -0,0 +1,82 @@ +"""Beanie / Motor connection helpers for DocumentDB. + +DocumentDB exposes a single gateway endpoint that speaks the MongoDB wire +protocol but advertises itself as a *standalone* server (not a replica set). +Beanie talks to it through Motor (async PyMongo), so the driver needs three +tweaks: + + - directConnection=True -> don't attempt replica-set topology + discovery (the gateway is standalone). + - tls=True -> the gateway only accepts TLS. + - tlsAllowInvalidCertificates -> the default install uses a self-signed + cert. Set TLS_INSECURE=false and mount a + CA bundle (tlsCAFile) for production-grade + verification. + +The connection string itself (MONGO_URI) carries the credentials. If it still +contains ``replicaSet=rs0`` we strip it here so the driver does not try to match +a replica-set name the gateway never advertises. +""" + +from __future__ import annotations + +import os +import re + +from beanie import init_beanie +from motor.motor_asyncio import AsyncIOMotorClient + +_client: AsyncIOMotorClient | None = None + + +def sanitize_uri(uri: str | None) -> str: + """Remove ``replicaSet=...`` from a DocumentDB connection string. + + ``replicaSet`` is incompatible with a direct connection to the gateway; the + driver raises "client is configured to connect to a replica set named 'rs0' + but this node belongs to a set named 'None'" otherwise. + """ + if not uri: + raise ValueError("MONGO_URI is not set. Provide a DocumentDB connection string.") + return re.sub(r"[?&]replicaSet=[^&]*", "", uri) + + +def build_client_kwargs() -> dict: + """Return the Motor client kwargs DocumentDB's gateway requires.""" + tls_insecure = os.environ.get("TLS_INSECURE", "true").lower() != "false" + return { + "directConnection": True, + "tls": True, + "tlsAllowInvalidCertificates": tls_insecure, + "serverSelectionTimeoutMS": int(os.environ.get("SERVER_SELECTION_TIMEOUT_MS", "10000")), + } + + +async def init(document_models, uri: str | None = None, db_name: str | None = None): + """Connect Motor to DocumentDB and initialise Beanie for ``document_models``.""" + global _client + clean_uri = sanitize_uri(uri or os.environ.get("MONGO_URI")) + database_name = db_name or os.environ.get("MONGO_DB", "beanie_demo") + + _client = AsyncIOMotorClient(clean_uri, **build_client_kwargs()) + await init_beanie(database=_client[database_name], document_models=document_models) + return _client + + +def get_client() -> AsyncIOMotorClient | None: + return _client + + +async def ping() -> bool: + """Return True if the gateway responds to an admin ``ping``.""" + if _client is None: + return False + result = await _client.admin.command("ping") + return result.get("ok") == 1 + + +async def close() -> None: + global _client + if _client is not None: + _client.close() + _client = None diff --git a/playgrounds/beanie/app/main.py b/playgrounds/beanie/app/main.py new file mode 100644 index 0000000..720da74 --- /dev/null +++ b/playgrounds/beanie/app/main.py @@ -0,0 +1,134 @@ +"""FastAPI + Beanie demo REST API for DocumentDB. + +Run locally with scripts/run-app.sh, or directly: + + MONGO_URI="mongodb://user:pass@localhost:10260/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true" \ + python main.py + +Endpoints: + GET /health -> liveness/readiness (checks the gateway ping) + POST /books -> create a book + GET /books -> list books (optional ?author= filter) + GET /books/{id} -> fetch one book by id (see _id known issue) + PATCH /books/{id} -> update a book + DELETE /books/{id} -> delete a book + GET /stats/genres -> aggregation: count of books per genre +""" + +from __future__ import annotations + +import os +from contextlib import asynccontextmanager +from datetime import datetime, timezone +from typing import List, Optional + +from beanie import PydanticObjectId +from fastapi import FastAPI, HTTPException +from fastapi.responses import JSONResponse +from pydantic import BaseModel + +import db +from models.book import Book + +PORT = int(os.environ.get("PORT", "3000")) + + +@asynccontextmanager +async def lifespan(_app: FastAPI): + await db.init([Book]) + print("Connected to DocumentDB via Beanie") + yield + await db.close() + + +app = FastAPI(title="DocumentDB Beanie demo", lifespan=lifespan) + + +class BookCreate(BaseModel): + title: str + author: str + genres: List[str] = [] + pages: Optional[int] = None + published: Optional[datetime] = None + in_stock: bool = True + rating: Optional[float] = None + + +class BookUpdate(BaseModel): + title: Optional[str] = None + author: Optional[str] = None + genres: Optional[List[str]] = None + pages: Optional[int] = None + published: Optional[datetime] = None + in_stock: Optional[bool] = None + rating: Optional[float] = None + + +@app.get("/health") +async def health(): + """Return 200 only when the gateway responds to a ping.""" + try: + if await db.ping(): + return {"status": "healthy", "db": "connected"} + except Exception as exc: # noqa: BLE001 - report any connection error as unhealthy + return JSONResponse(status_code=503, content={"status": "unhealthy", "db": str(exc)}) + return JSONResponse(status_code=503, content={"status": "unhealthy", "db": "disconnected"}) + + +@app.post("/books", status_code=201) +async def create_book(payload: BookCreate): + book = Book(**payload.model_dump()) + await book.insert() + return book + + +@app.get("/books") +async def list_books(author: Optional[str] = None): + query = {"author": author} if author else {} + books = await Book.find(query).sort("-created_at").limit(100).to_list() + return {"count": len(books), "books": books} + + +@app.get("/books/{book_id}") +async def get_book(book_id: PydanticObjectId): + book = await Book.get(book_id) + if book is None: + raise HTTPException(status_code=404, detail="not found") + return book + + +@app.patch("/books/{book_id}") +async def update_book(book_id: PydanticObjectId, payload: BookUpdate): + book = await Book.get(book_id) + if book is None: + raise HTTPException(status_code=404, detail="not found") + changes = payload.model_dump(exclude_unset=True) + if changes: + changes["updated_at"] = datetime.now(timezone.utc) + await book.set(changes) + return book + + +@app.delete("/books/{book_id}", status_code=204) +async def delete_book(book_id: PydanticObjectId): + book = await Book.get(book_id) + if book is None: + raise HTTPException(status_code=404, detail="not found") + await book.delete() + return None + + +@app.get("/stats/genres") +async def genre_stats(): + pipeline = [ + {"$unwind": "$genres"}, + {"$group": {"_id": "$genres", "count": {"$sum": 1}}}, + {"$sort": {"count": -1}}, + ] + return await Book.aggregate(pipeline).to_list() + + +if __name__ == "__main__": + import uvicorn + + uvicorn.run(app, host="0.0.0.0", port=PORT) diff --git a/playgrounds/beanie/app/models/__init__.py b/playgrounds/beanie/app/models/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/playgrounds/beanie/app/models/book.py b/playgrounds/beanie/app/models/book.py new file mode 100644 index 0000000..d1cb24a --- /dev/null +++ b/playgrounds/beanie/app/models/book.py @@ -0,0 +1,39 @@ +"""Example Beanie document model used by the demo API. + +Mirrors the Mongoose ``Book`` schema in the sibling playground so the two demos +are directly comparable. Note there is intentionally **no index collation**: +DocumentDB does not implement collation indexes. +""" + +from __future__ import annotations + +from datetime import datetime, timezone +from typing import List, Optional + +from beanie import Document +from pydantic import Field +from pymongo import ASCENDING, IndexModel + + +def _utcnow() -> datetime: + return datetime.now(timezone.utc) + + +class Book(Document): + title: str + author: str + genres: List[str] = Field(default_factory=list) + pages: Optional[int] = None + published: Optional[datetime] = None + in_stock: bool = True + rating: Optional[float] = None + created_at: datetime = Field(default_factory=_utcnow) + updated_at: datetime = Field(default_factory=_utcnow) + + class Settings: + name = "books" + # Compound index: exercises DocumentDB index creation via Beanie. + # No `collation` option here; DocumentDB does not implement it. + indexes = [ + IndexModel([("author", ASCENDING), ("title", ASCENDING)]), + ] diff --git a/playgrounds/beanie/app/requirements.txt b/playgrounds/beanie/app/requirements.txt new file mode 100644 index 0000000..8fd7ece --- /dev/null +++ b/playgrounds/beanie/app/requirements.txt @@ -0,0 +1,5 @@ +beanie>=1.27,<2 +fastapi>=0.115 +uvicorn[standard]>=0.32 +motor>=3.6 +pydantic>=2.9 diff --git a/playgrounds/beanie/scripts/lib.sh b/playgrounds/beanie/scripts/lib.sh new file mode 100755 index 0000000..84f722f --- /dev/null +++ b/playgrounds/beanie/scripts/lib.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# Shared helpers for the Beanie local playground scripts. +# +# Everything runs on your machine: the DocumentDB local emulator runs in Docker +# and the app/test run as local Python processes that connect to it directly. +set -euo pipefail + +# Connection defaults. Override any of these via environment variables. +DOCUMENTDB_CONTAINER="${DOCUMENTDB_CONTAINER:-documentdb-local}" +DOCUMENTDB_IMAGE="${DOCUMENTDB_IMAGE:-ghcr.io/documentdb/documentdb/documentdb-local:latest}" +DOCUMENTDB_HOST="${DOCUMENTDB_HOST:-localhost}" +DOCUMENTDB_PORT="${DOCUMENTDB_PORT:-10260}" +# Note: the emulator rejects some reserved names (e.g. "documentdb"); use a +# distinct admin username. +DOCUMENTDB_USERNAME="${DOCUMENTDB_USERNAME:-docdbadmin}" +DOCUMENTDB_PASSWORD="${DOCUMENTDB_PASSWORD:-Documentdb!Local1}" + +# Build the MongoDB connection string for the local emulator. The gateway only +# speaks TLS and advertises itself as a standalone server, so we request TLS, +# accept its self-signed cert, and use a direct connection. +build_uri() { + echo "mongodb://${DOCUMENTDB_USERNAME}:${DOCUMENTDB_PASSWORD}@${DOCUMENTDB_HOST}:${DOCUMENTDB_PORT}/?tls=true&tlsAllowInvalidCertificates=true&directConnection=true" +} + +# Wait until a TCP port accepts connections. +# Args: [tries] +wait_for_port() { + local host="$1" port="$2" tries="${3:-60}" i + for i in $(seq 1 "$tries"); do + if (exec 3<>"/dev/tcp/${host}/${port}") 2>/dev/null; then + exec 3>&- 3<&- 2>/dev/null || true + return 0 + fi + sleep 1 + done + return 1 +} + +# Create (once) a Python virtualenv in the app directory and install +# requirements. Echoes the path to the venv's python interpreter. +# Args: +setup_venv() { + local app_dir="$1" + local venv_dir="$app_dir/.venv" + + if [ ! -d "$venv_dir" ]; then + echo "Creating Python virtualenv in $venv_dir ..." >&2 + python3 -m venv "$venv_dir" + fi + "$venv_dir/bin/pip" install --quiet --disable-pip-version-check \ + -r "$app_dir/requirements.txt" >&2 + echo "$venv_dir/bin/python" +} + +require_docker() { + command -v docker >/dev/null || { + echo "docker is required (start Docker Desktop / the Docker daemon first)" >&2 + exit 1 + } + docker info >/dev/null 2>&1 || { + echo "Cannot reach the Docker daemon. Is Docker running?" >&2 + exit 1 + } +} + +container_running() { + [ "$(docker inspect -f '{{.State.Running}}' "$DOCUMENTDB_CONTAINER" 2>/dev/null)" = "true" ] +} + +container_exists() { + docker inspect "$DOCUMENTDB_CONTAINER" >/dev/null 2>&1 +} + +ensure_documentdb() { + require_docker + + if container_running; then + echo "DocumentDB container '$DOCUMENTDB_CONTAINER' is already running." + else + if container_exists; then + docker rm -f "$DOCUMENTDB_CONTAINER" >/dev/null 2>&1 || true + fi + + echo "Starting DocumentDB container '$DOCUMENTDB_CONTAINER' on port ${DOCUMENTDB_PORT} ..." + docker run -dt \ + -p "127.0.0.1:${DOCUMENTDB_PORT}:10260" \ + --name "$DOCUMENTDB_CONTAINER" \ + "$DOCUMENTDB_IMAGE" \ + --username "$DOCUMENTDB_USERNAME" \ + --password "$DOCUMENTDB_PASSWORD" >/dev/null + fi + + wait_for_documentdb +} + +wait_for_documentdb() { + local uri + uri="$(build_uri)" + echo "Waiting for DocumentDB to accept connections ..." + + local i + for i in $(seq 1 60); do + if docker exec "$DOCUMENTDB_CONTAINER" mongosh "$uri" \ + --quiet --eval 'db.adminCommand({ ping: 1 })' >/dev/null 2>&1; then + echo "DocumentDB is ready." + return 0 + fi + sleep 2 + done + + echo "DocumentDB did not become ready in time." >&2 + echo "Check logs with: docker logs $DOCUMENTDB_CONTAINER" >&2 + return 1 +} + +stop_documentdb() { + require_docker + if container_exists; then + echo "Removing DocumentDB container '$DOCUMENTDB_CONTAINER' ..." + docker rm -f "$DOCUMENTDB_CONTAINER" >/dev/null + echo "Done." + else + echo "No DocumentDB container named '$DOCUMENTDB_CONTAINER' found." + fi +} diff --git a/playgrounds/beanie/scripts/run-app.sh b/playgrounds/beanie/scripts/run-app.sh new file mode 100755 index 0000000..20342fd --- /dev/null +++ b/playgrounds/beanie/scripts/run-app.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Run the Beanie demo API locally against a local DocumentDB container. +# +# Starts DocumentDB in Docker (if not already running), sets up the app's +# Python virtualenv/dependencies, then runs the FastAPI + Beanie server. +# +# Prerequisites: docker, python3. +set -euo pipefail + +command -v python3 >/dev/null || { echo "python3 is required" >&2; exit 1; } + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=lib.sh +source "$SCRIPT_DIR/lib.sh" + +APP_DIR="$SCRIPT_DIR/../app" +PORT="${PORT:-3000}" + +ensure_documentdb + +echo "Setting up Python virtualenv and installing dependencies..." +VENV_PY=$(setup_venv "$APP_DIR") + +echo "" +echo "=== Beanie demo app running locally ===" +echo "API: http://localhost:${PORT}" +echo "Docs: http://localhost:${PORT}/docs" +echo "Health: curl http://localhost:${PORT}/health" +echo "Create: curl -X POST http://localhost:${PORT}/books -H 'Content-Type: application/json' \\" +echo " -d '{\"title\":\"Dune\",\"author\":\"Herbert\",\"genres\":[\"sci-fi\"],\"pages\":412}'" +echo "Press Ctrl-C to stop (DocumentDB keeps running; stop it with ./scripts/stop-documentdb.sh)." +echo "" + +cd "$APP_DIR" +MONGO_URI="$(build_uri)" PORT="$PORT" "$VENV_PY" main.py diff --git a/playgrounds/beanie/scripts/run-test.sh b/playgrounds/beanie/scripts/run-test.sh new file mode 100755 index 0000000..dc8ced7 --- /dev/null +++ b/playgrounds/beanie/scripts/run-test.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# Run the Beanie CRUD/compatibility test suite end-to-end against a local +# DocumentDB container. +# +# Starts DocumentDB in Docker (if not already running), then sets up the app +# virtualenv/dependencies and runs the standalone test suite locally. +# +# Prerequisites: docker, python3. +# +# Set KEEP_DB=0 to remove the DocumentDB container when the tests finish +# (default keeps it running for fast re-runs). +set -euo pipefail + +command -v python3 >/dev/null || { echo "python3 is required" >&2; exit 1; } + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=lib.sh +source "$SCRIPT_DIR/lib.sh" + +APP_DIR="$SCRIPT_DIR/../app" +KEEP_DB="${KEEP_DB:-1}" + +ensure_documentdb + +if [ "$KEEP_DB" != "1" ]; then + trap 'stop_documentdb' EXIT +fi + +echo "Setting up Python virtualenv and installing dependencies..." +VENV_PY=$(setup_venv "$APP_DIR") + +echo "" +cd "$APP_DIR" +MONGO_URI="$(build_uri)" "$VENV_PY" beanie_crud_test.py diff --git a/playgrounds/beanie/scripts/start-documentdb.sh b/playgrounds/beanie/scripts/start-documentdb.sh new file mode 100755 index 0000000..7d62c16 --- /dev/null +++ b/playgrounds/beanie/scripts/start-documentdb.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +# Start the DocumentDB local emulator in Docker on your machine. +# +# Idempotent: if the container is already running, this just verifies readiness. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=lib.sh +source "$SCRIPT_DIR/lib.sh" + +ensure_documentdb + +echo "" +echo "=== DocumentDB local emulator is ready ===" +echo "Container: $DOCUMENTDB_CONTAINER (image: $DOCUMENTDB_IMAGE)" +echo "Connection string:" +echo " $(build_uri)" +echo "" +echo "Next:" +echo " ./scripts/run-test.sh # start DocumentDB if needed, then run tests" +echo " ./scripts/run-app.sh # start DocumentDB if needed, then run API" +echo " ./scripts/stop-documentdb.sh # stop + remove the emulator" diff --git a/playgrounds/beanie/scripts/stop-documentdb.sh b/playgrounds/beanie/scripts/stop-documentdb.sh new file mode 100755 index 0000000..d045e94 --- /dev/null +++ b/playgrounds/beanie/scripts/stop-documentdb.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Stop and remove the DocumentDB local emulator container. +# +# This removes the container and all data it held (the emulator stores data +# inside the container, so this is a full reset). +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +# shellcheck source=lib.sh +source "$SCRIPT_DIR/lib.sh" + +stop_documentdb