diff --git a/.agents/skills/chatnow-orienting/SKILL.md b/.agents/skills/chatnow-orienting/SKILL.md index a5a0eb7..771147f 100644 --- a/.agents/skills/chatnow-orienting/SKILL.md +++ b/.agents/skills/chatnow-orienting/SKILL.md @@ -28,6 +28,7 @@ Verify secondary claims against higher-precedence evidence. Cite repository path 3. Trace entry points and calls across HTTP, brpc, RabbitMQ, WebSocket, Redis, MySQL, Elasticsearch, MinIO, and etcd as applicable. 4. Map state ownership, trusted identity derivation, sync/async boundaries, retries, idempotency keys, outboxes, and recovery behavior. 5. State current invariants before evaluating a proposal. Distinguish durable sources of truth from caches, routing state, and acceleration layers. + For worker-ID ownership loss, trace process fencing separately from service-registration recovery; a recovered registration never proves safe ID allocation. See the worker fail-stop contract in `references/core-flows.md`. 6. Separate **Current behavior** from **Proposed behavior**. Never describe an intended design as implemented. 7. Identify compatibility, partial-failure, observability, deployment, and test impact. 8. For any architecture, service-boundary, infrastructure-topology, or core-flow change, require updates to this Skill's affected references in the same PR. A follow-up Issue is not a substitute. diff --git a/.agents/skills/chatnow-orienting/references/core-flows.md b/.agents/skills/chatnow-orienting/references/core-flows.md index 4f4a0d9..972dacd 100644 --- a/.agents/skills/chatnow-orienting/references/core-flows.md +++ b/.agents/skills/chatnow-orienting/references/core-flows.md @@ -2,26 +2,33 @@ Target version: `3.0-dev` Status: Current -Verified: 2026-07-22 +Verified: 2026-09-13 These are current-state flows for the `3.0-dev` line. Re-verify affected symbols at the target commit and keep proposals in a separate section. +The release contract confirmed on 2026-09-13 preserves concurrent authenticated devices for one account. Logging in on another device must not invalidate the earlier device merely because it is newer. `DeleteMessages` removes only the authenticated caller's timeline references; it does not recall the shared message or remove another user's history. SC-07 and FN-MS-10 verify these boundaries with real sessions and persisted timelines. + ## HTTP request flow **Flow:** Client -> Gateway HTTP -> discovered brpc service -> Protobuf response envelope. - Entry/contracts: `gateway/source/gateway_server.h`; affected `proto/*/*_service.proto`; `proto/common/envelope.proto`. - Boundary: Gateway parses Protobuf, authenticates JWT except whitelisted Identity routes, derives `user_id`/`device_id`/JTI, adds trace context, and serializes `RpcMetadata` into the brpc attachment. -- Discovery: `ServiceManager` resolves service instances through etcd; calls are synchronous with route-specific timeouts. +- Discovery: `ServiceManager` resolves service instances through etcd; numeric endpoints retain direct brpc channels, while hostname endpoints use brpc's periodic DNS naming service. The same registration value can therefore recover after an IP change without restarting callers. Calls retain `baidu_std` and existing route-specific timeouts; DNS discovery does not add application retries or delivery guarantees. +- Registration: `Registry` grants a 30-second lease and replaces it every ten seconds using bounded grant/put/revoke calls. It publishes the remembered key/value under the fresh lease before revoking the previous one; repeated PUT values retain existing channels. Each etcd operation has a two-second timeout; unsuccessful replacement retries after one second. This avoids the pinned keepalive stream's unbounded cancellation path. Shutdown serializes with publication, stops and joins the worker, then revokes only its owned lease, so an old instance cannot delete a newer instance's registration. Initial registration failure retains its startup failure behavior. RL-DISCOVERY-02 expires the lease and requires registration and account RPC to recover in the same Identity and caller processes. - Failure: malformed input, unavailable backends, and RPC timeouts are translated into a Protobuf `ResponseHeader`; retries remain a client decision unless a flow states otherwise. - Tests: `tests/bvt`, affected `tests/func`, `tests/func/auth_middleware_test.go`, `tests/func/security_test.go`. - Invariants: client identity fields never override server-derived auth context; services validate required metadata; trace context propagates where supported. ## Message send and delivery +**Test boundary (3.0-dev, Current, 2026-09-14):** Broker confirmation remains the send API success boundary. `tests/pkg/fixture/message.go`'s `SendTextMessage` additionally waits for the Message/MySQL row before returning to tests that read or mutate it. The consumer commits the row, member timelines and conversation watermark together; this fixture wait adds no production retry or wire change. Tests of broker acceptance and Message outages retain the explicit client-ID or direct-RPC helper. RL-MESSAGE-02 distinguishes these boundaries; failed-response and CI snapshot diagnostics support investigation of service availability without declaring historical RPC 9002 failures resolved. + +Worker ownership failure (`3.0-dev`, 2026-09-13, Current): `TransmiteServer::start` polls the selected worker allocator once per second. After it reports loss, the watchdog writes the fixed stderr diagnostic `worker_lease_lost action=exit code=1` and calls `std::_Exit(EXIT_FAILURE)` to stop every thread. It does not wait for registry network calls, RPC draining, destructors, or async logging; the existing ownership detection policy remains unchanged. A surviving stale registration expires within its existing 30-second lease or is replaced by the restarted process. This is a fail-stop, not graceful request completion or proof of instantaneous lease-expiry detection. Container restart reacquires a worker before serving. RL-WORKER-01 revokes the isolated stack's sole worker lease and records the original process's exit, diagnostic, and recovery across subsequent watchdog cycles. The previous `abort()` path was observed as two SIGABRT deliveries followed by SIGSEGV at libc's `hlt` fallback while the service was container PID 1; exit code 139 alone is not evidence of a memory corruption in this path. + **Flow:** Client -> Gateway -> Transmite -> RabbitMQ message exchange -> Message/MySQL -> RabbitMQ push queue -> Push -> WebSocket. -- Entry/contracts: Gateway `/service/message/send`; `proto/transmite/transmite_service.proto`; `proto/message/message_internal.proto`; `proto/push/notify.proto`. +- Entry/contracts: Gateway `/service/transmite/send`; `proto/transmite/transmite_service.proto`; `proto/message/message_internal.proto`; `proto/push/notify.proto`. - Transmite: validates auth/membership and content, allocates conversation and per-user sequences in Redis, creates a Snowflake message ID, protects `client_msg_id` with a Redis idempotency key, and completes the brpc request after publisher confirm. - Message: consumes `InternalMessage`, persists the message then per-user timelines in MySQL, treats duplicate message inserts idempotently, publishes push only after persistence, and asynchronously indexes text in Elasticsearch. - Push: consumes the push event, resolves Redis/local routes, records per-device unacked payloads, sends locally or uses asynchronous cross-instance `PushBatch`, then WebSocket delivery reaches the client. @@ -29,6 +36,20 @@ These are current-state flows for the `3.0-dev` line. Re-verify affected symbols - Tests: `tests/bvt/message_test.go`, `tests/func/transmite_test.go`, `tests/func/message_test.go`, `tests/func/ws_notify_test.go`, `tests/func/scenarios_test.go`, `tests/perf/send_msg_test.go`, `tests/perf/sync_test.go`. - Invariants: Message/MySQL is the stored-message source of truth; persistence precedes normal push publication; `request_id`, `client_msg_id`, message ID, conversation sequence, and user sequence have distinct roles; do not claim exactly-once delivery. +An `accepted:` or `persisted:` Redis hit is a deduplication guard, not a complete message response. Transmite retrieves the original through the authenticated `SelectByClientMsgId` RPC with a one-second timeout and no transport retries. A missing/unreadable message or zero identity/sequence returns the existing `9002` / `duplicate request in flight` envelope without a message. The caller does not own that prior guard and must not delete it, allocate sequences, or republish. Once Message persists the original, a client retry returns its complete durable result. The pending-record lookup uses the same RPC budget. Cache formats and the 24-hour TTL remain unchanged; mixed-version older Transmite instances retain their old incomplete-success behavior until upgraded. FN-TM-02, RL-MESSAGE-01, and SC-06 cover the unreadable, unavailable, and recovered paths. + +Message commits the conversation `max_seq` watermark in the same MySQL transaction as the message and timeline inserts. The watermark never decreases when deliveries arrive out of order; Conversation metadata writers preserve the latest committed value under a row lock. The Message database principal therefore needs `SELECT, UPDATE` on `conversation`. + +Log-context storage must initialize in Release builds as well as Debug; initialization cannot depend on assertions. Transmite copies the authenticated RPC trace into MQ headers at publish time. Push invalidates its local route cache while binding a newly authenticated device under the same per-user lock used by route fills; it does not cache empty routes. Dismissed conversations are rejected by `GetMemberIds` before consulting cached membership. + +Redis availability classification includes the pinned client's exhausted shard-refresh wrapper, while Redis command errors remain separate. An unavailable member snapshot has an unknown version: Transmite queries Conversation for that request and does not publish the result into L1/L2 without a version fence. Known-version races retain retry behavior. Sequence allocation and Unacked persistence remain Redis truth-source operations and fail unavailable during an outage. + +Both MQ consumer overloads translate `NackRequeue` and callback exceptions to `reject(deliveryTag, AMQP::requeue)`. The library parameter is a bitmask; passing a boolean does not request requeue. `NackDiscard` uses zero flags. Push must requeue on Unacked persistence failure and deliver only after durable persistence succeeds. + +## Business notifications + +Relationship emits friend-request and accepted-request notifications, and Conversation emits creation notifications through a bounded Push `PushBatch` RPC after the domain write commits. Auth metadata and trace are forwarded. These online notifications are best effort, with no new durable retry guarantee. A creation broadcast omits the creator's `self` member state; each recipient obtains its own state through Conversation APIs. Tests: FN-WS-02/03/04. + ## Delivery ACK convergence **Flow:** Client WebSocket `MSG_PUSH_ACK` -> Push validation -> Redis unacked removal -> asynchronous Message `UpdateReadAck` -> MySQL convergence. @@ -52,6 +73,8 @@ These are current-state flows for the `3.0-dev` line. Re-verify affected symbols - Tests: `tests/bvt/auth_test.go`, `tests/func/identity_test.go`, `tests/func/auth_middleware_test.go`, `tests/func/security_test.go`, `tests/func/scenarios_test.go`, and `tests/func/ws_notify_test.go` (`FN-WS-09`). - Invariants: only Identity issues/refreshes tokens; access and refresh token purposes remain distinct; downstream identity comes from verified claims and forwarded metadata, not request bodies; Push admission must resolve revocation before publishing any authenticated-session side effect. +The shared `JwtCodec` reports `1002` for a correctly signed expired JWT regardless of its age. In the expired branch, signature verification and strict `iat`/`nbf` checks run before the unconditional expiration rejection; that secondary verifier does not check `exp` again. This does not extend token validity or return authenticated claims. Invalid signatures, unknown keys, and invalid future time claims remain rejected with `1003`. FN-AM-07 exercises Identity refresh responses and Gateway rejection using synthetic signed tokens. + ## Runtime secrets ### Current @@ -71,6 +94,8 @@ Redis authentication, dynamic reload, automatic rotation, and additional credent **Flow:** Apply/init -> presigned MinIO upload -> complete -> MySQL metadata/quota -> authenticated download request -> presigned MinIO GET. +Ordinary PUT URLs use the standard S3 presigner with the headers returned to the client, without implicit SSE-C headers. `use_path_style` disables AWS virtual addressing for internal MinIO hostnames; internal object verification and public presigning can use different endpoints. + - Entry/contracts: Gateway Media routes; `proto/media/media_service.proto`; `media/source/media_server.h`; `media/source/upload_handler.hpp`; `media/source/multipart_handler.hpp`; `media/source/download_handler.hpp`. - Stores: MinIO holds bytes; MySQL holds `media_file`, blob-ref/dedup, multipart, and per-user quota state; Redis coordinates cleanup/locks where implemented. - Boundaries: clients upload/download directly with short-lived presigned URLs. Apply validates size/MIME/hash/quota and records pending metadata; complete verifies object existence/size, converges dedup/refcount/quota, and is idempotent for committed files. @@ -78,11 +103,20 @@ Redis authentication, dynamic reload, automatic rotation, and additional credent - Async/retry: cleanup handles stale pending, quarantine, and unreferenced object paths; client retries must preserve file/upload identifiers and completion idempotency. - Tests: `tests/bvt/media_test.go`, `tests/func/media_test.go`, `tests/func/concurrency_test.go`, `tests/func/scenarios_test.go`, `tests/perf/upload_test.go`, `tests/pkg/verify/minio.go`. - Invariants: service processes metadata rather than normal file bytes; only committed objects are downloadable; MySQL metadata/quota and MinIO object state must converge; preserve dedup and completion idempotency. +- Deduplication shares the stored object, not the file identifier: repeated uploads receive distinct metadata references to the same bucket/object key. Functional checks must validate both the distinct references and shared bytes. + +Multipart routes require the same JWT boundary as single uploads. `partNumber` and `uploadId` are included before SigV4 signing. Test content follows the MIME allowlist and uses non-final parts of at least 5 MiB; FN-MD-07 verifies the downloaded bytes, and FN-MD-21 verifies signature tampering is rejected. + +Media `FileInfo.public_url` is additive field 6: it contains the canonical configured public prefix plus the committed object's key, and stays empty for private objects. Identity discovers Media and resolves stored avatar file IDs through authenticated `GetFileInfo`; it returns an empty avatar URL when resolution is unavailable. Media configuration is authoritative for the public prefix. FN-ID-08 checks an actual HTTP GET and persisted profile reads. + +Speech recognition rejects empty/unaligned PCM16 input. Until an ASR backend is integrated, valid input returns an explicit unavailable error rather than empty success (FN-MD-17/18/23). ## Presence and typing **Flow:** Push WebSocket lifecycle -> Redis presence/routes -> Presence aggregation/subscriptions -> Presence or Push notification -> WebSocket. +Push routes each per-device write pipeline by its Redis key and stores enum names (`ONLINE`/`OFFLINE`). Presence accepts those names and legacy numeric enum values; malformed states are ignored. Registration alone does not create an online connection. BVT-018 opens an authenticated socket before asserting ONLINE. + - Entry/contracts: `push/source/push_server.h`; `proto/presence/presence_service.proto`; `presence/source/presence_server.h`; `proto/push/notify.proto`. - Ownership: Push owns connections and route binding, writes per-device online/offline/heartbeat TTL state, and emits lifecycle notifications. Presence aggregates Redis device state, manages subscription sets and typing TTLs, and calls Push for delivery. - Async/retry: lifecycle and typing notifications are fail-soft asynchronous Push RPCs; Redis TTL supplies eventual offline behavior; multi-instance fanout uses Push routing and cross-instance RPC. @@ -92,3 +126,11 @@ Redis authentication, dynamic reload, automatic rotation, and additional credent ## Architecture-change synchronization Any change to these paths, ownership boundaries, stores, protocols, topology, ordering, retry, idempotency, trust, or failure semantics must update this reference and any affected `technology-stack.md` or `repository-map.md` content in the same PR. + +## Disposable CI startup + +CI builds the nine native services once in the pinned Ubuntu builder, restores that artifact into each fresh test checkout, generates synthetic credentials, and runs Compose initialization before semantic readiness. MySQL, Redis Cluster, RabbitMQ, and MinIO initialization must converge before application services and runtime tests proceed. Existing environment files or persisted data cause test bootstrap to fail closed. Runtime results must be reported separately from static contracts. + +Reliability RL-05 runs in its own disposable stack with Transmite user/session limits of 8/40 per minute. Its bounded outage burst exercises local fallback without depending on exhausting production defaults (600/3000). Push outage tests use the authenticated device ID and prohibit delivery of the particular unpersisted marker, while allowing redelivery of older durable messages. Multi-device login and caller-only deletion semantics remain unchanged. + +The Redis fault pauses processes without withdrawing container DNS. Its recovery evidence does not cover stop/recreate, resolver failure or topology changes; those remain separate from the tested circuit behavior. diff --git a/.agents/skills/chatnow-orienting/references/repository-map.md b/.agents/skills/chatnow-orienting/references/repository-map.md index 2697878..e5e4f38 100644 --- a/.agents/skills/chatnow-orienting/references/repository-map.md +++ b/.agents/skills/chatnow-orienting/references/repository-map.md @@ -11,30 +11,30 @@ Verified: 2026-07-22 | `common/` | Shared auth, DAO, errors, infrastructure, MQ, logging, and utilities | `common/auth/auth_context.hpp`, `common/dao/data_redis.hpp`, `common/error/handle_rpc.hpp`, `common/infra/etcd.hpp`, `common/mq/channel.hpp` | | `proto/` | External and internal Protobuf contracts | `proto/common/envelope.proto`, `proto/common/auth/metadata.proto`, affected `proto/*/*_service.proto` | | `gateway/` | HTTP entry, JWT validation, discovery-based routing, response envelopes | `gateway/source/gateway_server.h`, `gateway/source/gateway_auth.hpp`, `gateway/source/gateway_server.cc`, `gateway/CMakeLists.txt` | -| `identity/` | Registration, login/logout, token issue/refresh, profile and lookup | `identity/source/identity_server.h`, `identity/source/identity_server.cc`, `proto/identity/identity_service.proto` | +| `identity/` | Registration, login/logout, token issue/refresh, profile and Media-backed avatar lookup | `identity/source/identity_server.h`, `identity/source/identity_server.cc`, `proto/identity/identity_service.proto` | | `relationship/` | Friend requests, relationships, blocking | `relationship/source/relationship_server.h`, `relationship/source/relationship_server.cc`, `proto/relationship/relationship_service.proto` | | `conversation/` | Conversation lifecycle, membership, unread, visibility, pins, drafts | `conversation/source/conversation_server.h`, `conversation/source/conversation_server.cc`, `proto/conversation/conversation_service.proto` | | `transmite/` | Message authorization, sequencing, idempotency, MQ publication | `transmite/source/transmite_server.h`, `transmite/source/transmite_server.cc`, `proto/transmite/transmite_service.proto` | -| `message/` | Message persistence, timelines, sync/history, read-ACK convergence, search indexing | `message/source/message_server.h`, `message/source/message_server.cc`, `proto/message/message_service.proto`, `proto/message/message_internal.proto` | +| `message/` | Message persistence and atomic conversation watermark, timelines, sync/history, read-ACK convergence, search indexing | `message/source/message_server.h`, `message/source/message_server.cc`, `proto/message/message_service.proto`, `proto/message/message_internal.proto` | | `media/` | Presigned upload/download, multipart, dedup, quota, metadata, cleanup, speech | `media/source/media_server.h`, `media/source/upload_handler.hpp`, `media/source/download_handler.hpp`, `proto/media/media_service.proto` | | `presence/` | Presence aggregation, subscriptions, and typing coordination | `presence/source/presence_server.h`, `presence/source/presence_server.cc`, `proto/presence/presence_service.proto` | | `push/` | WebSocket connections, routes, cross-instance delivery, resend, client ACK ingestion | `push/source/push_server.h`, `push/source/connection.hpp`, `push/source/push_server.cc`, `proto/push/notify.proto` | | `odb/` | ODB entity definitions and durable relational fields | Affected entity, especially `message.hxx`, `user_timeline.hxx`, `conversation_member.hxx`, and `media_*.hxx` | -| `conf/` | Non-secret local/container flags and JSON configuration; tracked files are not a runtime secret source | `conf/local/`, `conf/docker/`, `conf/auth.json`, `conf/media.json` | -| `sql/` | Versioned schema migrations | `sql/V4__media.sql` and any migration matching affected ODB entities | -| `docker/` | Separate MinIO topology and initialization; not wired into the root application network | `docker/docker-compose.yml`, `docker/minio-init/entrypoint.sh` | -| `docker-compose.yml` | Application stack declaration; Media object-storage wiring is incomplete | Root `docker-compose.yml`, then affected `Dockerfile` and `conf/docker` file | -| `scripts/` | Operational support and monitoring configuration | `scripts/install_aws_sdk_linux.sh`, `scripts/prometheus/redis_alerts.yml` | +| `conf/` | Non-secret local/container flags and JSON configuration; tracked files are not a runtime secret source | `conf/local/`, `conf/docker/`, `conf/auth.json`, local `conf/media.json`, container `conf/docker/media.json` | +| `sql/` | Versioned forward-only schema migrations for all current ODB objects | `sql/V1__core.sql`, `sql/V4__media.sql`, `scripts/init_mysql.sh` | +| `docker/` | Reusable MinIO initialization script plus a supplemental standalone topology that is not combined with root Compose | `docker/minio-init/entrypoint.sh`, `docker/docker-compose.yml` | +| `docker-compose.yml` | Integrated local application topology, health conditions, and one-shot convergence services | Root `docker-compose.yml`, affected `Dockerfile`, `conf/docker`, and initializer script | +| `scripts/` | Runtime convergence/readiness, operational support, and monitoring | `scripts/init_mysql.sh`, `scripts/converge_mysql_users.sh`, `scripts/init_redis_cluster.sh`, `scripts/init_rabbitmq.py`, `scripts/wait_for_services.sh`, `scripts/prometheus/redis_alerts.yml` | | `tests/` | Pure-Go L1-L4 plus Redis-focused Reliability framework, clients, fixtures, cleanup, and store verification | `tests/Makefile`, `tests/config.yaml`, affected `tests/bvt`, `tests/func`, `tests/perf`, `tests/reliability`, `tests/pkg` | -| `docs/` | Secondary architecture/API context and canonical operations guidance | `docs/operations/runtime-secrets.md`, affected `docs/api/*.yaml`, then relevant architecture documents | +| `docs/` | Secondary architecture/API context and canonical operations guidance | `docs/operations/compose-runtime.md`, `docs/operations/runtime-secrets.md`, affected `docs/api/*.yaml`, then relevant architecture documents | ## Verified ports and infrastructure endpoints -Application ports come from `conf/local`, `conf/docker`, and root `docker-compose.yml`. MinIO ports come from the separate `docker/docker-compose.yml`; this is not an integrated container endpoint map. +Application and integrated infrastructure ports come from `conf/local`, `conf/docker`, and root `docker-compose.yml`. Published root-profile infrastructure ports bind to loopback. | Owner | Local endpoint/port | Container endpoint/port | Evidence | |---|---:|---:|---| -| Gateway HTTP | `127.0.0.1:9000` | `gateway_server:9000` | `gateway_server.conf` `http_listen_port` | +| Gateway HTTP and readiness | `127.0.0.1:9000`; unauthenticated `GET /health` | `gateway_server:9000` | `gateway_server.conf` `http_listen_port`; `GatewayServer::dependencies_ready` | | Media brpc | `127.0.0.1:10002` | `media_server:10002` | `media_server.conf` | | Identity brpc | `127.0.0.1:10003` | `identity_server:10003` | `identity_server.conf` | | Transmite brpc | `127.0.0.1:10004` | `transmite_server:10004` | `transmite_server.conf` | @@ -47,14 +47,18 @@ Application ports come from `conf/local`, `conf/docker`, and root `docker-compos | etcd | `127.0.0.1:2379` | `etcd:2379` | all service configs | | MySQL | `127.0.0.1:3306` | `mysql:3306` | root Compose | | Redis cluster | `127.0.0.1:6379`, `:6380`-`:6384` | `redis-node1:6379`, `redis-node2:6380` through `redis-node6:6384` | root Compose; service seed flags | -| RabbitMQ | `127.0.0.1:5672` | `rabbitmq:5672` | Transmite, Message, Push configs | +| RabbitMQ | `127.0.0.1:5672` | `rabbitmq:5672` | Transmite, Message, Push host-only `mq_host` configs; builders append `5672` | | Elasticsearch | HTTP `127.0.0.1:9200`; transport `:9300` | `elasticsearch:9200`; transport `:9300` | root Compose; service configs | -| MinIO S3 | Host `127.0.0.1:9000`, conflicting with Gateway | `minio:9000` only inside the separate MinIO Compose network | `conf/media.json`; supplemental Compose | -| MinIO console | Host `127.0.0.1:9001`, conflicting with Push WebSocket | `minio:9001` only inside the separate MinIO Compose network | supplemental Compose | +| MinIO S3 | `127.0.0.1:19000` for local access and presigned URLs | `minio:9000` for Media internal operations | root Compose; `conf/media.json`; `conf/docker/media.json` | +| MinIO console | `127.0.0.1:19001` | `minio:9001` | root Compose | -MySQL service configs set `mysql_port=0`, while root Compose exposes MySQL on `3306` and service entrypoints wait on `mysql:3306`; preserve that distinction when diagnosing driver defaults. Gateway's `websocket_listen_port=0` is not the client WebSocket endpoint; Push owns `ws_port=9001`. +MySQL service configs set `mysql_port=0`, while root Compose exposes MySQL on `3306` and service entrypoints wait on `mysql:3306`; preserve that distinction when diagnosing driver defaults. Gateway's `websocket_listen_port=0` is not the client WebSocket endpoint; Push owns `ws_port=9001`. Gateway `GET /health` returns `200` only when all eight discovered business-service channels are available and returns `503` otherwise; it is not a process-only liveness response. -Root Compose mounts `conf/media.json` into Media, but `s3.endpoint=http://127.0.0.1:9000` addresses the Media container itself. Root Compose has no MinIO service/dependency, while the supplemental MinIO Compose project has no declared shared external network with the root project. Do not present these declarations as a working integrated Media topology or recommend their current commands as a functional Media runtime. Any repair must explicitly reconcile the network, endpoint, dependency, and `9000`/`9001` host-port conflicts, then be verified from the affected containers. +Root Compose mounts `conf/docker/media.json` read-only. Media uses `s3.endpoint=http://minio:9000` for server-side S3 operations and `s3.public_endpoint=http://127.0.0.1:19000` to generate URLs reachable by local host clients. `common/infra/s3_client.hpp` owns separate internal and presign clients. Do not use the loopback public endpoint for a remote deployment without replacing it with a client-reachable address. + +`mysql-init` applies read-only `V*.sql` migrations through `scripts/init_mysql.sh` and a checksum ledger. `V1__core.sql` plus `V4__media.sql` cover all 17 current ODB object tables; `scripts/converge_mysql_users.sh` owns the five table-scoped application identities. Redis, RabbitMQ, and MinIO use their own bounded one-shot initializers. `scripts/wait_for_services.sh` is the cross-stack semantic gate; `entrypoint.sh` is only bounded TCP prerequisite polling. + +Root infrastructure state is bind-mounted under `middle/data`. `docker compose down -v` does not remove that state, and CI uses a fresh runner checkout per job as its disposable storage owner. The synthetic environment helper refuses existing data. Do not claim a repeatable cold start or passing runtime gate from the source topology or static contracts. ## Runtime credential ownership @@ -64,7 +68,7 @@ Current consumers at the verified commit are: - Conversation, Identity, Media, Message, and Relationship resolve service-specific MySQL password inputs through `common/config/secret_resolver.hpp`. - Transmite, Message, and Push resolve service-specific RabbitMQ password inputs through the same resolver. - Identity resolves its SMTP password; Media resolves separate S3 access-key and secret-key inputs. Non-secret S3 settings remain in `conf/media.json`. -- Root Compose requires MySQL, RabbitMQ, and supplemental MinIO bootstrap values through deployment environment references. These are separate from least-privileged application inputs. Redis has no configured password or ACL consumer. +- Root Compose requires MySQL, RabbitMQ, and MinIO bootstrap values through deployment environment references. One-shot initializers use those bootstrap inputs to converge least-privileged MySQL, RabbitMQ, and MinIO application identities; application containers consume only their own resolver inputs. Redis has no configured password or ACL consumer. Tracked runtime credential literals have been removed from the scoped source, configuration, Compose, and test-runtime surfaces. Do not reintroduce values in documentation, logs, test output, Issues, or PRs. Synthetic test-only credentials and API examples require narrow scanner exemptions rather than broad path allowlists. diff --git a/.agents/skills/chatnow-orienting/references/technology-stack.md b/.agents/skills/chatnow-orienting/references/technology-stack.md index 4232c93..479e502 100644 --- a/.agents/skills/chatnow-orienting/references/technology-stack.md +++ b/.agents/skills/chatnow-orienting/references/technology-stack.md @@ -2,7 +2,7 @@ Target version: `3.0-dev` Status: Current -Verified: 2026-07-22 +Verified: 2026-09-13 Use this reference for the `3.0-dev` architecture line, then verify task-sensitive details at the resolved commit. @@ -19,7 +19,7 @@ Use this reference for the `3.0-dev` architecture line, then verify task-sensiti | Cache and coordination | Redis 7 Cluster plus local L1 caches | `common/dao/data_redis.hpp`; `common/utils/local_cache.hpp` | | Message broker | RabbitMQ through AMQP-CPP/libev | `common/mq/`; Transmite, Message, and Push source | | Search | Elasticsearch 7 | `common/dao/data_es.hpp`; Message, Identity, Relationship source | -| Object storage | MinIO through the S3-compatible AWS C++ SDK | `common/infra/s3_client.hpp`; Media source; `docker/docker-compose.yml` | +| Object storage | MinIO through the S3-compatible AWS C++ SDK | `common/infra/s3_client.hpp`; Media source; root `docker-compose.yml`; `conf/docker/media.json` | | Service discovery and leases | etcd | `common/infra/etcd.hpp`; service entry files | | Authentication | JWT HS256 with multi-key rotation support | `common/auth/`; `conf/auth.json` | | Logging | spdlog JSON lines with propagated trace context | `common/infra/logger.hpp`; `common/log/`; `gateway/source/gateway_trace.hpp` | @@ -28,6 +28,12 @@ Use this reference for the `3.0-dev` architecture line, then verify task-sensiti ## Build entry points +The CI Reliability job selects `tests/compose/reliability.yml` in addition to root Compose. That isolated test-only network has explicit IPAM so the Identity endpoint fault can request and restore IPv4 addresses on supported Docker Engine versions; it does not change the normal development network. See the testing framework for subnet overrides and preflight behavior. + +`common/mq/channel.hpp` uses direct brpc initialization for numeric endpoints and `Init("http://:", "rr", options)` for DNS endpoints. Here `http://` selects the DNS naming service, not the wire protocol: internal RPC remains `baidu_std`. The pinned brpc implementation refreshes DNS every five seconds by default (`ns_access_interval`), including when the etcd registration string is unchanged. Connection timeout, RPC timeout and retry limits remain in `ServiceChannel`; literal IPv4/IPv6 endpoints keep the existing direct path. References: [brpc client naming services](https://brpc.apache.org/docs/client/basics/), [pinned periodic refresh implementation](https://github.com/apache/brpc/blob/041cec5fb84a5b4458bac6275ea7d34e048bc3f1/src/brpc/periodic_naming_service.cpp). + +`Registry` in `common/infra/etcd.hpp` owns registration recovery. The pinned [etcd-cpp KeepAlive implementation](https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/blob/ba6216385fc332b23d95683966824c2b86c2474e/src/KeepAlive.cpp) stops after a terminal error, while its [stream creation/cancellation](https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/blob/ba6216385fc332b23d95683966824c2b86c2474e/src/v3/AsyncGRPC.cpp) includes unbounded completion-queue waits. Registry therefore uses only deadline-bound unary operations: every ten seconds it grants a fresh 30-second lease, publishes the remembered key/value under it, then revokes the previous lease. This trades up to three operations and one repeated PUT per successful cycle for explicit timeout and shutdown control; `ServiceChannel::append` deduplicates unchanged host values, retaining caller channels. Each operation has a two-second deadline, failed cycles retry after one second, and an unsuccessful candidate is revoked or expires naturally. `unregister()` is idempotent and permanently disables this Registry, joins recovery, and revokes the owned lease rather than deleting an unconditionally named key. Wire contracts, key paths, registration values and lease TTL are unchanged; lease IDs now rotate during normal operation, and older instances still lack recovery until upgraded. This does not claim complete service shutdown bounds for other unrelated worker/discovery threads or production-scale performance. + The root CMake project adds all nine services. The CI-equivalent build is: ```bash @@ -42,18 +48,22 @@ Inspect root `CMakeLists.txt`, the affected service's `CMakeLists.txt`, and its - Local service flags: `conf/local/*_server.conf`. - Container service flags: `conf/docker/*_server.conf`. - JWT keys and TTLs: Identity, Gateway, and Push resolve `CHATNOW_JWT_CONFIG` or `CHATNOW_JWT_CONFIG_FILE` at process startup. The value is the complete JSON document. -- Media S3 application credentials are resolved through the common secret resolver. Buckets, endpoint, presign, and MIME policy remain in `conf/media.json` plus Media flags. +- Media S3 application credentials are resolved through the common secret resolver. `conf/media.json` is the local-process configuration; root Compose mounts `conf/docker/media.json` read-only. Container-internal S3 operations use `http://minio:9000`, while client-facing local presigned URLs use the published `http://127.0.0.1:19000` endpoint. - Example Transmite flags: `conf/transmite_server.conf.example`. - Service defaults and flag definitions: each `/source/_server.cc`. - MySQL passwords for Conversation, Identity, Media, Message, and Relationship use service-specific direct-environment or `_FILE` inputs through `common/config/secret_resolver.hpp`. - RabbitMQ passwords for Transmite, Message, and Push use the same resolver contract. Identity SMTP and Media S3 application credentials are also migrated. +- The `mq_host` flag is host-only for Transmite, Message, and Push. Their builders append the fixed AMQP port `5672`; configuration must not include a port. - The resolver accepts exactly one allowlisted direct environment variable or `_FILE` locator, fails closed on missing/conflicting input, and validates secret-file type, owner, mode, size, and content. It reads once at startup; there is no hot reload. - Bootstrap credentials in Compose remain deployment environment references rather than application resolver inputs. Redis has no configured password or ACL consumer. - The canonical names, consumers, deployment rules, and limitations are maintained in `docs/operations/runtime-secrets.md`. -- Root `docker-compose.yml` declares the application stack used by CI, but it is not a complete integrated Media/MinIO topology: it starts Media without a MinIO service or dependency. -- `docker/docker-compose.yml` separately declares MinIO and its initialization sidecar on a different default Compose network. Media mounts `conf/media.json`, whose `http://127.0.0.1:9000` endpoint resolves to the Media container itself, not to that separate MinIO container. +- Root `docker-compose.yml` is the integrated local application topology. It includes health-checked MySQL, six-node Redis Cluster, RabbitMQ, Elasticsearch, etcd, and MinIO plus one-shot `mysql-init`, `redis-cluster-init`, `rabbitmq-init`, and `minio-init` convergence services. Application dependencies use health or `service_completed_successfully` conditions instead of fixed startup delays. +- `mysql-init` mounts `sql/` read-only, applies ordered `V*.sql` files through a checksum ledger, rejects changes to an applied version, covers the 17 current ODB object tables, and converges five table-scoped MySQL application users. This is a forward-only repository bootstrap mechanism, not a general rollback engine. +- MinIO S3 and console ports are published on loopback `19000` and `19001`; Gateway HTTP remains `9000` and Push WebSocket remains `9001`. The supplemental `docker/docker-compose.yml` is not part of the root topology and must not be combined with it. +- `scripts/wait_for_services.sh` is the bounded semantic cross-stack readiness entry point. It checks Redis Cluster state and slots, the MySQL schema and users, RabbitMQ alarms, Elasticsearch health, MinIO health and buckets, eight exact etcd registrations, dependency-aware Gateway health, and Push listener reachability. The shared service `entrypoint.sh` performs bounded TCP prerequisite polling only. +- Readiness requires GNU `timeout` from coreutils: Docker Compose probes receive only the remaining deadline budget and a one-second kill grace period. The MinIO bucket probe passes bootstrap credentials on stdin inside a temporary container, not in process arguments. -The two Compose declarations also conflict on host ports: Gateway HTTP and MinIO S3 both publish `9000`; Push WebSocket and the MinIO console both publish `9001`. Therefore, neither `docker compose up -d --build` nor running both Compose files as written proves a functional containerized Media flow. Treat the network, Media S3 endpoint, service dependency, and host-port mapping as unresolved executable contradictions that must be fixed and verified before documenting a working container runtime command. +The source topology and static contracts do not prove a successful clean-slate start. Root infrastructure state uses bind mounts under `middle/data`, so `docker compose down -v` does not remove it. Issue #88 integrates the runtime into CI using a fresh checkout per job and synthetic credentials; until fresh dynamic evidence exists, report full-stack cold start and Go runtime gates as unverified. The canonical operating contract is `docs/operations/compose-runtime.md`. ## Verification entry points @@ -68,8 +78,8 @@ The current test framework is entirely Go. New or restored C++ test suites are p | L4 Performance | `tests/perf`, `perf` | `cd tests && make proto && make test-perf` | | Reliability | `tests/reliability`, `reliability` | `cd tests && make proto && make test-reliability` | -Reliability is an executable, Redis-focused layer. Its current tests exercise Redis circuit recovery and Push unacked requeue behavior through `tests/pkg/chaos/redis.go`. The Make target runs the whole layer and does not consume `TEST_RUN`; use a direct tagged `go test ... -run` command when exact selection is required. No current controller covers RabbitMQ, MySQL, arbitrary services, or general network faults, so do not describe this as a broad chaos platform. +Reliability exercises Redis circuit recovery and Push unacked requeue, a scoped Identity endpoint move, Message stop/start, and Identity lease expiry through the controllers in `tests/pkg/chaos`. The lease controller suspends Identity until its exact registration disappears, then resumes the same process; it requires account RPC recovery without restarting Identity, Gateway or Transmite. The Make target runs the whole layer and does not consume `TEST_RUN`; use a direct tagged `go test ... -run` command when exact selection is required. No current controller covers RabbitMQ, MySQL, arbitrary services, or general network faults. -The CI definition has a dedicated `reliability` job that depends on `service-artifacts`, independently of BVT. At this verification date the job exists, but the inspected PR run was skipped after an upstream failure; that is not green runtime evidence. Shared clients, fixtures, polling, cleanup, and direct store verification live under `tests/pkg`. +The CI definition has a dedicated `reliability` job that depends on `service-artifacts`, independently of BVT. The job uses a dedicated test stack with bounded RL-05 limits. Report the actual current-head runtime result; the existence of a job is not green runtime evidence. Shared clients, fixtures, polling, cleanup, and direct store verification live under `tests/pkg`. The CI definition is `.github/workflows/ci.yml`; verify its commands against files present at the target commit before copying them into local instructions. diff --git a/.agents/skills/chatnow-testing/references/case-catalog.md b/.agents/skills/chatnow-testing/references/case-catalog.md index 09c0456..e64e29b 100644 --- a/.agents/skills/chatnow-testing/references/case-catalog.md +++ b/.agents/skills/chatnow-testing/references/case-catalog.md @@ -24,10 +24,12 @@ Case IDs are stable identities, not completion claims. Put the ID in the case co | Performance | `PF-01` through `PF-08` | | Reliability | `RL--` | -Reliability is a distinct reserved namespace and layer. The current tree has no executable Reliability suite; reserving an ID never authorizes inventing a directory, target, command, or successful run. +Reliability has an executable Compose gate in `tests/reliability` (`make -C tests test-reliability`). The existing RL-05 category contains circuit recovery and Push persistence cases; new cases use the category-number allocation below. A reserved ID is never evidence of a successful run. ## Allocation procedure +`RL-WORKER-01` is allocated to `tests/reliability/worker_test.go` for explicit process fencing after worker lease revocation, including container PID 1 exit diagnostics and delayed recovery observation. Allocation is not a test-pass claim. + 1. Inspect the current tree immediately before allocation. Search test source, comments, documentation, and the diff for the exact namespace; do not rely on this catalog to identify availability. 2. Confirm that the behavior is not already represented under another ID. 3. Format Functional numbers as zero-padded two-digit values `01` through `99`, matching the current executable tests (for example, `FN-ID-08`, `FN-MD-20`, and `FN-DC-07`). Choose the lowest unused number after considering concurrent reservations. If a Functional namespace reaches `99`, stop and update this catalog deliberately instead of silently widening the ID. For Reliability, choose the lowest unused category number after considering concurrent reservations. For bounded BVT, Scenario, and Performance namespaces, use only a free ID within the stated range. @@ -38,6 +40,20 @@ A reservation means only "claimed for coordination." It does not mean the test e ## Naming and comments +RL-MESSAGE-02 is allocated to `TestRL_TextFixtureWaitsForPersistence` for Issue #104. It observes broker acceptance while Message is stopped, rejects a fixture return without a durable row, and requires convergence after restoring the consumer. Its cleanup joins the fixture and waits for persistence before deleting synthetic state. Allocation is not evidence that historical RPC 9002 or Identity crashes are resolved. + +FN-TM-02 is allocated to `TestFN_TM_IdempotentRecordWithoutReadableMessage` for Issue #105. It exercises both existing accepted/persisted Redis records against a real healthy Message lookup with no readable row, requires an unavailable response, and preserves the guard without publishing another message. + +RL-MESSAGE-01 is allocated to `TestRL_IdempotentResponseDuringMessageOutage` for Issue #105. It stops Message before or after persistence, observes Transmite directly through the existing internal RPC client, checks both cache formats, and requires complete original results after recovery with one message and one timeline entry per member. Allocation alone is not execution evidence. + +RL-REDIS-01 is allocated to `TestRL_RedisCircuitCleanupAfterAssertionFailure` for Issue #102. It injects a failing assertion in an isolated subprocess of RL-05, then requires an already prepared caller to send successfully after cleanup. RL-05 retains Gateway outage assertions while measuring the single Transmite's Open rejections using per-request counters and direct internal RPC. Allocation is not execution evidence. + +RL-DISCOVERY-01 is allocated to `TestRL_DiscoveryRecoversAfterIdentityAddressChange` for Issue #97. It covers automatic RPC recovery after changing an isolated Compose Identity endpoint's IPv4 address, without restarting callers. The allocation alone is not passing evidence. + +RL-DISCOVERY-02 is allocated to `TestRL_RegistryRecoversAfterLeaseExpiry` for Issue #99. It suspends Identity renewal until the exact etcd registration expires, resumes that same process, and requires the key and authenticated account RPC to recover within 60 seconds. Identity, Gateway and Transmite process identities must remain unchanged. Failure cleanup restores Identity before synthetic data cleanup; allocation alone is not passing evidence. + +FN-AM-07 is allocated to `TestFN_AM_ExpiredJWTClassification` in `tests/func/jwt_expiry_test.go` for Issue #28. It covers signed JWT expiration classification and rejection controls through Identity and Gateway. Its synthetic signing fixture is `tests/pkg/fixture/jwt.go`; execution requires the isolated stack's synthetic `CHATNOW_JWT_CONFIG`. The case ID alone is not a passing verification result. + Use a descriptive Go identifier such as `TestFN_MS_UpdateReadAck_Idempotent`, `TestScenario_MessageSearchES`, or `BenchmarkSendMessage`. Place a concise English case comment immediately above it. This example is an existing BVT identity, not a new reservation: ```go diff --git a/.agents/skills/chatnow-testing/references/framework.md b/.agents/skills/chatnow-testing/references/framework.md index c4cea8a..43e9779 100644 --- a/.agents/skills/chatnow-testing/references/framework.md +++ b/.agents/skills/chatnow-testing/references/framework.md @@ -2,7 +2,7 @@ Target version: `3.0-dev` Status: Current -Verified: 2026-07-22 +Verified: 2026-09-13 Read this reference before selecting, implementing, running, or reporting a test. Reinspect `tests/Makefile`, `.github/workflows/ci.yml`, and the current `tests/` tree before relying on a path, target, or command; executable files are authoritative. @@ -17,14 +17,29 @@ Read this reference before selecting, implementing, running, or reporting a test | L4 Performance | `tests/perf`, `perf` | Throughput and latency baselines | `make -C tests test-perf` | | Reliability | `tests/reliability`, `reliability` | Redis failure injection, recovery, and Push unacked convergence | `make -C tests test-reliability` | -The current `tests/Makefile` also provides `make -C tests proto`, `make -C tests deps`, `make -C tests test-agent-policy`, and `make -C tests clean`. Run `proto` only when generated Go protobuf is required; `clean` removes generated `tests/proto/chatnow` content. Repository policy checks are static evidence and never substitute for a behavior RED or runtime gate. +The current `tests/Makefile` also provides `make -C tests proto`, `make -C tests deps`, `make -C tests test-agent-policy`, and `make -C tests clean`. Run `proto` only when generated Go protobuf is required; `clean` removes generated `tests/proto/chatnow` content. Repository policy checks and `tests/pkg/contracts` Compose/runtime contracts are static evidence and never substitute for a behavior RED or runtime gate. Repository contracts do not consume a BVT/Functional case ID namespace. -The Reliability target runs the complete tagged package and does not consume `TEST_RUN`. For an exact test, invoke the same tagged Go package with an anchored `-run` expression, then run `make -C tests test-reliability` for the layer regression. The current fault controller is Redis-only; there is no RabbitMQ, MySQL, arbitrary-service, or general-network controller. +Reliability requires a dedicated disposable stack. From the repository root, export `COMPOSE_FILE=docker-compose.yml:tests/compose/reliability.yml`, then start it with `TRANSMITE_RATE_LIMIT_USER_MAX=8 TRANSMITE_RATE_LIMIT_SESSION_MAX=40 docker compose up -d --build`. Keep the same Compose files selected for tests and teardown. The override gives the test network an explicit IPv4 subnet (default `172.30.97.0/24`, overridable with `CHATNOW_RELIABILITY_SUBNET` before creating the stack); older Docker engines require this for endpoint move/restore operations. Do not retrofit an existing shared network. The limits make the RL-05 fallback burst deterministic; they are not production recommendations. Run Functional/BVT on their separate default-limit stacks. The Push test uses its issued device ID and restores Redis/paused containers on failure. + +The executable RL-05 cases use Redis pause/unpause to retain DNS while Redis stops responding. They prove process-stall circuit/recovery behavior and Unacked ordering, not container stop/recreate, DNS withdrawal, cluster topology changes or rolling recovery. Compose fault commands have a 20-second execution limit. + +RL-05 preserves the bounded Gateway outage burst but measures fast rejection directly at one isolated Transmite endpoint. Gateway has its own Redis circuit, so its full HTTP duration is not a Transmite Open-state measurement. The pure-Go `DoInternalRPC` helper implements bounded, uncompressed `baidu_std` frames and forwards synthetic fixture `RpcMetadata`; `make proto` includes `common/auth/metadata.proto`. This uses the existing trusted test-network boundary and does not establish internal authentication or a public client API. + +The test deliberately waits past the one-second Open interval to exercise a recovery probe; the delay is fault scheduling, not proof of readiness. For each direct request it records elapsed time separately from bvar reads. Increased connection-failure and open counters identify an admitted failed recovery attempt. Zero connection failures, unchanged open generation, and increased rejections identify an Open rejection; every such sample must take less than 50 ms. A bounded eight-request set must include a recovery probe and three Open rejection samples. Slow qualifying samples fail immediately and cannot be discarded by retrying. Run these tests without concurrent traffic on the same stack. + +RL-05 cleanup unpauses Redis, waits for cluster health, then polls actual account and message RPCs with a 30-second convergence deadline. One recovery idempotency key bounds writes across retries. RL-REDIS-01 injects an assertion failure in a bounded child process, then requires its prepared parent's send to succeed immediately after cleanup. The subsequent Push test remains independent of residual Open circuits. + +The Reliability target runs the complete tagged package and does not consume `TEST_RUN`. For an exact test, invoke the same tagged Go package with an anchored `-run` expression, then run `make -C tests test-reliability` for the layer regression. Current controllers cover Redis, the scoped Identity endpoint move and lease expiry, and Message stop/start; there is no RabbitMQ, MySQL, arbitrary-service, or general-network controller. + +RL-MESSAGE-01 requires exclusive access to the disposable synthetic stack. `tests/pkg/chaos/message.go` stops only the Compose `message_server` with a five-second shutdown grace and registers idempotent restore before fault injection. The test creates one broker-confirmed message while Message is stopped, or stops after persistence, and exercises accepted/persisted guard formats using its unique synthetic sender and client ID. Direct Transmite RPC avoids masking its response behind Gateway's own timeout. Every success must retain the original nonzero identity and sequence; an unavailable result must have no partial message. After restore, it waits for the original durable row and requires successful retries, exactly one message, and one timeline row per member. Cleanup restores Message and waits for the queued write before the existing disposable-stack cleanup removes synthetic state, including on assertion failure. Do not run this test against shared data or concurrently with another suite. Run `go test -tags=reliability ./reliability/... -run '^TestRL_IdempotentResponseDuringMessageOutage$' -v -count=1 -timeout=120s` from `tests`, followed by the complete Reliability gate. FN-TM-02 covers a healthy lookup with no readable original and deletes only its synthetic guard. SC-06 explicitly requires recovery success after persistence instead of accepting a failed duplicate as sufficient evidence. The L0 commands currently encoded in `.github/workflows/ci.yml` are: ```bash -mkdir -p build && cd build && cmake .. && cmake --build . -j$(nproc) +docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci bash -lc ' +cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release +cmake --build build --parallel "$(nproc)" --target conversation_server gateway_server identity_server media_server message_server presence_server push_server relationship_server transmite_server +' cd tests && go vet ./... cd tests unformatted=$(gofmt -l .) @@ -40,22 +55,54 @@ These are Linux workflow commands. On another platform, report the workflow as n CI runs `build` independently and builds reusable service artifacts. BVT needs `service-artifacts`; Functional needs both `service-artifacts` and BVT; Reliability needs `service-artifacts` but does not wait for BVT. Scheduled Performance runs after Functional, while the cache-performance job depends directly on `service-artifacts`. Preserve each gate's actual dependency when changing workflows or selecting local risk checks. -The dedicated Reliability job exists, but the inspected PR run was skipped after an upstream failure. Its existence is executable-surface evidence, not a successful runtime result. +The dedicated Reliability job is executable. Its existence is not a successful runtime result; record the actual current-head result separately from BVT and Functional. + +## Full-stack readiness boundary + +`scripts/wait_for_services.sh` is the bounded pre-suite gate for the root Compose runtime when a full-stack job or operator explicitly invokes it. It verifies Redis Cluster state and slots, all 17 ODB tables and five MySQL application users, RabbitMQ running/alarm state, Elasticsearch yellow-or-green health, MinIO readiness and both media buckets, eight exact etcd service registrations, dependency-aware Gateway `GET /health`, and Push listener reachability. The Push check is TCP reachability, not WebSocket delivery evidence. + +Container health, one-shot initializer completion, and the shared `entrypoint.sh` bounded TCP polling are startup prerequisites; none replaces `scripts/wait_for_services.sh`. Conversely, a passing static contract for the helper or Compose shape does not prove that any container started or that a runtime gate passed. + +The readiness shell contracts in `tests/pkg/contracts/readiness_execution_test.go` execute the real helper with controlled external command boundaries. They verify stdin-only MinIO credentials and interruption of a stalled Docker probe within the deadline, without accessing real credentials or services. GNU `timeout` limits each Compose invocation to the remaining budget, followed by a one-second kill grace period. These process-boundary tests do not replace full-stack readiness or BVT. + +Issue #88 integrates the Issue #78 runtime into CI. Each job owns a fresh checkout and its middle/data directory. scripts/create_test_env.py generates synthetic credentials and refuses existing state. Until a fresh run exists for the exact commit, report cold start, BVT, Functional, Reliability, and Performance as `not run` or `blocked`, not passed. ## Shared framework +Issue #104 distinguishes message acceptance from fixture readiness. `fixture.SendTextMessage` submits exactly once, then uses `DBVerifier.WaitMessageExists` with a ten-second convergence budget before returning. The Message consumer commits the row, member timelines, and conversation watermark in one transaction. Message API tests delegate to this shared fixture; explicit client-ID and outage tests retain `SendTextMessageWithClientMsgId` or direct RPC when broker acceptance is the behavior under test. A failed send is never retried by the fixture. `TestSearchMessages_Success` waits for the exact synthetic message's Elasticsearch index entry and then requires a successful search containing that message; receiving a failure envelope is not a search success. + +RL-MESSAGE-02 (`TestRL_TextFixtureWaitsForPersistence`) stops the disposable stack's Message consumer, starts one text fixture, observes that sender's `accepted` Redis guard, and checks that the fixture cannot return with zero durable rows. A 250 ms negative observation window applies only while the fault is established; recovery waits for actual MySQL state. Cleanup restores Message, joins the fixture goroutine, waits for the accepted row and service readiness, then removes synthetic data. Run `go test -tags=reliability ./reliability/... -run '^TestRL_TextFixtureWaitsForPersistence$' -v -count=1 -timeout=120s`, followed by the complete Reliability gate. This case proves fixture persistence timing, not the cause of historical RPC 9002 errors. + +With `CHATNOW_TEST_DIAGNOSTICS=1`, the shared HTTP client emits one sanitized JSON event for each failed Protobuf response, preserves the original response, and performs no retry. It records success, business error code, header/request-ID presence, classified error text, numeric brpc codes, and a locally generated trace ID where the caller supplied none. Raw error text, response request IDs, authorization headers, and message bodies are omitted. `TestHTTPFailureDiagnostics` exercises this client boundary, including secret exclusion and exactly one request. + +BVT, Functional and Reliability CI jobs capture initial process/registration state before the suite and collect failure state before teardown. `scripts/collect_test_diagnostics.py` reads only selected Docker process fields, exact etcd registration presence and filtered Gateway/Message/Transmite/Identity error events. Each subprocess has a two-second limit within a shared twenty-second budget. Unavailable evidence is explicitly marked, never inferred as a healthy service. The collector retains at most 500 events from bounded log tails; it never copies raw logs, container configuration, environment, registration values, user/device IDs or command stderr. Failed jobs upload `test-diagnostics` for seven days. Compare initial and failure timestamps/PIDs/restart counts; Reliability deliberately changes process state and must be interpreted with its fault schedule. This is diagnostic coverage, not a claim that an unreproduced historical crash is fixed. + +RL-MESSAGE-01 intentionally starts Message again during recovery; it does not claim recovery of an expired registry lease in a surviving process (Issue #99). The recovery requests poll the actual duplicate response within ten seconds and preserve its original client ID. Initial and final stack-readiness checks reject a missing registration; a database row alone is not RPC-readiness evidence. + - `tests/pkg/client`: configuration plus shared HTTP and WebSocket clients. Use `client.NewRequestID()` and `client.NewDeviceID()` for collision-resistant request, idempotency, device, and test-data suffixes. - `tests/pkg/fixture`: reusable authenticated users, friendships, conversations, groups, messages, media, and WebSocket setup. Extend a fixture instead of copying setup. - `tests/pkg/cleanup`: stack-readiness polling and suite cleanup. `tests/bvt/setup_test.go` and `tests/func/setup_test.go` call it from `TestMain`. - `tests/pkg/verify`: direct MySQL, Elasticsearch, and MinIO checks. Use these when an API success alone cannot prove persistence, indexing, object state, idempotency, or cross-store convergence. +Device fixtures must use the device ID issued by Identity. A second WebSocket payload using the same JWT does not create a second authenticated device; log in separately for each device. BVar reads request the brpc console representation and parse the named integer response. MinIO verification needs `MINIO_ENDPOINT` as well as the synthetic access credentials. Rate-limit correctness can initialize a test-owned exhausted bucket, while throughput belongs in Performance. Message persistence and search assertions wait for their bounded observable state rather than assuming the send response or index write implies immediate visibility. + Use unique IDs for every request and collision-prone resource. Do not rely on a fixed username, message idempotency key, group name, device ID, file key, or search token shared across runs. +`tests/pkg/fixture/jwt.go` signs temporal and invalid-signature variants of fixture-account JWTs for FN-AM-07. It requires `CHATNOW_JWT_CONFIG` from the isolated synthetic stack (CI exports the configuration generated by `scripts/create_test_env.py`). Supply the same synthetic environment to local Go tests; never use production keys. The fixture keeps keys and token contents out of diagnostics. The test checks Identity error codes and Gateway rejection, including 30-second, 120-second and one-day expiration, bad signatures, unknown keys, future `iat`/`nbf`, and valid-token controls. The older fixed invalid-token rejection test is named `TestJWTRequired_InvalidToken`; it is not expiration evidence. + ## Waiting and cleanup +RL-WORKER-01 (`TestRL_WorkerLeaseLossFencesProcess`) uses `tests/pkg/chaos/worker.go` only on a dedicated synthetic Compose stack. It requires a single running Transmite and etcd in the same project and exactly one leased worker slot, records the original process and restart policy, registers cleanup before mutation, disables automatic restart, and revokes that sole lease. It observes the original process for up to 20 seconds and requires exit code 1, no OOM, and the fixed worker-loss diagnostic. Raw container logs remain in memory; test output contains only process/lease metadata and the diagnostic's presence. Cleanup restores the exact restart policy and starts the service on both RED and GREEN, followed by full-stack readiness. GREEN additionally observes the recovered process for seven seconds, covering five TTL-check intervals and a watchdog interval. Run `go test -tags=reliability ./reliability/... -run '^TestRL_WorkerLeaseLossFencesProcess$' -v -count=1 -timeout=120s`, then the complete Reliability layer and BVT/Functional/Scenario gates. Never overlap faults or suites on the same stack. This case does not prove arbitrary network-partition fencing, change the existing lease-loss detector, or cover allocator exhaustion. + +Before moving Identity, the address controller creates an inert container from the already available Identity image, overrides its entrypoint with `/bin/true`, and checks an explicit address on the selected test network. It prefers high host addresses and allows at most 16 probes when Docker reports an address reservation absent from network inspection. It removes each probe before retrying or performing any live endpoint mutation. Unsupported networks therefore fail without disconnecting Identity. `TestAddressFaultRejectsUnsupportedNetworkBeforeDisconnect` exercises create/start rejection and an occupied-candidate retry across the Docker command boundary; the real Reliability case remains the runtime acceptance test. + +RL-DISCOVERY-01 (`TestRL_DiscoveryRecoversAfterIdentityAddressChange`) uses `tests/pkg/chaos/address.go` to move only Identity's IPv4 endpoint in the selected isolated Compose project. It requires one Identity network owned by that project, preserves aliases, selects an unused address, bounds Docker commands, and restores the original address and aliases with `t.Cleanup`, including on RED. Identity is restarted at both the changed and restored addresses to reconnect its outgoing datastore clients and registry lease; Gateway and Transmite process identities must remain unchanged. Account/profile RPC and an uncached sender's message must recover within a shared 45-second budget. Run it with `go test -tags=reliability ./reliability/... -run '^TestRL_DiscoveryRecoversAfterIdentityAddressChange$' -v -count=1 -timeout=180s`, then the complete Reliability gate. Never run network fault tests concurrently with other suites on the same stack. This controller does not cover shared/external networks, IPv6 fault injection, registry lease recovery without a service restart, full container recreation or Redis address changes. + +RL-DISCOVERY-02 (`TestRL_RegistryRecoversAfterLeaseExpiry`) uses `tests/pkg/chaos/lease.go` on a dedicated disposable stack. It records Identity, Gateway and Transmite process identities in one Compose project, pauses only Identity, and polls the exact `/service/identity_service/instance` key for up to 45 seconds until the real lease has expired. It then unpauses the same process and requires both registration and authenticated profile RPC within 60 seconds. Process preservation is checked before cleanup can repair a failed baseline by restarting Identity. Cleanup is registered before pause, restores the service and checks RPC convergence before deleting synthetic data. Run `go test -tags=reliability ./reliability/... -run '^TestRL_RegistryRecoversAfterLeaseExpiry$' -v -count=1 -timeout=150s` from `tests`, then the complete Reliability gate. This test proves lease expiry recovery, not a general network partition or all concurrent unregister schedules. + Poll the externally observable condition with a bounded deadline and useful failure message. Suitable conditions include service reachability, a WebSocket event, a database row/state, an Elasticsearch hit, a MinIO object, or an API state transition. A polling interval is allowed; a fixed delay used as proof of readiness or convergence is not. -Assign exactly one owner for each created resource. Prefer suite-level cleanup through `tests/pkg/cleanup`; add `t.Cleanup` for per-test resources such as clients, sockets, temporary objects, or state that suite cleanup cannot safely own. Cleanup must run on assertion failure. CI owns `docker compose down -v` in its full-stack jobs. +Assign exactly one owner for each created resource. Prefer suite-level cleanup through `tests/pkg/cleanup`; add `t.Cleanup` for per-test resources such as clients, sockets, temporary objects, or state that suite cleanup cannot safely own. Cleanup must run on assertion failure. Root Compose persists infrastructure through bind mounts under `middle/data`; `docker compose down -v` does not remove that state. A clean-slate test must use a fresh CI checkout or another explicitly disposable storage path and must never delete a shared tree. ## Change-to-layer matrix @@ -67,7 +114,9 @@ Assign exactly one owner for each created resource. Prefer suite-level cleanup t | Cross-service flow, MQ/WebSocket delivery, idempotency, ordering, or MySQL/Elasticsearch/MinIO consistency | L3 Scenario | Also run affected L2 and L1 gates. | | Throughput, latency, allocation, or benchmark threshold | L4 Performance | Also run correctness layers for behavior used by the benchmark. | | Redis failure injection, restart recovery, or Push unacked convergence | Reliability | Run the exact tagged test, then `make -C tests test-reliability`; also run lower correctness layers selected by the affected behavior. | -| RabbitMQ/MySQL/service/network fault behavior | Reliability | No current controller exists for these faults. Do not expand the framework unless the scoped Issue authorizes it; use the nearest executable correctness layer and report the gap. | +| Identity hostname address recovery | Reliability | Use RL-DISCOVERY-01 and its scoped Compose endpoint controller, then lower correctness gates. | +| Expired service registration in a surviving process | Reliability | Use RL-DISCOVERY-02 and its scoped Identity lease controller, then the complete layer and lower correctness gates. | +| Other RabbitMQ/MySQL/service/network fault behavior | Reliability | No general controller exists for these faults. Do not expand the framework unless the scoped Issue authorizes it; use the nearest executable correctness layer and report the gap. | Choose the lowest layer that can fail for the required behavior, not the cheapest layer that happens to run. Run the target test first, its same-layer regressions second, and broader layers according to risk. diff --git a/.agents/skills/chatnow-verifying-changes/SKILL.md b/.agents/skills/chatnow-verifying-changes/SKILL.md index 031a661..1ac506e 100644 --- a/.agents/skills/chatnow-verifying-changes/SKILL.md +++ b/.agents/skills/chatnow-verifying-changes/SKILL.md @@ -32,7 +32,7 @@ Inspect the current `tests/Makefile`, `.github/workflows/ci.yml`, and test tree | 3 | Target test | Run the smallest exact test that proves the changed behavior. | | 4 | Same-layer regression | Run the affected package or complete layer after the target passes. | | 5 | BVT, Functional, Scenario | Run each applicable correctness gate; record them separately rather than collapsing them into "tests." | -| 6 | Performance, Reliability | Run the risk-relevant layer. Reliability is reserved in the current architecture: inspect for an executable surface, but never invent a directory, tag command, Make target, or successful run. | +| 6 | Performance, Reliability | Run the risk-relevant layer. Redis-focused Reliability is executable through `make -C tests test-reliability`; inspect the current fault controller and report uncovered fault categories without inventing commands or successful runs. | | 7 | CI | Record the current commit's actual workflow and job results; an old green run is not evidence for the current commit. | Do not skip a lower rung because a higher rung passed. A broad check does not replace the exact target result, and a target pass does not replace regressions. @@ -58,7 +58,7 @@ Use statuses literally: - `not run`: it was not executed or does not apply; state which and why. Never attach a predicted outcome. - `blocked`: it is required and applicable, but an unavailable tool, service, credential, platform, or environment prevented execution; name the blocker and the exact command that remains. -For CI, include the commit SHA, workflow/run identity, job results, and failing or pending details. For an unavailable reserved Reliability layer, leave `Command` as `No executable command exists in the current repository`, set `not run`, and explain whether its absence is a readiness gap for this change. +For CI, include the commit SHA, workflow/run identity, job results, and failing or pending details. For a required fault category without an executable controller, leave `Command` as `No executable command exists for this fault category`, set `not run`, and explain the readiness gap. Do not use that exception to omit the existing Redis Reliability gate. Do not replace full output with "clean," "looks good," a success count, an agent summary, or "see above." Preserve the exact command, exit status, and complete stdout/stderr in the verification evidence or an identified durable log artifact. diff --git a/.github/workflows/agent-policy.yml b/.github/workflows/agent-policy.yml index 48ccf49..9b0b940 100644 --- a/.github/workflows/agent-policy.yml +++ b/.github/workflows/agent-policy.yml @@ -29,7 +29,8 @@ jobs: - name: Build trusted policy binary run: cd tests && go build -o "$RUNNER_TEMP/agent-policy" ./cmd/agent-policy - name: Validate Issue - run: "$RUNNER_TEMP/agent-policy" issue + run: | + "$RUNNER_TEMP/agent-policy" issue pull-request-policy: if: github.event_name == 'pull_request' @@ -68,14 +69,19 @@ jobs: echo 'AGENT_POLICY_COMMITS_EOF' } >> "$GITHUB_ENV" - name: Validate branch - run: "$RUNNER_TEMP/agent-policy" branch + run: | + "$RUNNER_TEMP/agent-policy" branch - name: Validate commit subjects - run: "$RUNNER_TEMP/agent-policy" commits + run: | + "$RUNNER_TEMP/agent-policy" commits - name: Validate pull request body - run: "$RUNNER_TEMP/agent-policy" pull-request + run: | + "$RUNNER_TEMP/agent-policy" pull-request - name: Validate Skill synchronization - run: "$RUNNER_TEMP/agent-policy" skill-sync + run: | + "$RUNNER_TEMP/agent-policy" skill-sync - name: Validate Skill packages env: AGENT_POLICY_REPO_ROOT: ${{ github.workspace }} - run: "$RUNNER_TEMP/agent-policy" skills + run: | + "$RUNNER_TEMP/agent-policy" skills diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1cfb6ab..2d6222d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,5 +1,8 @@ name: CI +env: + CHATNOW_TEST_DIAGNOSTICS: '1' + on: push: branches: [main, develop, 3.0-dev] @@ -16,9 +19,7 @@ jobs: - name: Install dependencies run: | sudo apt-get update - sudo apt-get install -y cmake build-essential protobuf-compiler netcat-openbsd - - name: Build C++ services - run: mkdir -p build && cd build && cmake .. && cmake --build . -j$(nproc) + sudo apt-get install -y protobuf-compiler netcat-openbsd - uses: actions/setup-go@v5 with: go-version: '1.24' @@ -30,6 +31,10 @@ jobs: run: cd tests && go mod download - name: Validate CI gate contracts run: cd tests && go test ./pkg/contracts -count=1 + - name: Validate fault-controller boundaries + run: cd tests && go test ./pkg/chaos -count=1 + - name: Validate internal RPC transport + run: cd tests && go test ./pkg/client -count=1 - name: Go vet run: cd tests && go vet ./... - name: Go fmt check @@ -112,6 +117,8 @@ jobs: - &validate-compose-artifacts name: Validate downloaded Compose service artifacts run: docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts + - name: Initialize synthetic test environment + run: python3 scripts/create_test_env.py --github-env - name: Start full stack run: docker compose up -d --build - name: Wait for services @@ -120,8 +127,23 @@ jobs: run: cd tests && make proto - name: Download Go deps run: cd tests && go mod download + - &before-diagnostics + name: Capture initial service state + run: python3 scripts/collect_test_diagnostics.py --output test-diagnostics/before.json - name: Run BVT smoke tests run: cd tests && make test-bvt + - &failure-diagnostics + name: Collect failure diagnostics + if: failure() + run: python3 scripts/collect_test_diagnostics.py --output test-diagnostics/failure.json + - &upload-diagnostics + name: Preserve sanitized failure diagnostics + if: failure() + uses: actions/upload-artifact@v4 + with: + name: test-diagnostics-${{ github.job }} + path: test-diagnostics + retention-days: 7 - name: Tear down if: always() run: docker compose down -v @@ -145,6 +167,8 @@ jobs: - *restore-compose-artifacts - *chmod-compose-artifacts - *validate-compose-artifacts + - name: Initialize synthetic test environment + run: python3 scripts/create_test_env.py --github-env - name: Start full stack run: docker compose up -d --build - name: Wait for services @@ -153,8 +177,11 @@ jobs: run: cd tests && make proto - name: Download Go deps run: cd tests && go mod download + - *before-diagnostics - name: Run functional tests run: cd tests && make test-func + - *failure-diagnostics + - *upload-diagnostics - name: Tear down if: always() run: docker compose down -v @@ -163,6 +190,8 @@ jobs: needs: service-artifacts runs-on: ubuntu-22.04 if: github.event_name == 'pull_request' || github.event_name == 'schedule' + env: + COMPOSE_FILE: docker-compose.yml:tests/compose/reliability.yml steps: - uses: actions/checkout@v4 - uses: actions/setup-go@v5 @@ -178,7 +207,12 @@ jobs: - *restore-compose-artifacts - *chmod-compose-artifacts - *validate-compose-artifacts - - name: Start full stack + - name: Initialize synthetic test environment + run: python3 scripts/create_test_env.py --github-env + - name: Start full stack with RL-05 rate limits + env: + TRANSMITE_RATE_LIMIT_USER_MAX: '8' + TRANSMITE_RATE_LIMIT_SESSION_MAX: '40' run: docker compose up -d --build - name: Wait for services run: ./scripts/wait_for_services.sh @@ -186,8 +220,11 @@ jobs: run: cd tests && make proto - name: Download Go deps run: cd tests && go mod download + - *before-diagnostics - name: Run cache reliability gate run: cd tests && make test-reliability + - *failure-diagnostics + - *upload-diagnostics - name: Tear down if: always() run: docker compose down -v @@ -211,6 +248,8 @@ jobs: - *restore-compose-artifacts - *chmod-compose-artifacts - *validate-compose-artifacts + - name: Initialize synthetic test environment + run: python3 scripts/create_test_env.py --github-env - name: Start full stack with PF-09 rate limits env: # Transmite flags are int32; use the maximum valid value for gate-only headroom. @@ -230,7 +269,7 @@ jobs: run: docker compose down -v perf: - needs: func + needs: [service-artifacts, func] runs-on: ubuntu-22.04 if: github.event_name == 'schedule' steps: @@ -244,6 +283,12 @@ jobs: sudo apt-get install -y protobuf-compiler netcat-openbsd - name: Install Go protobuf generator run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11 + - *download-compose-artifacts + - *restore-compose-artifacts + - *chmod-compose-artifacts + - *validate-compose-artifacts + - name: Initialize synthetic test environment + run: python3 scripts/create_test_env.py --github-env - name: Start full stack run: docker compose up -d --build - name: Wait for services diff --git a/README.md b/README.md index 3c82b36..4053c70 100644 --- a/README.md +++ b/README.md @@ -45,14 +45,21 @@ sudo apt install -y build-essential cmake libprotobuf-dev libbrpc-dev \ sudo bash scripts/install_aws_sdk_linux.sh ``` -### 2. 启动中间件 +### 2. Start the Compose runtime + +The root Compose profile contains the infrastructure and all nine ChatNow processes. Prepare synthetic local secret inputs as described in [Runtime Secret Management](docs/operations/runtime-secrets.md); never use production values in this profile. ```bash -# 一键启动: MySQL + Redis Cluster(6n) + ES + RabbitMQ + etcd + MinIO -docker compose up -d +docker compose config +docker compose up -d --build +./scripts/wait_for_services.sh ``` -### 3. 编译 +Compose expects the nine service binaries and their dependency directories to be prepared first, as described in [Compose Runtime Operations](docs/operations/compose-runtime.md). CI builds and restores those artifacts into each disposable test job. Consult PR #89 for current gate results; this local profile is not a production deployment. + +### 3. Optional native build + +The CI builder compiles the services before Compose packages their runtime images. For host-side development, build the native targets first: ```bash mkdir build && cd build @@ -60,7 +67,7 @@ cmake .. cmake --build . -j$(nproc) ``` -### 4. 启动服务 +### 4. Optional native service startup ```bash # 例: 以 flagfile 启动某个服务 @@ -174,7 +181,8 @@ ChatNow/ | [设计 Spec](docs/superpowers/specs/) | 20+ 份设计文档，覆盖缓存 / MQ / Proto / 可靠性 | | [实施 Plan](docs/superpowers/plans/) | 15+ 份实施计划，按分支独立 | | [运维 Runbook](docs/operations/) | JWT 轮换 / 日志规范 / 监控 / 烟雾测试 | -| [API 交接](API_HANDOVER.md) | 客户端 SDK 契约 & 错误码 | +| [Compose runtime operations](docs/operations/compose-runtime.md) | Root topology, bootstrap, semantic readiness, persistence, and evidence status | +| [Client retry and error handling](docs/client-sdk/error-retry.md) | Client retry boundaries and error categories | ### 运维 diff --git a/common/auth/jwt_codec.hpp b/common/auth/jwt_codec.hpp index bdd3b40..2bbc198 100644 --- a/common/auth/jwt_codec.hpp +++ b/common/auth/jwt_codec.hpp @@ -173,7 +173,11 @@ inline JwtClaims JwtCodec::verify(const std::string& token, bool require_refresh if (decoded.get_expires_at() <= now) { // 仍需先校验签名，避免泄漏过期 token 也被接受的可能。 try { - auto v = jwt::verify().allow_algorithm(jwt::algorithm::hs256{it->second}).leeway(60); + // This branch always rejects expiration below. Verify the signature + // and other time claims without reclassifying an old exp as invalid. + auto v = jwt::verify().allow_algorithm(jwt::algorithm::hs256{it->second}) + .leeway(0) + .with_claim("exp", [](const auto&, std::error_code&) {}); v.verify(decoded); } catch (const std::exception&) { throw ServiceError(kAuthTokenInvalid, "signature invalid"); diff --git a/common/dao/data_es.hpp b/common/dao/data_es.hpp index 95f8a4a..1338b5a 100644 --- a/common/dao/data_es.hpp +++ b/common/dao/data_es.hpp @@ -317,7 +317,7 @@ class ESConversation builder.append_should_match("chat_session_name", key) .append_should_match("chat_session_id.keyword", key) .append_must_term("status", std::to_string(0)) - .append_must_term("member_ids", caller_uid) + .append_must_term("member_ids.keyword", caller_uid) .sort_by("update_time", "desc") .page(0, size); Json::Value json_session = builder.search(); diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp index 59a6027..ea97369 100644 --- a/common/dao/data_redis.hpp +++ b/common/dao/data_redis.hpp @@ -62,6 +62,14 @@ inline bool is_redis_pool_wait_error(const sw::redis::Error &error) noexcept { return true; } +inline bool is_redis_availability_error(const sw::redis::Error &error) noexcept { + if (is_redis_pool_wait_error(error)) return true; + // Pinned redis++ wraps exhausted topology-refresh attempts in a base Error. + // Keep command ReplyError/WRONGTYPE and other typed errors out of the breaker. + return typeid(error) == typeid(sw::redis::Error) && + std::string_view(error.what()) == "Failed to update shards info"; +} + class RedisPipeline { public: RedisPipeline(sw::redis::Pipeline pipeline, @@ -128,7 +136,7 @@ class RedisPipeline { settle_success_(); throw; } catch (const sw::redis::Error &error) { - if (is_redis_pool_wait_error(error)) record_connection_failure_(); + if (is_redis_availability_error(error)) record_connection_failure_(); else abandon_(); throw; } catch (...) { @@ -153,7 +161,7 @@ class RedisPipeline { record_connection_failure_(); throw; } catch (const sw::redis::Error &error) { - if (is_redis_pool_wait_error(error)) record_connection_failure_(); + if (is_redis_availability_error(error)) record_connection_failure_(); else abandon_(); throw; } catch (...) { @@ -343,7 +351,7 @@ class RedisClient record_success_(permit); throw; } catch (const sw::redis::Error &error) { - if (is_redis_pool_wait_error(error)) record_connection_failure_(permit); + if (is_redis_availability_error(error)) record_connection_failure_(permit); else abandon_(permit); throw; } catch (...) { @@ -436,7 +444,7 @@ class RedisClient record_success_(permit); throw; } catch (const sw::redis::Error &error) { - if (is_redis_pool_wait_error(error)) record_connection_failure_(permit); + if (is_redis_availability_error(error)) record_connection_failure_(permit); else abandon_(permit); throw; } catch (...) { @@ -461,7 +469,7 @@ class RedisClient record_success_(permit); throw; } catch (const sw::redis::Error &error) { - if (is_redis_pool_wait_error(error)) record_connection_failure_(permit); + if (is_redis_availability_error(error)) record_connection_failure_(permit); else abandon_(permit); throw; } catch (...) { @@ -954,6 +962,7 @@ class Members } catch(std::exception &e) { LOG_ERROR("Members.list_snapshot 失败 {}: {}", ssid, e.what()); snap.stable = false; + snap.version = kUnknownCacheVersion; } return snap; } diff --git a/common/dao/mysql_conversation.hpp b/common/dao/mysql_conversation.hpp index c7f1cc7..3a7e8e7 100644 --- a/common/dao/mysql_conversation.hpp +++ b/common/dao/mysql_conversation.hpp @@ -47,7 +47,7 @@ class ConversationTable odb::transaction trans(_db->begin()); using query = odb::query; std::shared_ptr c(_db->query_one( - query::conversation_id == cid)); + (query::conversation_id == cid) + " FOR UPDATE")); if(!c) { trans.commit(); return false; @@ -116,6 +116,13 @@ class ConversationTable try { c->update_time(boost::posix_time::microsec_clock::universal_time()); odb::transaction trans(_db->begin()); + using query = odb::query; + std::shared_ptr current(_db->query_one( + (query::conversation_id == c->conversation_id()) + " FOR UPDATE")); + if (!current) return false; + // Metadata was read before this transaction. Preserve the watermark + // committed by Message while this request was in flight. + if (c->max_seq() < current->max_seq()) c->max_seq(current->max_seq()); _db->update(*c); trans.commit(); } catch(std::exception &e) { diff --git a/common/infra/etcd.hpp b/common/infra/etcd.hpp index a3ffd1a..da35a7b 100644 --- a/common/infra/etcd.hpp +++ b/common/infra/etcd.hpp @@ -5,22 +5,27 @@ * etcd 服务注册 / 发现封装 * --------------------------------------------------------------------------- * 设计要点： - * 1. Registry: 改用 lease keep-alive；析构走 Cancel（旧版调 Lease() 是错的） + * 1. Registry uses bounded lease replacement, with serialized shutdown. * 2. Discovery: 监听 basedir 下的 PUT/DELETE 事件，回调送给 ServiceManager * 3. 日志统一改为 LOG_xxx 宏（旧版用了 SPDLOG_xxx 不走我们的格式） * 4. 新增 Registry::unregister()：服务退出前主动撤销，避免依赖 lease 过期 - * 5. KeepAlive 间隔 3s 写死过短；改为 30s lease + 默认 keepalive 内置心跳 + * 5. Replace 30-second leases every 10 seconds; retry failures after one second. * =========================================================================== */ #include -#include #include #include #include +#include +#include +#include #include #include +#include +#include #include +#include #include "infra/logger.hpp" namespace chatnow @@ -30,7 +35,7 @@ inline constexpr int kLeaseSeconds = 30; // 30s lease，避免 3s 过短的瞬 /* brief: 服务注册客户端 * - 启动时调 registry(key, host) 写入实例信息 - * - lease 周期内由后台 keep-alive 自动续期 + * - A worker renews registration by publishing a fresh lease before expiry. * - 进程退出前应主动调 unregister() 撤销，让上游服务发现立即感知下线 */ class Registry @@ -39,20 +44,18 @@ class Registry using ptr = std::shared_ptr; explicit Registry(const std::string &host) - : _client(std::make_shared(host)), - _keep_alive(_client->leasekeepalive(kLeaseSeconds).get()), - _lease_id(_keep_alive->Lease()) {} - - ~Registry() { - try { - // 撤销 lease，关联的 key 立刻失效；旧实现错调 Lease() 实际是 getter - _keep_alive->Cancel(); - _client->leaserevoke(_lease_id).wait(); - } catch(...) { /* 析构吞异常 */ } + : _client(std::make_shared(host)) { + _client->set_grpc_timeout(std::chrono::seconds(2)); + _lease_id = make_lease_(); + _recovery_thread = std::thread([this] { recover_(); }); } + ~Registry() { unregister(); } + /* brief: 注册服务实例 (key, value=host) */ bool registry(const std::string &key, const std::string &val) { + std::lock_guard lock(_mutex); + if (_stopping) return false; try { auto resp = _client->put(key, val, _lease_id).get(); if(!resp.is_ok()) { @@ -60,6 +63,7 @@ class Registry return false; } _registered_key = key; + _registered_value = val; return true; } catch(std::exception &e) { LOG_ERROR("注册数据异常 key={}: {}", key, e.what()); @@ -67,22 +71,81 @@ class Registry } } - /* brief: 主动注销 — 进程优雅退出时应在主循环停止后调用 */ - void unregister() { - if(_registered_key.empty()) return; + // Stop recovery before removing the owned registration. Revoke only our + // lease, so shutdown cannot delete a replacement instance's newer value. + void unregister() noexcept { try { - _client->rm(_registered_key).wait(); - _registered_key.clear(); - } catch(std::exception &e) { - LOG_WARN("主动注销失败 key={}: {}", _registered_key, e.what()); - } + std::call_once(_shutdown_once, [this] { + { + std::lock_guard lock(_mutex); + _stopping = true; + _registered_key.clear(); + _registered_value.clear(); + } + _wake.notify_all(); + if (_recovery_thread.joinable()) _recovery_thread.join(); + discard_lease_(_lease_id); + }); + } catch (...) {} } private: + int64_t make_lease_() { + auto granted = _client->leasegrant(kLeaseSeconds).get(); + if (!granted.is_ok() || granted.value().lease() == 0) { + throw std::runtime_error("registry lease grant failed"); + } + return granted.value().lease(); + } + + void discard_lease_(int64_t lease_id) noexcept { + try { if (lease_id != 0) _client->leaserevoke(lease_id).get(); } catch (...) {} + } + + void recover_() { + std::unique_lock lock(_mutex); + auto interval = std::chrono::seconds(kLeaseSeconds / 3); + bool retrying = false; + while (!_stopping) { + _wake.wait_for(lock, interval, [this] { return _stopping; }); + if (_stopping) break; + if (_registered_key.empty()) continue; + try { + // The pinned KeepAlive implementation has unbounded stream + // creation/cancellation waits. Use only deadline-bound unary + // calls: publish the same value under a fresh lease before + // revoking the previous one. Existing channels deduplicate PUT. + auto replacement = make_lease_(); + try { + auto response = _client->put(_registered_key, _registered_value, replacement).get(); + if (!response.is_ok()) throw std::runtime_error("registry recovery put failed"); + } catch (...) { + discard_lease_(replacement); + throw; + } + auto previous = _lease_id; + _lease_id = replacement; + discard_lease_(previous); + if (retrying) LOG_INFO("registry_recovery outcome=restored"); + retrying = false; + interval = std::chrono::seconds(kLeaseSeconds / 3); + } catch (const std::exception &) { + retrying = true; + interval = std::chrono::seconds(1); + LOG_WARN("registry_recovery outcome=retry"); + } + } + } + std::shared_ptr _client; - std::shared_ptr _keep_alive; - uint64_t _lease_id; - std::string _registered_key; // 注册的 key，析构 / 主动注销时使用 + int64_t _lease_id{0}; + std::string _registered_key; + std::string _registered_value; + std::mutex _mutex; + std::condition_variable _wake; + bool _stopping{false}; + std::once_flag _shutdown_once; + std::thread _recovery_thread; }; /* brief: 服务发现客户端 diff --git a/common/infra/s3_client.hpp b/common/infra/s3_client.hpp index 6a749e3..cd6b392 100644 --- a/common/infra/s3_client.hpp +++ b/common/infra/s3_client.hpp @@ -47,6 +47,7 @@ namespace chatnow { struct S3Options { std::string endpoint; + std::string public_endpoint; std::string region {"us-east-1"}; std::string access_key; std::string secret_key; @@ -56,17 +57,24 @@ struct S3Options { class S3Client { public: explicit S3Client(const S3Options& o) : _opt(o) { - Aws::Client::ClientConfiguration cfg; - cfg.endpointOverride = o.endpoint; - cfg.scheme = (o.endpoint.rfind("https", 0) == 0) - ? Aws::Http::Scheme::HTTPS : Aws::Http::Scheme::HTTP; - cfg.region = o.region; - cfg.verifySSL = false; - _client = std::make_shared( - Aws::Auth::AWSCredentials(o.access_key, o.secret_key), - cfg, - Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, - o.use_path_style); + auto make_client = [&o](const std::string& endpoint) { + Aws::Client::ClientConfiguration cfg; + cfg.endpointOverride = endpoint; + cfg.scheme = (endpoint.rfind("https", 0) == 0) + ? Aws::Http::Scheme::HTTPS : Aws::Http::Scheme::HTTP; + cfg.region = o.region; + cfg.verifySSL = false; + return std::make_shared( + Aws::Auth::AWSCredentials(o.access_key, o.secret_key), + cfg, + Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, + !o.use_path_style); + }; + + _client = make_client(o.endpoint); + _presign_client = o.public_endpoint.empty() || o.public_endpoint == o.endpoint + ? _client + : make_client(o.public_endpoint); } /* brief: 签发 PUT presigned URL，客户端按 headers 直传 */ @@ -75,15 +83,15 @@ class S3Client { const std::map& headers) const { Aws::Http::HeaderValueCollection h; for (const auto& kv : headers) h.emplace(kv.first, kv.second); - auto url = _client->GeneratePresignedUrlWithSSEC( - bucket, key, Aws::Http::HttpMethod::HTTP_PUT, h, /*sseKey*/"", seconds); + auto url = _presign_client->GeneratePresignedUrl( + bucket, key, Aws::Http::HttpMethod::HTTP_PUT, h, seconds); if (url.empty()) throw_failed("presigned_put empty url"); return url; } /* brief: 签发 GET presigned URL（公共/私密资源都用） */ std::string presigned_get(const std::string& bucket, const std::string& key, int seconds) const { - auto url = _client->GeneratePresignedUrl( + auto url = _presign_client->GeneratePresignedUrl( bucket, key, Aws::Http::HttpMethod::HTTP_GET, seconds); if (url.empty()) throw_failed("presigned_get empty url"); return url; @@ -160,11 +168,18 @@ class S3Client { std::string presigned_part(const std::string& bucket, const std::string& key, const std::string& upload_id, int part_number, int seconds) const { - auto url = _client->GeneratePresignedUrl( + // Resolve the configured endpoint/bucket style with the S3 client, then + // sign the complete query. Adding upload parameters after signing invalidates SigV4. + auto resolved = _presign_client->GeneratePresignedUrl( bucket, key, Aws::Http::HttpMethod::HTTP_PUT, seconds); + if (resolved.empty()) throw_failed("presigned_part empty endpoint"); + Aws::Http::URI uri(resolved); + uri.SetQueryString(""); + uri.AddQueryStringParameter("partNumber", std::to_string(part_number)); + uri.AddQueryStringParameter("uploadId", upload_id); + auto url = _presign_client->Aws::Client::AWSClient::GeneratePresignedUrl( + uri, Aws::Http::HttpMethod::HTTP_PUT, _opt.region.c_str(), "s3", seconds); if (url.empty()) throw_failed("presigned_part empty url"); - url += "&partNumber=" + std::to_string(part_number) + - "&uploadId=" + upload_id; return url; } @@ -241,6 +256,7 @@ class S3Client { S3Options _opt; std::shared_ptr _client; + std::shared_ptr _presign_client; }; } // namespace chatnow diff --git a/common/log/log_context.hpp b/common/log/log_context.hpp index bd57483..cac0b58 100644 --- a/common/log/log_context.hpp +++ b/common/log/log_context.hpp @@ -25,7 +25,7 @@ #include -#include +#include #include namespace chatnow::log { @@ -48,7 +48,10 @@ inline bthread_key_t& log_fields_key() { struct KeyInit { bthread_key_t key; KeyInit() { - assert(0 == bthread_key_create(&key, &log_fields_dtor)); + // Release builds must initialize the key too; assert removes its expression. + if (bthread_key_create(&key, &log_fields_dtor) != 0) { + std::abort(); + } } }; static KeyInit init; diff --git a/common/mq/amqp_url.hpp b/common/mq/amqp_url.hpp new file mode 100644 index 0000000..0a3bbba --- /dev/null +++ b/common/mq/amqp_url.hpp @@ -0,0 +1,43 @@ +#pragma once + +#include +#include +#include +#include + +namespace chatnow { + +inline std::string percent_encode_userinfo(std::string_view value) { + static constexpr char kHex[] = "0123456789ABCDEF"; + std::string encoded; + encoded.reserve(value.size()); + for (const unsigned char byte : value) { + const bool unreserved = + (byte >= 'A' && byte <= 'Z') || + (byte >= 'a' && byte <= 'z') || + (byte >= '0' && byte <= '9') || + byte == '-' || byte == '.' || byte == '_' || byte == '~'; + if (unreserved) { + encoded.push_back(static_cast(byte)); + continue; + } + encoded.push_back('%'); + encoded.push_back(kHex[byte >> 4]); + encoded.push_back(kHex[byte & 0x0F]); + } + return encoded; +} + +inline std::string make_amqp_url(const std::string& user, + const std::string& password, + const std::string& host, + uint16_t port = 5672) { + if (host.empty() || host.find_first_of("@:/?#% \t\r\n") != std::string::npos) { + throw std::invalid_argument("RabbitMQ host must be a hostname without a port"); + } + return "amqp://" + percent_encode_userinfo(user) + ":" + + percent_encode_userinfo(password) + "@" + host + ":" + + std::to_string(port) + "/"; +} + +} // namespace chatnow diff --git a/common/mq/business_notify.hpp b/common/mq/business_notify.hpp new file mode 100644 index 0000000..4f4170d --- /dev/null +++ b/common/mq/business_notify.hpp @@ -0,0 +1,43 @@ +#pragma once + +#include +#include + +#include "auth/forward_auth.hpp" +#include "infra/logger.hpp" +#include "mq/channel.hpp" +#include "push/push_service.pb.h" + +namespace chatnow { + +inline constexpr const char* kBusinessPushService = "/service/push_service"; + +// Domain writes are already committed. Online notifications are best effort; +// clients retain the domain APIs as their durable source of truth. +inline void notify_online_users(const ServiceManager::ptr& channels, + brpc::Controller* incoming, + const std::string& request_id, + const std::vector& recipients, + const ::chatnow::push::NotifyMessage& notification) { + if (recipients.empty()) return; + auto channel = channels->choose(kBusinessPushService); + if (!channel) { + LOG_WARN("Business notification unavailable: no Push channel"); + return; + } + ::chatnow::push::PushService_Stub stub(channel.get()); + ::chatnow::push::PushBatchReq request; + ::chatnow::push::PushBatchRsp response; + request.set_request_id(request_id); + for (const auto& recipient : recipients) request.add_user_id_list(recipient); + request.mutable_notify()->CopyFrom(notification); + brpc::Controller outgoing; + outgoing.set_timeout_ms(1000); + ::chatnow::auth::forward_auth_metadata(incoming, &outgoing); + stub.PushBatch(&outgoing, &request, &response, nullptr); + if (outgoing.Failed() || !response.header().success()) { + LOG_WARN("Business notification delivery failed"); + } +} + +} // namespace chatnow diff --git a/common/mq/channel.hpp b/common/mq/channel.hpp index 315ad88..38463b3 100644 --- a/common/mq/channel.hpp +++ b/common/mq/channel.hpp @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -53,7 +54,14 @@ class ServiceChannel options.timeout_ms = kRpcTimeoutMs; options.max_retry = kRpcMaxRetry; options.protocol = "baidu_std"; - if(channel->Init(host.c_str(), &options) != 0) { + // Numeric endpoints keep their direct channel. Hostnames need brpc's + // periodic DNS naming service so an unchanged etcd value can move IPs. + // The http:// prefix selects DNS discovery; the RPC stays baidu_std. + butil::EndPoint endpoint; + const int initialized = butil::str2endpoint(host.c_str(), &endpoint) == 0 + ? channel->Init(endpoint, &options) + : channel->Init(("http://" + host).c_str(), "rr", &options); + if(initialized != 0) { LOG_ERROR("初始化 {}-{} 信道失败", _service_name, host); return; } @@ -123,6 +131,21 @@ class ServiceManager return it->second->choose(); } + /* brief: Return whether a declared service currently has at least one channel. + * Copy the shared owner while holding the manager lock, then inspect the + * channel after releasing it so readiness checks do not extend the nested + * manager -> channel lock scope used by choose(). */ + bool available(const std::string &service_name) const { + ServiceChannel::ptr service; + { + std::unique_lock lock(_mutex); + auto it = _services.find(service_name); + if(it == _services.end()) return false; + service = it->second; + } + return service->size() != 0; + } + /* brief: 声明关注哪些服务（不关注的服务上下线事件会被忽略，节省内存） */ void declared(const std::string &service_name) { std::unique_lock lock(_mutex); @@ -179,7 +202,7 @@ class ServiceManager return service_instance.substr(0, pos); } - std::mutex _mutex; + mutable std::mutex _mutex; std::unordered_set _follow_services; std::unordered_map _services; }; diff --git a/common/mq/rabbitmq.hpp b/common/mq/rabbitmq.hpp index e195410..ea28f83 100644 --- a/common/mq/rabbitmq.hpp +++ b/common/mq/rabbitmq.hpp @@ -29,6 +29,7 @@ #include #include #include "infra/logger.hpp" +#include "mq/amqp_url.hpp" namespace chatnow { @@ -220,15 +221,15 @@ class MQClient ConsumeAction action = callback(message.body(), message.bodySize(), redelivered); switch(action) { case ConsumeAction::Ack: _channel.ack(deliveryTag); break; - case ConsumeAction::NackRequeue: _channel.reject(deliveryTag, true); break; - case ConsumeAction::NackDiscard: _channel.reject(deliveryTag, false); break; + case ConsumeAction::NackRequeue: _channel.reject(deliveryTag, AMQP::requeue); break; + case ConsumeAction::NackDiscard: _channel.reject(deliveryTag, 0); break; } } catch(const std::exception &e) { LOG_ERROR("消费回调异常: {}", e.what()); - _channel.reject(deliveryTag, true); + _channel.reject(deliveryTag, AMQP::requeue); } catch(...) { LOG_ERROR("消费回调发生未知异常"); - _channel.reject(deliveryTag, true); + _channel.reject(deliveryTag, AMQP::requeue); } }) .onError([&promise, queue](const char *message) { @@ -270,15 +271,15 @@ class MQClient redelivered, headers); switch(action) { case ConsumeAction::Ack: _channel.ack(deliveryTag); break; - case ConsumeAction::NackRequeue: _channel.reject(deliveryTag, true); break; - case ConsumeAction::NackDiscard: _channel.reject(deliveryTag, false); break; + case ConsumeAction::NackRequeue: _channel.reject(deliveryTag, AMQP::requeue); break; + case ConsumeAction::NackDiscard: _channel.reject(deliveryTag, 0); break; } } catch(const std::exception &e) { LOG_ERROR("消费回调异常: {}", e.what()); - _channel.reject(deliveryTag, true); + _channel.reject(deliveryTag, AMQP::requeue); } catch(...) { LOG_ERROR("消费回调发生未知异常"); - _channel.reject(deliveryTag, true); + _channel.reject(deliveryTag, AMQP::requeue); } }) .onError([&promise, queue](const char *message) { diff --git a/common/utils/avatar_url.hpp b/common/utils/avatar_url.hpp index 411ea18..82949ef 100644 --- a/common/utils/avatar_url.hpp +++ b/common/utils/avatar_url.hpp @@ -18,7 +18,7 @@ * * 输入参数： * - public_url_prefix：来自 conf/media.json `media.public_url_prefix`， - * 例如 "http://127.0.0.1:9000/chatnow-media-public" 或 "https://cdn.example.com"。 + * 例如 "http://127.0.0.1:19000/chatnow-media-public" 或 "https://cdn.example.com"。 * - file_id：snowflake 16 hex（见 file/source/upload_handler.hpp）。 */ diff --git a/conf/docker/conversation_server.conf b/conf/docker/conversation_server.conf index 6236537..8ea37d4 100644 --- a/conf/docker/conversation_server.conf +++ b/conf/docker/conversation_server.conf @@ -24,4 +24,4 @@ -mysql_cset=utf8mb4 -mysql_port=0 -mysql_pool_count=4 --public_url_prefix=http://gateway_server:9000/chatnow-media-public +-public_url_prefix=http://127.0.0.1:19000/chatnow-media-public diff --git a/conf/docker/media.json b/conf/docker/media.json new file mode 100644 index 0000000..eab194a --- /dev/null +++ b/conf/docker/media.json @@ -0,0 +1,32 @@ +{ + "_comment": "Container Media configuration. Credentials are injected through the runtime secret resolver.", + + "s3": { + "endpoint": "http://minio:9000", + "public_endpoint": "http://127.0.0.1:19000", + "region": "us-east-1" + }, + + "media": { + "public_bucket": "chatnow-media-public", + "private_bucket": "chatnow-media-private", + "public_url_prefix": "http://127.0.0.1:19000/chatnow-media-public", + "presign_seconds": 900, + "asr_endpoint": "", + "mime_whitelist": [ + {"prefix": "image/jpeg", "max_mb": 20}, + {"prefix": "image/png", "max_mb": 20}, + {"prefix": "image/gif", "max_mb": 20}, + {"prefix": "image/webp", "max_mb": 20}, + {"prefix": "video/mp4", "max_mb": 500}, + {"prefix": "video/webm", "max_mb": 500}, + {"prefix": "video/quicktime", "max_mb": 500}, + {"prefix": "audio/aac", "max_mb": 50}, + {"prefix": "audio/mp4", "max_mb": 50}, + {"prefix": "audio/ogg", "max_mb": 50}, + {"prefix": "audio/webm", "max_mb": 50}, + {"prefix": "application/pdf", "max_mb": 100}, + {"prefix": "text/plain", "max_mb": 5} + ] + } +} diff --git a/conf/docker/message_server.conf b/conf/docker/message_server.conf index 6015034..8d15c6b 100644 --- a/conf/docker/message_server.conf +++ b/conf/docker/message_server.conf @@ -17,7 +17,7 @@ -mysql_port=0 -mysql_pool_count=4 -mq_user=root --mq_host=rabbitmq:5672 +-mq_host=rabbitmq -mq_msg_exchange=chat_msg_exchange -mq_msg_queue_db=msg_queue_db -mq_msg_queue_es=msg_queue_es diff --git a/conf/docker/push_server.conf b/conf/docker/push_server.conf index e8605e9..c782dee 100644 --- a/conf/docker/push_server.conf +++ b/conf/docker/push_server.conf @@ -17,7 +17,7 @@ -redis_keep_alive=true -redis_pool_size=16 -mq_user=root --mq_host=rabbitmq:5672 +-mq_host=rabbitmq -mq_push_exchange=chat_push_exchange -mq_push_queue=msg_push_queue -mq_push_binding_key=push diff --git a/conf/docker/transmite_server.conf b/conf/docker/transmite_server.conf index e1d17de..0abba87 100644 --- a/conf/docker/transmite_server.conf +++ b/conf/docker/transmite_server.conf @@ -18,7 +18,7 @@ -redis_keep_alive=true -redis_pool_size=8 -mq_user=root --mq_host=rabbitmq:5672 +-mq_host=rabbitmq -mq_msg_exchange=chat_msg_exchange -mq_msg_queue= -mq_msg_binding_key= diff --git a/conf/local/conversation_server.conf b/conf/local/conversation_server.conf index 51f0b97..62a7ead 100644 --- a/conf/local/conversation_server.conf +++ b/conf/local/conversation_server.conf @@ -24,4 +24,4 @@ -mysql_cset=utf8mb4 -mysql_port=0 -mysql_pool_count=4 --public_url_prefix=http://127.0.0.1:9000/chatnow-media-public +-public_url_prefix=http://127.0.0.1:19000/chatnow-media-public diff --git a/conf/local/message_server.conf b/conf/local/message_server.conf index f680702..2337a4e 100644 --- a/conf/local/message_server.conf +++ b/conf/local/message_server.conf @@ -17,7 +17,7 @@ -mysql_port=0 -mysql_pool_count=4 -mq_user=root --mq_host=127.0.0.1:5672 +-mq_host=127.0.0.1 -mq_msg_exchange=chat_msg_exchange -mq_msg_queue_db=msg_queue_db -mq_msg_queue_es=msg_queue_es diff --git a/conf/local/push_server.conf b/conf/local/push_server.conf index 10bef71..d145a86 100644 --- a/conf/local/push_server.conf +++ b/conf/local/push_server.conf @@ -17,7 +17,7 @@ -redis_keep_alive=true -redis_pool_size=16 -mq_user=root --mq_host=127.0.0.1:5672 +-mq_host=127.0.0.1 -mq_push_exchange=chat_push_exchange -mq_push_queue=msg_push_queue -mq_push_binding_key=push diff --git a/conf/local/transmite_server.conf b/conf/local/transmite_server.conf index 84b865c..85b7785 100644 --- a/conf/local/transmite_server.conf +++ b/conf/local/transmite_server.conf @@ -18,7 +18,7 @@ -redis_keep_alive=true -redis_pool_size=8 -mq_user=root --mq_host=127.0.0.1:5672 +-mq_host=127.0.0.1 -mq_msg_exchange=chat_msg_exchange -mq_msg_queue= -mq_msg_binding_key= diff --git a/conf/media.json b/conf/media.json index 515fa39..a69ad16 100644 --- a/conf/media.json +++ b/conf/media.json @@ -2,14 +2,15 @@ "_comment": "P4 媒体子系统 JSON 配置：仅 s3 + media 段；mysql/redis/etcd/listen_port 等走 gflags", "s3": { - "endpoint": "http://127.0.0.1:9000", - "region": "us-east-1" + "endpoint": "http://127.0.0.1:19000", + "public_endpoint": "http://127.0.0.1:19000", + "region": "us-east-1" }, "media": { "public_bucket": "chatnow-media-public", "private_bucket": "chatnow-media-private", - "public_url_prefix": "http://127.0.0.1:9000/chatnow-media-public", + "public_url_prefix": "http://127.0.0.1:19000/chatnow-media-public", "presign_seconds": 900, "asr_endpoint": "", "mime_whitelist": [ diff --git a/conf/transmite_server.conf.example b/conf/transmite_server.conf.example index 84b865c..85b7785 100644 --- a/conf/transmite_server.conf.example +++ b/conf/transmite_server.conf.example @@ -18,7 +18,7 @@ -redis_keep_alive=true -redis_pool_size=8 -mq_user=root --mq_host=127.0.0.1:5672 +-mq_host=127.0.0.1 -mq_msg_exchange=chat_msg_exchange -mq_msg_queue= -mq_msg_binding_key= diff --git a/conversation/CMakeLists.txt b/conversation/CMakeLists.txt index 9a46d66..5abb9e0 100644 --- a/conversation/CMakeLists.txt +++ b/conversation/CMakeLists.txt @@ -10,7 +10,8 @@ set(proto_files common/types.proto common/error.proto common/envelope.proto message/message_types.proto message/message_service.proto identity/identity_service.proto - conversation/conversation_service.proto) + conversation/conversation_service.proto + push/notify.proto push/push_service.proto) set(proto_srcs "") foreach(proto_file ${proto_files}) string(REPLACE ".proto" ".pb.cc" proto_cc ${proto_file}) @@ -54,11 +55,12 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lodb-mysql -lodb -lodb-boost - -lcpr -lelasticlient -ljsoncpp + -lcpr -lelasticlient -lhiredis -lredis++) include_directories(${CMAKE_CURRENT_BINARY_DIR}) diff --git a/conversation/Dockerfile b/conversation/Dockerfile index fe7b7ec..cc16ebf 100644 --- a/conversation/Dockerfile +++ b/conversation/Dockerfile @@ -1,9 +1,9 @@ ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends WORKDIR /im RUN mkdir -p /im/logs && mkdir -p /im/data && mkdir -p /im/conf && mkdir -p /im/bin COPY ./build/conversation_server /im/bin COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends CMD /im/bin/conversation_server -flagfile=/im/conf/conversation_server.conf diff --git a/conversation/source/conversation_server.cc b/conversation/source/conversation_server.cc index 7bc4150..d1aa14f 100644 --- a/conversation/source/conversation_server.cc +++ b/conversation/source/conversation_server.cc @@ -34,7 +34,7 @@ DEFINE_string(mysql_cset, "utf8mb4", "MySQL客户端字符集"); DEFINE_int32(mysql_port, 0, "MySQL服务器访问端口"); DEFINE_int32(mysql_pool_count, 4, "MySQL连接池最大连接数量"); -DEFINE_string(public_url_prefix, "http://127.0.0.1:9000/chatnow-media-public", +DEFINE_string(public_url_prefix, "http://127.0.0.1:19000/chatnow-media-public", "Media 公开 bucket URL 前缀（avatar_file_id → URL 转换用）"); int main(int argc, char *argv[]) diff --git a/conversation/source/conversation_server.h b/conversation/source/conversation_server.h index 2c4150b..71f106e 100644 --- a/conversation/source/conversation_server.h +++ b/conversation/source/conversation_server.h @@ -11,6 +11,7 @@ #include "infra/etcd.hpp" #include "mq/channel.hpp" +#include "mq/business_notify.hpp" #include "infra/logger.hpp" #include "infra/metrics.hpp" #include @@ -229,6 +230,15 @@ class ConversationServiceImpl : public ::chatnow::conversation::ConversationServ auto* self = out->mutable_self(); self->set_role(static_cast<::chatnow::conversation::MemberRole>(owner_role)); self->set_joined_at_ms(_to_ms(now)); + ::chatnow::push::NotifyMessage notification; + notification.set_notify_event_id(req->request_id()); + notification.set_notify_type(::chatnow::push::CONVERSATION_CREATE_NOTIFY); + notification.set_trace_id(auth.trace_id); + auto public_conversation = *out; + public_conversation.clear_self(); + notification.mutable_new_conversation_info()->set_conversation_payload( + public_conversation.SerializeAsString()); + notify_online_users(_mm_channels, cntl, req->request_id(), all_member_ids, notification); }); } @@ -666,6 +676,11 @@ class ConversationServiceImpl : public ::chatnow::conversation::ConversationServ brpc::ClosureGuard done_guard(done); auto* cntl = static_cast(base_cntl); HANDLE_RPC(cntl, req, rsp, { + const auto conversation = _mysql_conv->select(req->conversation_id()); + if (!conversation || conversation->status() == ConversationStatus::DISMISSED) { + throw ServiceError(::chatnow::error::kConversationNotMember, + "conversation unavailable"); + } // 1. 优先走 Redis 缓存；snapshot 前后版本一致才可信 auto snap = _members_cache->list_snapshot(req->conversation_id()); if (!snap.stable) metrics::g_members_cache_snapshot_race_total << 1; @@ -1156,6 +1171,7 @@ class ConversationServerBuilder _mm_channels->declared(_identity_service_name); _mm_channels->declared(_media_service_name); _mm_channels->declared(_message_service_name); + _mm_channels->declared(kBusinessPushService); auto put_cb = std::bind(&ServiceManager::onServiceOnline, _mm_channels.get(), std::placeholders::_1, std::placeholders::_2); diff --git a/docker-compose.yml b/docker-compose.yml index a0d2293..e26ed78 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,5 +1,3 @@ -version: "3.8" - services: etcd: image: quay.io/coreos/etcd:v3.4.30 @@ -12,103 +10,158 @@ services: volumes: - ./middle/data/etcd:/var/lib/etcd:rw ports: - - 2379:2379 + - "127.0.0.1:2379:2379" + healthcheck: + test: ["CMD", "etcdctl", "--endpoints=http://127.0.0.1:2379", "endpoint", "health"] + interval: 2s + timeout: 2s + retries: 30 restart: always mysql: image: mysql:8.0.44 container_name: mysql-service environment: MYSQL_ROOT_PASSWORD: ${CHATNOW_MYSQL_ROOT_PASSWORD:?CHATNOW_MYSQL_ROOT_PASSWORD is required} + MYSQL_DATABASE: chatnow volumes: - - ./sql:/docker-entrypoint-initdb.d/:rw - ./middle/data/mysql:/var/lib/mysql:rw ports: - - 3306:3306 + - "127.0.0.1:3306:3306" + healthcheck: + test: ["CMD-SHELL", "mysqladmin ping --host=127.0.0.1 --silent"] + interval: 2s + timeout: 2s + retries: 60 + start_period: 10s restart: always + mysql-init: + image: mysql:8.0.44 + container_name: chatnow-mysql-init + depends_on: + mysql: + condition: service_healthy + environment: + MYSQL_ROOT_PASSWORD: ${CHATNOW_MYSQL_ROOT_PASSWORD:?CHATNOW_MYSQL_ROOT_PASSWORD is required} + CHATNOW_IDENTITY_MYSQL_PASSWORD: ${CHATNOW_IDENTITY_MYSQL_PASSWORD:?CHATNOW_IDENTITY_MYSQL_PASSWORD is required} + CHATNOW_CONVERSATION_MYSQL_PASSWORD: ${CHATNOW_CONVERSATION_MYSQL_PASSWORD:?CHATNOW_CONVERSATION_MYSQL_PASSWORD is required} + CHATNOW_RELATIONSHIP_MYSQL_PASSWORD: ${CHATNOW_RELATIONSHIP_MYSQL_PASSWORD:?CHATNOW_RELATIONSHIP_MYSQL_PASSWORD is required} + CHATNOW_MESSAGE_MYSQL_PASSWORD: ${CHATNOW_MESSAGE_MYSQL_PASSWORD:?CHATNOW_MESSAGE_MYSQL_PASSWORD is required} + CHATNOW_MEDIA_MYSQL_PASSWORD: ${CHATNOW_MEDIA_MYSQL_PASSWORD:?CHATNOW_MEDIA_MYSQL_PASSWORD is required} + volumes: + - ./sql:/migrations:ro + - ./scripts/init_mysql.sh:/init/init_mysql.sh:ro + - ./scripts/converge_mysql_users.sh:/init/converge_mysql_users.sh:ro + entrypoint: /bin/bash -c "exec /init/init_mysql.sh" + restart: "no" redis-node1: image: redis:7.2.5 container_name: redis-node1 - command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --appendonly yes --appendfilename appendonly-1.aof + command: redis-server --port 6379 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --cluster-announce-hostname redis-node1 --cluster-preferred-endpoint-type hostname --appendonly yes --appendfilename appendonly-1.aof volumes: - ./middle/data/redis/node1:/data:rw ports: - - "6379:6379" + - "127.0.0.1:6379:6379" + healthcheck: + test: ["CMD", "redis-cli", "-p", "6379", "ping"] + interval: 2s + timeout: 2s + retries: 30 restart: always redis-node2: image: redis:7.2.5 container_name: redis-node2 - command: redis-server --port 6380 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --appendonly yes --appendfilename appendonly-2.aof + command: redis-server --port 6380 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --cluster-announce-hostname redis-node2 --cluster-preferred-endpoint-type hostname --appendonly yes --appendfilename appendonly-2.aof volumes: - ./middle/data/redis/node2:/data:rw ports: - - "6380:6380" + - "127.0.0.1:6380:6380" + healthcheck: + test: ["CMD", "redis-cli", "-p", "6380", "ping"] + interval: 2s + timeout: 2s + retries: 30 restart: always redis-node3: image: redis:7.2.5 container_name: redis-node3 - command: redis-server --port 6381 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --appendonly yes --appendfilename appendonly-3.aof + command: redis-server --port 6381 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --cluster-announce-hostname redis-node3 --cluster-preferred-endpoint-type hostname --appendonly yes --appendfilename appendonly-3.aof volumes: - ./middle/data/redis/node3:/data:rw ports: - - "6381:6381" + - "127.0.0.1:6381:6381" + healthcheck: + test: ["CMD", "redis-cli", "-p", "6381", "ping"] + interval: 2s + timeout: 2s + retries: 30 restart: always redis-node4: image: redis:7.2.5 container_name: redis-node4 - command: redis-server --port 6382 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --appendonly yes --appendfilename appendonly-4.aof + command: redis-server --port 6382 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --cluster-announce-hostname redis-node4 --cluster-preferred-endpoint-type hostname --appendonly yes --appendfilename appendonly-4.aof volumes: - ./middle/data/redis/node4:/data:rw ports: - - "6382:6382" + - "127.0.0.1:6382:6382" + healthcheck: + test: ["CMD", "redis-cli", "-p", "6382", "ping"] + interval: 2s + timeout: 2s + retries: 30 restart: always redis-node5: image: redis:7.2.5 container_name: redis-node5 - command: redis-server --port 6383 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --appendonly yes --appendfilename appendonly-5.aof + command: redis-server --port 6383 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --cluster-announce-hostname redis-node5 --cluster-preferred-endpoint-type hostname --appendonly yes --appendfilename appendonly-5.aof volumes: - ./middle/data/redis/node5:/data:rw ports: - - "6383:6383" + - "127.0.0.1:6383:6383" + healthcheck: + test: ["CMD", "redis-cli", "-p", "6383", "ping"] + interval: 2s + timeout: 2s + retries: 30 restart: always redis-node6: image: redis:7.2.5 container_name: redis-node6 - command: redis-server --port 6384 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --appendonly yes --appendfilename appendonly-6.aof + command: redis-server --port 6384 --cluster-enabled yes --cluster-config-file /data/nodes.conf --cluster-node-timeout 5000 --cluster-announce-hostname redis-node6 --cluster-preferred-endpoint-type hostname --appendonly yes --appendfilename appendonly-6.aof volumes: - ./middle/data/redis/node6:/data:rw ports: - - "6384:6384" + - "127.0.0.1:6384:6384" + healthcheck: + test: ["CMD", "redis-cli", "-p", "6384", "ping"] + interval: 2s + timeout: 2s + retries: 30 restart: always redis-cluster-init: image: redis:7.2.5 container_name: redis-cluster-init depends_on: - - redis-node1 - - redis-node2 - - redis-node3 - - redis-node4 - - redis-node5 - - redis-node6 - entrypoint: | - /bin/sh -c " - echo 'Waiting for all Redis nodes...' && - sleep 10 && - echo 'Creating 3-master 3-slave cluster...' && - echo yes | redis-cli --cluster create \ - redis-node1:6379 redis-node2:6380 redis-node3:6381 \ - redis-node4:6382 redis-node5:6383 redis-node6:6384 \ - --cluster-replicas 1 && - echo 'Verifying cluster...' && - redis-cli --cluster check redis-node1:6379 && - echo 'Cluster ready.' && - tail -f /dev/null - " + redis-node1: + condition: service_healthy + redis-node2: + condition: service_healthy + redis-node3: + condition: service_healthy + redis-node4: + condition: service_healthy + redis-node5: + condition: service_healthy + redis-node6: + condition: service_healthy + volumes: + - ./scripts/init_redis_cluster.sh:/init/init_redis_cluster.sh:ro + entrypoint: /bin/sh -c "REDIS_INIT_MAX_ATTEMPTS=60 exec /init/init_redis_cluster.sh" restart: "no" elasticsearch: image: elasticsearch:7.17.21 @@ -118,20 +171,84 @@ services: volumes: - ./middle/data/elasticsearch:/var/lib/elasticsearch:rw ports: - - 9200:9200 - - 9300:9300 + - "127.0.0.1:9200:9200" + - "127.0.0.1:9300:9300" + healthcheck: + test: ["CMD-SHELL", "curl --fail --silent 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=1s' >/dev/null"] + interval: 3s + timeout: 2s + retries: 40 + start_period: 10s restart: always rabbitmq: - image: rabbitmq:3.12.1 + image: rabbitmq:3.12.1-management container_name: rabbitmq-service + hostname: rabbitmq environment: - RABBITMQ_DEFAULT_USER: root + RABBITMQ_NODENAME: rabbit@rabbitmq + RABBITMQ_DEFAULT_USER: ${CHATNOW_RABBITMQ_BOOTSTRAP_USER:-chatnow_bootstrap} RABBITMQ_DEFAULT_PASS: ${CHATNOW_RABBITMQ_BOOTSTRAP_PASSWORD:?CHATNOW_RABBITMQ_BOOTSTRAP_PASSWORD is required} volumes: - ./middle/data/rabbitmq:/var/lib/rabbitmq:rw ports: - - 5672:5672 + - "127.0.0.1:5672:5672" + healthcheck: + test: ["CMD", "rabbitmq-diagnostics", "-q", "check_running"] + interval: 3s + timeout: 3s + retries: 40 + start_period: 10s restart: always + rabbitmq-init: + image: rabbitmq:3.12.1-management + container_name: rabbitmq-init + depends_on: + rabbitmq: + condition: service_healthy + environment: + RABBITMQ_INIT_MAX_ATTEMPTS: 60 + RABBITMQ_BOOTSTRAP_USER: ${CHATNOW_RABBITMQ_BOOTSTRAP_USER:-chatnow_bootstrap} + RABBITMQ_BOOTSTRAP_PASSWORD: ${CHATNOW_RABBITMQ_BOOTSTRAP_PASSWORD:?CHATNOW_RABBITMQ_BOOTSTRAP_PASSWORD is required} + CHATNOW_TRANSMITE_MQ_PASSWORD: ${CHATNOW_TRANSMITE_MQ_PASSWORD:?CHATNOW_TRANSMITE_MQ_PASSWORD is required} + CHATNOW_MESSAGE_MQ_PASSWORD: ${CHATNOW_MESSAGE_MQ_PASSWORD:?CHATNOW_MESSAGE_MQ_PASSWORD is required} + CHATNOW_PUSH_MQ_PASSWORD: ${CHATNOW_PUSH_MQ_PASSWORD:?CHATNOW_PUSH_MQ_PASSWORD is required} + volumes: + - ./scripts/init_rabbitmq.py:/init/init_rabbitmq.py:ro + entrypoint: python3 /init/init_rabbitmq.py + restart: "no" + minio: + image: quay.io/minio/minio:RELEASE.2024-10-13T13-34-11Z@sha256:9535594ad4122b7a78c6632788a989b96d9199b483d3bd71a5ceae73a922cdfa + container_name: chatnow-minio + command: server /data --console-address ":9001" + environment: + MINIO_ROOT_USER: ${CHATNOW_MINIO_ROOT_USER:?CHATNOW_MINIO_ROOT_USER is required} + MINIO_ROOT_PASSWORD: ${CHATNOW_MINIO_ROOT_PASSWORD:?CHATNOW_MINIO_ROOT_PASSWORD is required} + volumes: + - ./middle/data/minio:/data:rw + ports: + - "127.0.0.1:19000:9000" + - "127.0.0.1:19001:9001" + healthcheck: + test: ["CMD", "curl", "--fail", "--silent", "http://127.0.0.1:9000/minio/health/live"] + interval: 3s + timeout: 2s + retries: 40 + start_period: 5s + restart: always + minio-init: + build: docker/minio-init + image: chatnow-minio-init:local + container_name: chatnow-minio-init + depends_on: + minio: + condition: service_healthy + environment: + MINIO_ROOT_USER: ${CHATNOW_MINIO_ROOT_USER:?CHATNOW_MINIO_ROOT_USER is required} + MINIO_ROOT_PASSWORD: ${CHATNOW_MINIO_ROOT_PASSWORD:?CHATNOW_MINIO_ROOT_PASSWORD is required} + MINIO_APP_ACCESS_KEY: ${CHATNOW_MEDIA_S3_ACCESS_KEY:?CHATNOW_MEDIA_S3_ACCESS_KEY is required} + MINIO_APP_SECRET_KEY: ${CHATNOW_MEDIA_S3_SECRET_KEY:?CHATNOW_MEDIA_S3_SECRET_KEY is required} + MINIO_INIT_MAX_ATTEMPTS: 60 + restart: "no" media_server: build: ./media container_name: media_server-service @@ -140,8 +257,8 @@ services: CHATNOW_MEDIA_S3_ACCESS_KEY: ${CHATNOW_MEDIA_S3_ACCESS_KEY:?CHATNOW_MEDIA_S3_ACCESS_KEY is required} CHATNOW_MEDIA_S3_SECRET_KEY: ${CHATNOW_MEDIA_S3_SECRET_KEY:?CHATNOW_MEDIA_S3_SECRET_KEY is required} volumes: - - ./conf/docker/media_server.conf:/im/conf/media_server.conf - - ./conf/media.json:/im/conf/media.json + - ./conf/docker/media_server.conf:/im/conf/media_server.conf:ro + - ./conf/docker/media.json:/im/conf/media.json:ro - ./middle/data/logs:/var/lib/logs:rw - ./middle/data/data:/var/lib/data:rw - ./entrypoint.sh:/im/bin/entrypoint.sh @@ -149,11 +266,16 @@ services: - 10002:10002 restart: always depends_on: - - etcd - - mysql - - redis-cluster-init + etcd: + condition: service_healthy + mysql-init: + condition: service_completed_successfully + redis-cluster-init: + condition: service_completed_successfully + minio-init: + condition: service_completed_successfully entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384 -c "/im/bin/media_server -flagfile=/im/conf/media_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" + /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,minio:9000 -c "/im/bin/media_server -flagfile=/im/conf/media_server.conf -mysql_user=chatnow_media -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" relationship_server: build: ./relationship container_name: relationship_server-service @@ -168,11 +290,14 @@ services: - 10006:10006 restart: always depends_on: - - etcd - - mysql - - elasticsearch + etcd: + condition: service_healthy + mysql-init: + condition: service_completed_successfully + elasticsearch: + condition: service_healthy entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,elasticsearch:9200 -c "/im/bin/relationship_server -flagfile=/im/conf/relationship_server.conf" + /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,elasticsearch:9200 -c "/im/bin/relationship_server -flagfile=/im/conf/relationship_server.conf -mysql_user=chatnow_relationship" conversation_server: build: ./conversation container_name: conversation_server-service @@ -187,12 +312,16 @@ services: - 10007:10007 restart: always depends_on: - - etcd - - mysql - - redis-cluster-init - - elasticsearch + etcd: + condition: service_healthy + mysql-init: + condition: service_completed_successfully + redis-cluster-init: + condition: service_completed_successfully + elasticsearch: + condition: service_healthy entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,elasticsearch:9200 -c "/im/bin/conversation_server -flagfile=/im/conf/conversation_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" + /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,elasticsearch:9200 -c "/im/bin/conversation_server -flagfile=/im/conf/conversation_server.conf -mysql_user=chatnow_conversation -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" gateway_server: build: ./gateway container_name: gateway_server-service @@ -207,8 +336,10 @@ services: - 9000:9000 restart: always depends_on: - - etcd - - redis-cluster-init + etcd: + condition: service_healthy + redis-cluster-init: + condition: service_completed_successfully entrypoint: /im/bin/entrypoint.sh -d etcd:2379,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384 -c "/im/bin/gateway_server -flagfile=/im/conf/gateway_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" push_server: @@ -227,11 +358,14 @@ services: - 9001:9001 restart: always depends_on: - - etcd - - redis-cluster-init - - rabbitmq + etcd: + condition: service_healthy + redis-cluster-init: + condition: service_completed_successfully + rabbitmq-init: + condition: service_completed_successfully entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,rabbitmq:5672 -c "/im/bin/push_server -flagfile=/im/conf/push_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" + /im/bin/entrypoint.sh -d etcd:2379,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,rabbitmq:5672 -c "/im/bin/push_server -flagfile=/im/conf/push_server.conf -mq_user=chatnow_push -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" message_server: build: ./message container_name: message_server-service @@ -247,13 +381,18 @@ services: - 10005:10005 restart: always depends_on: - - etcd - - mysql - - elasticsearch - - rabbitmq - - redis-cluster-init + etcd: + condition: service_healthy + mysql-init: + condition: service_completed_successfully + elasticsearch: + condition: service_healthy + rabbitmq-init: + condition: service_completed_successfully + redis-cluster-init: + condition: service_completed_successfully entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,elasticsearch:9200,rabbitmq:5672 -c "/im/bin/message_server -flagfile=/im/conf/message_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" + /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,elasticsearch:9200,rabbitmq:5672 -c "/im/bin/message_server -flagfile=/im/conf/message_server.conf -mysql_user=chatnow_message -mq_user=chatnow_message -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" transmite_server: build: ./transmite container_name: transmite_server-service @@ -268,12 +407,14 @@ services: - 10004:10004 restart: always depends_on: - - etcd - - mysql - - rabbitmq - - redis-cluster-init + etcd: + condition: service_healthy + rabbitmq-init: + condition: service_completed_successfully + redis-cluster-init: + condition: service_completed_successfully entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,rabbitmq:5672 -c "/im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384 -rate_limit_user_max=${TRANSMITE_RATE_LIMIT_USER_MAX:-600} -rate_limit_session_max=${TRANSMITE_RATE_LIMIT_SESSION_MAX:-3000}" + /im/bin/entrypoint.sh -d etcd:2379,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,rabbitmq:5672 -c "/im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf -mq_user=chatnow_transmite -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384 -rate_limit_user_max=${TRANSMITE_RATE_LIMIT_USER_MAX:-600} -rate_limit_session_max=${TRANSMITE_RATE_LIMIT_SESSION_MAX:-3000}" identity_server: build: ./identity container_name: identity_server-service @@ -290,12 +431,16 @@ services: - 10003:10003 restart: always depends_on: - - etcd - - mysql - - redis-cluster-init - - elasticsearch + etcd: + condition: service_healthy + mysql-init: + condition: service_completed_successfully + redis-cluster-init: + condition: service_completed_successfully + elasticsearch: + condition: service_healthy entrypoint: - /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,elasticsearch:9200 -c "/im/bin/identity_server -flagfile=/im/conf/identity_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" + /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,elasticsearch:9200 -c "/im/bin/identity_server -flagfile=/im/conf/identity_server.conf -mysql_user=chatnow_identity -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" presence_server: build: ./presence container_name: presence_server-service @@ -308,7 +453,9 @@ services: - 9050:9050 restart: always depends_on: - - etcd - - redis-cluster-init + etcd: + condition: service_healthy + redis-cluster-init: + condition: service_completed_successfully entrypoint: /im/bin/entrypoint.sh -d etcd:2379,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384 -c "/im/bin/presence_server -flagfile=/im/conf/presence_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384" diff --git a/docker/ci/Dockerfile b/docker/ci/Dockerfile index a4c9b06..bdd6d34 100644 --- a/docker/ci/Dockerfile +++ b/docker/ci/Dockerfile @@ -30,6 +30,7 @@ RUN apt-get -o Acquire::Retries=5 update \ libodb-dev \ libodb-mysql-dev \ libprotobuf-dev \ + libprotoc-dev \ libsnappy-dev \ libspdlog-dev \ libssl-dev \ @@ -115,6 +116,8 @@ RUN set -euo pipefail; \ cmake --install /tmp/cpr-build; \ rm -rf /tmp/cpr /tmp/cpr-build +COPY docker/ci/elasticlient-jsoncpp.cmake /tmp/elasticlient-jsoncpp.cmake + RUN set -euo pipefail; \ . /tmp/dependencies.lock; \ git init /tmp/elasticlient; \ @@ -123,6 +126,7 @@ RUN set -euo pipefail; \ git -C /tmp/elasticlient checkout --detach FETCH_HEAD; \ test "$(git -C /tmp/elasticlient rev-parse HEAD)" = "$ELASTICLIENT_REV"; \ cmake -S /tmp/elasticlient -B /tmp/elasticlient-build -G Ninja \ + -DCMAKE_PROJECT_Elasticlient_INCLUDE=/tmp/elasticlient-jsoncpp.cmake \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/usr/local \ -DBUILD_ELASTICLIENT_EXAMPLE=OFF \ @@ -145,6 +149,7 @@ RUN set -euo pipefail; \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/usr/local \ -DAMQP-CPP_BUILD_EXAMPLES=OFF \ + -DAMQP-CPP_LINUX_TCP=ON \ -DAMQP-CPP_BUILD_SHARED=ON; \ cmake --build /tmp/amqp-cpp-build --parallel "$(nproc)"; \ cmake --install /tmp/amqp-cpp-build; \ @@ -174,3 +179,8 @@ RUN printf '%s\n' '/usr/local/lib' '/usr/local/lib64' > /etc/ld.so.conf.d/chatno && ldconfig WORKDIR /workspace + +# Ubuntu's ODB compiler invokes the GCC version it was packaged with. +RUN apt-get -o Acquire::Retries=5 update \ + && apt-get -o Acquire::Retries=5 install -y --no-install-recommends g++-12 \ + && rm -rf /var/lib/apt/lists/* diff --git a/docker/ci/elasticlient-jsoncpp.cmake b/docker/ci/elasticlient-jsoncpp.cmake new file mode 100644 index 0000000..c44c13f --- /dev/null +++ b/docker/ci/elasticlient-jsoncpp.cmake @@ -0,0 +1,3 @@ +# Elasticlient discovers JsonCpp in external/, then links its imported target +# from the sibling src/ directory. Import it at project scope for both children. +find_package(jsoncpp CONFIG REQUIRED) diff --git a/docker/minio-init/Dockerfile b/docker/minio-init/Dockerfile new file mode 100644 index 0000000..99221b6 --- /dev/null +++ b/docker/minio-init/Dockerfile @@ -0,0 +1,8 @@ +FROM quay.io/minio/mc:RELEASE.2024-10-08T09-37-26Z AS mc + +FROM busybox:1.36.1-musl + +COPY --from=mc /usr/bin/mc /usr/bin/mc +COPY entrypoint.sh /usr/local/bin/entrypoint.sh + +ENTRYPOINT ["/bin/sh", "/usr/local/bin/entrypoint.sh"] diff --git a/docker/minio-init/entrypoint.sh b/docker/minio-init/entrypoint.sh index b408a74..a1ce26b 100755 --- a/docker/minio-init/entrypoint.sh +++ b/docker/minio-init/entrypoint.sh @@ -5,20 +5,79 @@ # chatnow-media-public ：公共可读（avatar/sticker），anonymous download # chatnow-media-private ：会话媒体，关闭匿名（默认） # --------------------------------------------------------------------------- -# alias "local" 已通过 MC_HOST_local 环境变量预置（见 docker-compose.yml）。 # --------------------------------------------------------------------------- -set -e +set -eu -# 1) 创建 bucket（幂等：-p 让父级路径存在；存在则不报错） -mc mb -p local/chatnow-media-public || true -mc mb -p local/chatnow-media-private || true +MAX_ATTEMPTS=${MINIO_INIT_MAX_ATTEMPTS:-60} +required="MINIO_ROOT_USER MINIO_ROOT_PASSWORD MINIO_APP_ACCESS_KEY MINIO_APP_SECRET_KEY" +for name in $required; do + value=$(printenv "$name" 2>/dev/null || true) + if [ -z "$value" ]; then + echo "minio-init: required credential is missing: ${name}" >&2 + exit 1 + fi +done -# 2) 公共 bucket：任意 GET 都允许（download policy） -mc anonymous set download local/chatnow-media-public +MC_CONFIG_DIR=$(mktemp -d) +export MC_CONFIG_DIR +policy_file="" +cleanup() { + if [ -n "$policy_file" ]; then + rm -f "$policy_file" + fi + rm -rf "$MC_CONFIG_DIR" +} +trap cleanup EXIT + +printf '%s\n%s\n' "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" | + mc alias set local http://minio:9000 --api S3v4 --path on >/dev/null + +attempt=1 +while [ "$attempt" -le "$MAX_ATTEMPTS" ]; do + if mc admin info local >/dev/null 2>&1; then + break + fi + attempt=$((attempt + 1)) + sleep 1 +done + +if [ "$attempt" -gt "$MAX_ATTEMPTS" ]; then + echo "minio-init: server did not become ready" >&2 + exit 1 +fi -# 3) 私密 bucket：显式声明无匿名（默认即如此，但显式声明便于自检 / 防误改） +mc mb --ignore-existing local/chatnow-media-public +mc mb --ignore-existing local/chatnow-media-private +mc anonymous set download local/chatnow-media-public mc anonymous set none local/chatnow-media-private -echo "[minio-init] buckets ready:" -mc ls local +policy_file=$(mktemp) +cat >"$policy_file" <<'JSON' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": ["s3:GetBucketLocation", "s3:ListBucketMultipartUploads"], + "Resource": ["arn:aws:s3:::chatnow-media-public", "arn:aws:s3:::chatnow-media-private"] + }, + { + "Effect": "Allow", + "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload"], + "Resource": ["arn:aws:s3:::chatnow-media-public/*", "arn:aws:s3:::chatnow-media-private/*"] + } + ] +} +JSON + +# `user add` and `policy create` converge existing entries to the supplied state. +printf '%s\n%s\n' "$MINIO_APP_ACCESS_KEY" "$MINIO_APP_SECRET_KEY" | + mc admin user add local +mc admin policy create local chatnow-media-app "$policy_file" +mc admin policy attach local chatnow-media-app --user "$MINIO_APP_ACCESS_KEY" + +mc stat local/chatnow-media-public >/dev/null +mc stat local/chatnow-media-private >/dev/null +mc admin user info local "$MINIO_APP_ACCESS_KEY" >/dev/null +echo "minio-init: buckets and application identity are ready" diff --git a/docs/api/openapi-media.yaml b/docs/api/openapi-media.yaml index 73af9f6..2a3808d 100644 --- a/docs/api/openapi-media.yaml +++ b/docs/api/openapi-media.yaml @@ -5,7 +5,9 @@ info: description: | 媒体上传/下载域。Proto: proto/media/media_service.proto Service: chatnow.media.MediaService - 注意: 分片上传接口（InitMultipartUpload/ApplyPartUpload/CompleteMultipartUpload/AbortMultipartUpload）尚未暴露为 HTTP 路由。 + Multipart operations below are exposed through JWT-authenticated Gateway routes. + Upload each part using its complete signed URL; do not alter query parameters. + Every non-final S3 multipart part must contain at least 5 MiB. servers: - url: http://localhost:9000 @@ -35,7 +37,7 @@ paths: application/x-protobuf: schema: allOf: - - $ref: '../openapi-common.yaml#/components/schemas/ResponseHeader' + - $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' - $ref: '#/components/schemas/ApplyUploadRsp' '4xx': { $ref: '#/components/responses/BusinessError' } @@ -57,10 +59,86 @@ paths: application/x-protobuf: schema: allOf: - - $ref: '../openapi-common.yaml#/components/schemas/ResponseHeader' + - $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' - $ref: '#/components/schemas/CompleteUploadRsp' '4xx': { $ref: '#/components/responses/BusinessError' } + /service/media/init_multipart: + post: + summary: Initialize a multipart upload + tags: [Media] + x-protobuf: { service: chatnow.media.MediaService, rpc: InitMultipartUpload, request: InitMultipartReq, response: InitMultipartRsp, file: proto/media/media_service.proto } + requestBody: + required: true + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/InitMultipartReq' } + responses: + '200': + description: Protobuf response; inspect header.success and header.error_code. + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/InitMultipartRsp' } + '401': + description: Missing, invalid or revoked JWT; request is rejected before RPC dispatch. + + /service/media/apply_part_upload: + post: + summary: Presign one multipart part + tags: [Media] + x-protobuf: { service: chatnow.media.MediaService, rpc: ApplyPartUpload, request: ApplyPartReq, response: ApplyPartRsp, file: proto/media/media_service.proto } + requestBody: + required: true + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/ApplyPartReq' } + responses: + '200': + description: Protobuf response; inspect header.success and header.error_code. + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/ApplyPartRsp' } + '401': + description: Missing, invalid or revoked JWT; request is rejected before RPC dispatch. + + /service/media/complete_multipart: + post: + summary: Complete parts in ascending part-number order + tags: [Media] + x-protobuf: { service: chatnow.media.MediaService, rpc: CompleteMultipartUpload, request: CompleteMultipartReq, response: CompleteMultipartRsp, file: proto/media/media_service.proto } + requestBody: + required: true + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/CompleteMultipartReq' } + responses: + '200': + description: Protobuf response; inspect header.success and header.error_code. + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/CompleteMultipartRsp' } + '401': + description: Missing, invalid or revoked JWT; request is rejected before RPC dispatch. + + /service/media/abort_multipart: + post: + summary: Abort a pending multipart upload + tags: [Media] + x-protobuf: { service: chatnow.media.MediaService, rpc: AbortMultipartUpload, request: AbortMultipartReq, response: AbortMultipartRsp, file: proto/media/media_service.proto } + requestBody: + required: true + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/AbortMultipartReq' } + responses: + '200': + description: Protobuf response; inspect header.success and header.error_code. + content: + application/x-protobuf: + schema: { $ref: '#/components/schemas/AbortMultipartRsp' } + '401': + description: Missing, invalid or revoked JWT; request is rejected before RPC dispatch. + /service/media/apply_download: post: summary: 申请下载 @@ -79,7 +157,7 @@ paths: application/x-protobuf: schema: allOf: - - $ref: '../openapi-common.yaml#/components/schemas/ResponseHeader' + - $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' - $ref: '#/components/schemas/ApplyDownloadRsp' '4xx': { $ref: '#/components/responses/BusinessError' } @@ -100,14 +178,18 @@ paths: application/x-protobuf: schema: allOf: - - $ref: '../openapi-common.yaml#/components/schemas/ResponseHeader' + - $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' - $ref: '#/components/schemas/GetFileInfoRsp' '4xx': { $ref: '#/components/responses/BusinessError' } /service/media/speech_recognition: post: summary: 语音识别 - description: 短音频 ASR，直接传 bytes，不经过 S3。 + description: | + Accepts inline PCM16 bytes up to 2 MiB, without S3. Empty input or an odd + byte count is invalid. A configured ASR backend is not yet available, so + valid input returns a business failure with speech recognition unavailable; + it must not be treated as a successful transcription. tags: [Media] x-protobuf: { service: chatnow.media.MediaService, rpc: SpeechRecognition, request: SpeechRecognitionReq, response: SpeechRecognitionRsp, file: proto/media/media_service.proto } requestBody: @@ -122,7 +204,7 @@ paths: application/x-protobuf: schema: allOf: - - $ref: '../openapi-common.yaml#/components/schemas/ResponseHeader' + - $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' - $ref: '#/components/schemas/SpeechRecognitionRsp' '4xx': { $ref: '#/components/responses/BusinessError' } @@ -138,7 +220,7 @@ components: description: 业务错误（error_code != 0） content: application/x-protobuf: - schema: { $ref: '../openapi-common.yaml#/components/schemas/ResponseHeader' } + schema: { $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' } schemas: MediaPurpose: @@ -155,6 +237,9 @@ components: file_size: { type: integer, format: int64 } mime_type: { type: string } uploaded_at_ms: { type: integer, format: int64 } + public_url: + type: string + description: Canonical unsigned URL for committed public objects returned by ApplyDownload or GetFileInfo; empty for private files. Protobuf field 6. ApplyUploadReq: x-protobuf: { message: ApplyUploadReq, file: proto/media/media_service.proto } @@ -246,3 +331,67 @@ components: recognition_result: type: string description: 识别出的文字 + + InitMultipartReq: + allOf: + - $ref: '#/components/schemas/ApplyUploadReq' + x-protobuf: { message: InitMultipartReq, file: proto/media/media_service.proto } + + InitMultipartRsp: + type: object + properties: + header: { $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' } + file_id: { type: string } + upload_id: { type: string } + recommended_part_size_bytes: { type: integer, format: int32 } + + ApplyPartReq: + type: object + required: [request_id, upload_id, part_number] + properties: + request_id: { type: string } + upload_id: { type: string } + part_number: { type: integer, minimum: 1, maximum: 10000 } + + ApplyPartRsp: + type: object + properties: + header: { $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' } + upload_url: { type: string, description: Signed PUT URL binding uploadId and partNumber. } + expires_in_sec: { type: integer, format: int32 } + + PartETag: + type: object + required: [part_number, etag] + properties: + part_number: { type: integer, minimum: 1, maximum: 10000 } + etag: { type: string, description: ETag returned by the successful part PUT. } + + CompleteMultipartReq: + type: object + required: [request_id, upload_id, parts] + properties: + request_id: { type: string } + upload_id: { type: string } + parts: + type: array + description: Parts in ascending part-number order. + items: { $ref: '#/components/schemas/PartETag' } + + CompleteMultipartRsp: + type: object + properties: + header: { $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' } + file_info: { $ref: '#/components/schemas/FileInfo' } + + AbortMultipartReq: + type: object + required: [request_id, upload_id] + properties: + request_id: { type: string } + upload_id: { type: string } + + AbortMultipartRsp: + type: object + properties: + header: { $ref: 'openapi-common.yaml#/components/schemas/ResponseHeader' } diff --git a/docs/operations/ci-runtime-repair.md b/docs/operations/ci-runtime-repair.md new file mode 100644 index 0000000..9cd9340 --- /dev/null +++ b/docs/operations/ci-runtime-repair.md @@ -0,0 +1,56 @@ +# CI runtime repair for Issue #88 + +Target version: `3.0-dev` +Status: Current +Reviewed: 2026-09-13 + +Draft PR #89. The owner approved extending #88 on 2026-09-13 to cover defects exposed by the restored runtime gates. This work does not close the larger follow-up Issues or change multi-device coexistence and caller-only message deletion. + +## Reproduced failures and repairs + +| Failure | Cause and repair | Runtime regression | +|---|---|---| +| Agent Policy | The PR Target Version section mixed the branch with prose; it now contains exactly `3.0-dev`. | Local validator RED/GREEN and GitHub Agent Policy run 34715937467 | +| Conversation search | An exact membership filter targeted an analyzed string; use the existing dynamic mapping's `member_ids.keyword`, consistently with other identifier filters. | `TestSearchConversations_Success` and non-member search cases | +| Multipart HTTP failures | Four RPCs had no Gateway routes. Restore JWT-protected routes. Fixtures must use an allowed MIME and valid S3 part sizes. | FN-MD-04 through FN-MD-10; SC-05; FN-MD-22 | +| Multipart PUT HTTP 403 | Upload ID and part number were appended after SigV4 signing. Sign the complete query. | FN-MD-07 downloads and compares the full object; FN-MD-21 rejects a changed part number | +| Avatar cannot be downloaded | Identity fabricated a URL from a file ID, although storage keys use a content hash. Resolve Media's canonical public object URL through discovery. | FN-ID-08 downloads the uploaded bytes and re-reads the profile | +| Missing online notifications | Friend application, accepted application and conversation creation did not call Push. Send bounded best-effort notifications after domain persistence, without broadcasting the creator's personal member state. | FN-WS-02/03/04 | +| Reconnect loses immediate delivery | Push cached an empty route and did not invalidate on authentication. Fence route fills and invalidation with the same per-user lock, and avoid negative route caching. | SC-03, FN-WS-01/05/07 | +| MQ trace absent | Release builds removed the log-context key initialization because it was inside `assert`. Initialize the key unconditionally and abort on allocation failure. Transmite also copies its authenticated trace explicitly into the MQ header. | FN-WS-08 | +| Unread count remains zero | Message persistence never advanced the Conversation watermark. Commit it atomically with message/timeline rows and prevent stale metadata writes from decreasing it. | SC-09 and FN-CC-02 | +| Sending to a dismissed group succeeds | Cold member-cache fills did not validate conversation status. Check the authoritative status before serving membership. | Dismissed-conversation send case and membership regressions | +| Invalid audio reports success | The placeholder ASR handler acknowledged bytes without processing them. Reject malformed PCM16 and return unavailable for valid input until a backend is integrated. | FN-MD-17/18/23; replaces the contradictory fake-audio success assertion | +| Reliability timeout / no rate limiting | A slow serial 650-request stream refilled the default token bucket. The dedicated test stack uses limits 8/40 and a bounded 32-request burst; defaults remain 600/3000. | RL-05 circuit/recovery | +| Redis circuit never opens | Pinned Redis++ wraps exhausted shard refresh attempts in a base `Error`. Classify that exact availability error, while retaining separate command-error handling. | RL-05 circuit counters and fast failure | +| Cache outage masks sequencing failure | An unavailable member snapshot returned before consulting the authoritative service. Mark its version unknown and use an uncached Conversation lookup without publishing an unfenced cache fill. | RL-05 sequencing failure and membership regressions | +| Fast-failure sample takes a recovery probe | Three unrelated outage RPCs crossed the one-second Open deadline before measurement. Measure Open immediately, then exercise those RPCs; retain the 50 ms limit and recovery assertions. | RL-05 fast-failure timing | +| Push reliability socket missing | The test authenticated using a device ID different from its JWT. Use the issued ID and check only the unpersisted outage marker. | RL-05 Push persistence/requeue | +| Push records requeue but loses the message | AMQP-CPP `reject` takes bit flags; boolean `true` does not set `AMQP::requeue`. Use the explicit flag for retry actions and exceptions in both consumer overloads. | RL-05 durable Unacked and post-recovery delivery | +| Readiness exposes bootstrap credentials in process arguments | The MinIO probe bypassed the initializer's stdin-only boundary. Feed `mc alias set` through stdin in the disposable probe container. | `TestReadinessMinIOCredentialsStayOffCommandLine` | +| Readiness deadline cannot interrupt a stalled Docker command | The outer polling deadline only ran after a probe returned. Bound each Compose invocation with GNU `timeout` using the remaining budget. | `TestReadinessDeadlineBoundsStalledProbe` | + +## Compatibility and operational boundaries + +`FileInfo.public_url` is additive field 6. It is populated only for committed public-bucket objects; private objects keep an empty value. Identity returns an empty avatar URL when Media resolution is unavailable. The Media public prefix is authoritative. The HTTP routes, multipart request/response shapes, additive field and ASR failure semantics are recorded in `docs/api/openapi-media.yaml`. + +Message gains only `SELECT, UPDATE` on `conversation` through the existing idempotent principal convergence script. No table migration or production data repair is performed. Deployments must converge this grant before starting the new Message binary. + +The existing Elasticsearch index-creation helper still produces dynamic mappings. This repair follows that deployed field contract; it does not recreate or migrate indexes. A future explicit-mapping migration must update the exact-match queries together. + +Notifications remain online best effort. The existing domain query APIs are authoritative after lost notifications; this patch does not introduce a business-notification outbox or claim durable notification delivery. + +RL-05 pauses all six Redis processes while preserving container DNS and networking. This injects an unresponsive Redis service and tests socket timeouts, circuit opening, recovery and durable Unacked ordering. Stopping the containers also withdraws their DNS records and exposed a separate resolver/restart limitation in the local RED run; pause-based results do not establish stop/recreate recovery or resolve #73. + +## Research + +- [AWS SigV4 query authentication](https://docs.aws.amazon.com/AmazonS3/latest/developerguide/sigv4-query-string-auth.html): the canonical query includes request query parameters before signing. +- [AWS C++ S3Client API](https://docs.aws.amazon.com/sdk-for-cpp/latest/api/aws-cpp-sdk-s3/html/class_aws_1_1_s3_1_1_s3_client.html): endpoint resolution and presigning APIs. +- [Elasticsearch dynamic field mapping](https://www.elastic.co/docs/manage-data/data-store/mapping/dynamic-field-mapping): dynamically mapped strings provide a keyword subfield for exact matching. +- [Pinned Redis++ shard refresh implementation](https://github.com/sewenew/redis-plus-plus/blob/a63ac43bf192772910b52e27cd2b42a6098a0071/src/sw/redis++/shards_pool.cpp): exhausted topology refresh attempts produce the base error `Failed to update shards info`. +- [Docker container pause](https://docs.docker.com/reference/cli/docker/container/pause/): Linux containers use the freezer mechanism to suspend processes; RL-05 uses this fault without removing the container network endpoint. +- [Pinned AMQP-CPP flags](https://github.com/CopernicaMarketingSoftware/AMQP-CPP/blob/ca49382bfc5bc165dfb7988b891bb010a939a786/src/flags.cpp) and [channel API](https://github.com/CopernicaMarketingSoftware/AMQP-CPP/blob/ca49382bfc5bc165dfb7988b891bb010a939a786/include/amqpcpp/channel.h): `reject` requires the `requeue` bit for redelivery; boolean `true` selects no supported rejection flag. + +## Verification record + +Gate results are commit-specific. Consult [PR #89](https://github.com/ULookup/ChatNow/pull/89)'s GREEN Evidence, Regression Verification and current-head checks for execution results. Full commands, exits and logs are retained in the task audit directory `pr89-ci-followup`. Historical passes do not verify later edits, and current-head CI must pass before a completion claim. The PR remains Draft pending human review. diff --git a/docs/operations/compose-runtime.md b/docs/operations/compose-runtime.md new file mode 100644 index 0000000..60d6478 --- /dev/null +++ b/docs/operations/compose-runtime.md @@ -0,0 +1,132 @@ +# Compose Runtime Operations + +Target version: `3.0-dev` +Status: Unverified +Reviewed: 2026-09-13 + +This is the canonical operating contract for the repository-root Compose runtime. Issue #88 has exercised cold startup, all nine native service builds, BVT, Functional, and Redis-focused Reliability on Linux CI. Results are commit-specific and do not validate a release deployment. PR #89 records the exact tested commits and individual gate results. Always obtain fresh evidence for the intended release pair. + +The root profile is a local-development runtime intended for disposable environments. It is not the production HA topology tracked by Issue #73. Issue #88 integrates CI using fresh runner checkouts, shared native service artifacts, and synthetic credentials. Existing environment files and persistent data are never overwritten by the bootstrap helper. + +## Inputs and scope + +The root `docker-compose.yml` requires synthetic local values for the bootstrap and application secret names it references. Supply them through an ignored local environment file or another approved runtime source. Never commit values or copy production credentials into this profile. The authoritative variable inventory and handling rules are in [Runtime Secret Management](runtime-secrets.md). + +The root profile owns the integrated application topology. Do not combine it with `docker/docker-compose.yml`; that supplemental file is not part of the root runtime contract. + +## Startup and convergence + +On a fresh checkout, restore the `compose-service-artifacts` CI artifact under `compose-artifacts`, validate it, and populate the runtime build contexts before starting Compose: + +```bash +./scripts/validate_compose_artifacts.sh compose-artifacts +for service in conversation gateway identity media message presence push relationship transmite; do + mkdir -p "$service/build" "$service/depends" + cp -a "compose-artifacts/$service/build/." "$service/build/" + cp -a "compose-artifacts/$service/depends/." "$service/depends/" +done +``` + +Alternatively compile those nine service targets in the pinned CI builder and run `./scripts/package_compose_artifacts.sh build compose-artifacts`. The runtime Dockerfiles package binaries; they do not compile C++ from a source-only checkout. + +The startup sequence is: + +```bash +docker compose config +docker compose up -d --build +./scripts/wait_for_services.sh +``` + +Each intended checkout and environment requires its own cold-start evidence. `docker compose config` validates interpolation and topology only. `scripts/wait_for_services.sh` is the cross-stack readiness gate; container creation or a successful TCP connection alone is insufficient. + +The root topology uses bounded health checks and one-shot convergence services: + +- `mysql-init` waits for MySQL health, runs the checksum-tracked SQL migrator, converges application users and grants, then exits. Application services that use MySQL wait for `service_completed_successfully`. +- `redis-cluster-init` waits for all six Redis nodes, creates the three-master/three-replica cluster only when no healthy cluster metadata exists, verifies all 16,384 slots, then exits. It refuses to recreate degraded existing metadata. Every node advertises its stable Compose service hostname so persisted cluster metadata does not retain ephemeral container IPs. +- `rabbitmq-init` waits for RabbitMQ health, converges the Transmite, Message, and Push application identities and their scoped permissions through the Management API, then exits. RabbitMQ uses a fixed hostname and node name so its persisted Mnesia path remains stable. +- `minio-init` waits for MinIO health, converges the public and private media buckets, anonymous-access policies, and the Media application identity, then exits. +- Application containers depend on the relevant infrastructure health checks and one-shot initializers. Their shared `entrypoint.sh` performs bounded TCP dependency polling; it is not the semantic full-stack readiness gate. + +Every initializer fails on missing required inputs and must remain non-resident. The Redis, RabbitMQ, and MinIO initializers also fail when their bounded attempt limits are exhausted. Fixed startup sleeps and `tail -f` sentinels are not readiness mechanisms. + +## MySQL bootstrap and migrations + +`mysql-init` mounts `sql/` read-only and executes `scripts/init_mysql.sh`. The runner creates `chatnow.schema_migrations`, computes a SHA-256 checksum for every ordered `V*.sql` file, skips an already-applied version only when its checksum matches, and fails closed if an applied migration was changed. New schema changes require a new versioned file; never edit an applied migration. + +The current ODB object set contains 17 tables: + +```text +conversation +conversation_member +friend_apply +media_blob_ref +media_file +media_user_quota +message +message_attachment +message_mention +message_pin +message_reaction +message_read +relation +user +user_block +user_device +user_timeline +``` + +`sql/V1__core.sql` owns the 14 non-media tables and `sql/V4__media.sql` owns the three media tables. Their table creation is idempotent. After schema convergence, `scripts/converge_mysql_users.sh` creates or updates five application identities (`chatnow_identity`, `chatnow_conversation`, `chatnow_relationship`, `chatnow_message`, and `chatnow_media`) and reapplies their table-scoped grants. + +This is a small forward-only repository migrator, not a general rollback engine. MySQL DDL is not treated as transactional across a whole migration and the migration record insertion. Preserve a database backup before operating on non-disposable data, stop on checksum mismatch or partial DDL, and resolve the state explicitly rather than deleting the migration ledger. + +## Media and MinIO endpoints + +Media has two endpoint roles: + +| Role | Local process configuration | Root Compose container configuration | Owner | +|---|---|---|---| +| Internal S3 operations | `http://127.0.0.1:19000` | `http://minio:9000` | Media process to MinIO | +| Client-facing presigned URLs | `http://127.0.0.1:19000` | `http://127.0.0.1:19000` | Test or local client to published MinIO S3 port | + +`conf/media.json` is the local-process configuration. Root Compose mounts `conf/docker/media.json` read-only into Media. `S3Client` uses the internal endpoint for bucket and object operations and a separate presign client for generated URLs. The public endpoint must be reachable by the actual client; the loopback value is valid only for clients running on the Compose host and must be replaced by deployment-specific configuration elsewhere. + +Root Compose publishes MinIO S3 on `127.0.0.1:19000` and its console on `127.0.0.1:19001`, leaving Gateway HTTP on `9000` and Push WebSocket on `9001`. + +## RabbitMQ host contract + +`mq_host` is a host name or address only. Transmite, Message, and Push builders append the fixed AMQP port `5672` when constructing the URL and percent-encode userinfo credentials. Container configuration therefore uses `rabbitmq`; local configuration uses `127.0.0.1`. Supplying `host:5672` produces an invalid double-port URL and is rejected. + +## Semantic readiness + +`scripts/wait_for_services.sh` uses one bounded deadline and polls the following observable conditions: + +- Redis reports `cluster_state:ok`, all 16,384 slots assigned and healthy, and exactly six known nodes. +- MySQL contains all 17 ODB tables and the five required application users. +- RabbitMQ is running without local alarms and all three scoped application identities have permissions. +- Elasticsearch reaches yellow or green cluster health. +- MinIO reports ready and both required buckets are addressable. +- etcd contains exactly the eight expected service registration keys for Identity, Media, Transmite, Message, Relationship, Conversation, Presence, and Push. +- unauthenticated Gateway `GET /health` returns success. +- the Push WebSocket listener is reachable on host port `9001`. + +Each Docker Compose invocation is limited to the remaining readiness budget through GNU `timeout`, with a one-second kill grace period. The host therefore requires GNU coreutils in addition to Docker, curl, Bash, and netcat. The MinIO bucket probe sends its synthetic bootstrap identity through standard input to `mc`, never through process arguments; its temporary container is removed after the probe. + +Gateway `/health` is dependency-aware, not a process-liveness response. It returns HTTP `200` only when the Gateway service manager currently has at least one discovered channel for each of the eight business services; it returns HTTP `503` otherwise. It does not replace the stateful infrastructure probes above. + +Channel presence does not prove RPC reachability after container IP changes. The current brpc channel cache can retain an old IP for a reused service hostname. Redis Cluster metadata can also retain old peer IPs after a simultaneous container restart. Do not treat a successful cold start as rolling-update or restart-recovery evidence. Keep persisted data and investigate stale discovery/cluster addresses before attempting recovery; never recreate an existing cluster merely to make readiness pass. + +The Push check is currently bounded TCP reachability because Push has no separate semantic health endpoint. Do not describe that individual probe as end-to-end WebSocket delivery evidence. + +## Shutdown and clean-state boundary + +Stop the profile with: + +```bash +docker compose down +``` + +The root profile persists infrastructure under `middle/data` through bind mounts. `docker compose down -v` does not remove bind-mounted state and therefore does not prove a clean-slate restart. Do not delete a shared `middle/data` tree or claim repeatable cold-start behavior without an explicitly disposable path, a fresh CI checkout, and fresh dynamic evidence. + +## Evidence required before a readiness claim + +Static Compose contracts, shell syntax, formatting, compilation, and `docker compose config` are supporting checks only. A full runtime claim requires fresh evidence for the exact commit from an empty disposable state, successful `scripts/wait_for_services.sh`, and the applicable Go gates. Report each gate as passed, failed, blocked, or not run from its actual result. Any applicable failure keeps the release verdict `Unverified`. diff --git a/docs/operations/runtime-secrets.md b/docs/operations/runtime-secrets.md index 49f69f6..e20d1fd 100644 --- a/docs/operations/runtime-secrets.md +++ b/docs/operations/runtime-secrets.md @@ -17,8 +17,8 @@ The following behavior is implemented at the verified commit: | RabbitMQ password | Transmite, Message, and Push | Service-specific direct or `_FILE` input through the common resolver | | SMTP password | Identity | `CHATNOW_IDENTITY_SMTP_PASSWORD` or its `_FILE` companion | | S3 application access and secret keys | Media | Separate service-specific direct or `_FILE` inputs; non-secret S3 settings remain in `conf/media.json` | -| MinIO bootstrap credential | Supplemental MinIO deployment | Required Compose deployment environment references | -| MySQL root and RabbitMQ bootstrap passwords | Root Compose infrastructure | Required Compose deployment environment references | +| MinIO bootstrap credential | Root Compose `minio` and `minio-init` services | Required Compose deployment environment references | +| MySQL root and RabbitMQ bootstrap passwords | Root Compose infrastructure and one-shot initializers | Required Compose deployment environment references | Redis has no configured password or ACL consumer. The ASR helper can accept credentials, but Media startup does not currently wire them. The CI workflow does not pull real credentials; local and CI stack inputs must be synthetic. @@ -41,7 +41,17 @@ Tracked runtime literals have been removed from the scoped source, configuration | Media S3 access key | `CHATNOW_MEDIA_S3_ACCESS_KEY` | `CHATNOW_MEDIA_S3_ACCESS_KEY_FILE` | | Media S3 secret key | `CHATNOW_MEDIA_S3_SECRET_KEY` | `CHATNOW_MEDIA_S3_SECRET_KEY_FILE` | -Compose bootstrap variables (`CHATNOW_MYSQL_ROOT_PASSWORD`, `CHATNOW_RABBITMQ_BOOTSTRAP_PASSWORD`, `CHATNOW_MINIO_ROOT_USER`, and `CHATNOW_MINIO_ROOT_PASSWORD`) are required deployment inputs, not common-resolver inputs. Do not append `_FILE` and assume Compose supports it. +Compose bootstrap variables (`CHATNOW_MYSQL_ROOT_PASSWORD`, `CHATNOW_RABBITMQ_BOOTSTRAP_PASSWORD`, `CHATNOW_MINIO_ROOT_USER`, and `CHATNOW_MINIO_ROOT_PASSWORD`) are required deployment inputs, not common-resolver inputs. `CHATNOW_RABBITMQ_BOOTSTRAP_USER` may override the synthetic local bootstrap user name. Do not append `_FILE` and assume Compose supports it. + +The root Compose one-shot initializers receive only the credentials they need to converge disposable local application identities: + +- `mysql-init` uses the MySQL root credential plus the five service-specific MySQL passwords to create or update table-scoped users and grants. +- `rabbitmq-init` uses the RabbitMQ bootstrap credential plus the Transmite, Message, and Push passwords to create or update scoped users and permissions. +- `minio-init` uses the MinIO root credential plus the Media S3 application credentials to create or update the two buckets, policies, and application identity. + +RabbitMQ secrets are sent in Management API JSON bodies rather than command arguments. MinIO feeds root and application secret material to `mc` through standard input, isolates `mc` state in a temporary configuration directory, and removes it on exit. These initialization boundaries do not make bootstrap credentials application inputs. Application containers continue to receive only their own direct or `_FILE` resolver inputs. See [Compose Runtime Operations](compose-runtime.md) for ordering and readiness; neither document is evidence that a cold start has passed. + +The MinIO readiness probe follows the same stdin-only credential boundary. It runs in a temporary `docker compose run --rm` container; the root identity and password must never become `mc alias set` arguments. ## Current injection contract @@ -76,6 +86,7 @@ Never reuse a bootstrap credential as an application credential. Never mount a f - Use unique synthetic values generated for the disposable environment. They must not be copied from staging or production and must carry no external privilege. - Inject them at runtime through ignored local environment files or ephemeral secret mounts. `.env` being ignored does not make it an approved production store. - CI must source synthetic values from ephemeral job setup or the CI secret mechanism, mask values, avoid command tracing, and tear down volumes and temporary files on every exit path. +- CI generates disposable values with `scripts/create_test_env.py --github-env`, registers masks, and injects them into each job. The generator refuses an existing `.env` or `middle/data`; gate results must still be verified for the current commit. - Scanner exemptions must match exact synthetic fixtures or documented API examples. Do not exempt an entire `conf/`, `tests/`, `docs/`, Compose, or source subtree. - Tests may assert source selection and error categories, but must not print the resolved value or any derivative. diff --git a/entrypoint.sh b/entrypoint.sh index eac2d0b..1dc33ff 100755 --- a/entrypoint.sh +++ b/entrypoint.sh @@ -1,38 +1,41 @@ -#!/bin/bash -# 端口检测函数：等待指定 host:port 可达 -wait_for() { - local host=$1 - local port=$2 - while ! nc -z $host $port - do - echo "$host:$port 端口连接失败，休眠等待"; - sleep 1; - done - echo "$host:$port 检测成功"; -} +#!/usr/bin/env bash -# 解析参数 -declare deps -declare command -while getopts "d:c:" arg -do - case $arg in - d) - deps=$OPTARG;; - c) - command=$OPTARG;; - esac -done +set -euo pipefail + +DEPENDENCY_WAIT_TIMEOUT_SEC=${DEPENDENCY_WAIT_TIMEOUT_SEC:-120} +deps="" +command="" -# 对每个 host:port 对进行端口检测 -for dep in ${deps//,/ } -do - host=${dep%:*} - port=${dep#*:} - wait_for $host $port +while getopts "d:c:" arg; do + case "$arg" in + d) deps=$OPTARG ;; + c) command=$OPTARG ;; + *) exit 2 ;; + esac done -echo "端口检测完毕" +if [[ -z "$command" ]]; then + echo "entrypoint: service command is required" >&2 + exit 2 +fi + +deadline=$((SECONDS + DEPENDENCY_WAIT_TIMEOUT_SEC)) +for dependency in ${deps//,/ }; do + host=${dependency%:*} + port=${dependency##*:} + if [[ -z "$host" || -z "$port" || "$host" == "$dependency" ]]; then + echo "entrypoint: invalid dependency locator" >&2 + exit 2 + fi + + until nc -z -w 1 "$host" "$port"; do + if (( SECONDS >= deadline )); then + echo "entrypoint: dependency did not become reachable: ${host}:${port}" >&2 + exit 1 + fi + sleep 1 + done +done -# 执行命令 -eval $command +echo "entrypoint: dependencies are reachable" +exec /bin/bash -c "$command" diff --git a/gateway/CMakeLists.txt b/gateway/CMakeLists.txt index f375952..fa52f8a 100644 --- a/gateway/CMakeLists.txt +++ b/gateway/CMakeLists.txt @@ -34,11 +34,12 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) # 5. 声明目标及依赖 add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcurl -lodb-mysql -lodb -lodb-boost - -lhiredis -lcpr -lelasticlient -ljsoncpp + -lhiredis -lcpr -lelasticlient -lhiredis -lredis++ -lpthread -lboost_system -lcpprest -lcurl) @@ -57,4 +58,4 @@ include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../common) include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../third/include) # 7. 设置需要连接的库 # 8. 设置安装路径 -INSTALL(TARGETS ${target} RUNTIME DESTINATION bin) \ No newline at end of file +INSTALL(TARGETS ${target} RUNTIME DESTINATION bin) diff --git a/gateway/Dockerfile b/gateway/Dockerfile index 9a1473e..e3c8b91 100644 --- a/gateway/Dockerfile +++ b/gateway/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/gateway_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/gateway_server -flagfile=/im/conf/gateway_server.conf diff --git a/gateway/source/gateway_server.h b/gateway/source/gateway_server.h index 077e01b..efdb520 100644 --- a/gateway/source/gateway_server.h +++ b/gateway/source/gateway_server.h @@ -57,6 +57,7 @@ class GatewayServer { GatewayServer(int http_port, const ServiceManager::ptr &channels, + const Discovery::ptr &discovery, const std::shared_ptr<::chatnow::auth::JwtCodec> &jwt_codec, const std::shared_ptr<::chatnow::auth::JwtStore> &jwt_store, const std::string &identity_service_name, @@ -68,7 +69,7 @@ class GatewayServer { const std::string &presence_service_name, const std::string &push_service_name) : _jwt_codec(jwt_codec), _jwt_store(jwt_store), - _channels(channels), _http_port(http_port), + _channels(channels), _discovery(discovery), _http_port(http_port), _identity_svc(identity_service_name), _relationship_svc(relationship_service_name), _conversation_svc(conversation_service_name), @@ -194,6 +195,18 @@ class GatewayServer { matched->handler(req, res, a, _channels, matched->timeout_ms, dummy_cntl); } + bool dependencies_ready() const { + return _channels && + _channels->available(_identity_svc) && + _channels->available(_relationship_svc) && + _channels->available(_conversation_svc) && + _channels->available(_message_svc) && + _channels->available(_transmite_svc) && + _channels->available(_media_svc) && + _channels->available(_presence_svc) && + _channels->available(_push_svc); + } + // ====== 路由注册 ====== void register_routes(); @@ -210,6 +223,7 @@ class GatewayServer { std::shared_ptr<::chatnow::auth::JwtCodec> _jwt_codec; std::shared_ptr<::chatnow::auth::JwtStore> _jwt_store; ServiceManager::ptr _channels; + Discovery::ptr _discovery; int _http_port; std::string _identity_svc; @@ -233,6 +247,18 @@ inline void GatewayServer::register_routes() { namespace med = ::chatnow::media; namespace pres = ::chatnow::presence; + _http_server.Get("/health", + [this](const httplib::Request&, httplib::Response& res) { + res.set_header("Cache-Control", "no-store"); + if (!dependencies_ready()) { + res.status = 503; + res.set_content("{\"status\":\"unavailable\"}", "application/json"); + return; + } + res.status = 200; + res.set_content("{\"status\":\"ready\"}", "application/json"); + }); + // ====== Identity (白名单) ====== route( "/service/identity/register", _identity_svc, GatewayAuth::WHITELISTED, @@ -414,6 +440,19 @@ inline void GatewayServer::register_routes() { route( "/service/media/get_file_info", _media_svc, GatewayAuth::JWT_REQUIRED, &med::MediaService_Stub::GetFileInfo); + route( + "/service/media/init_multipart", _media_svc, GatewayAuth::JWT_REQUIRED, + &med::MediaService_Stub::InitMultipartUpload); + route( + "/service/media/apply_part_upload", _media_svc, GatewayAuth::JWT_REQUIRED, + &med::MediaService_Stub::ApplyPartUpload); + route( + "/service/media/complete_multipart", _media_svc, GatewayAuth::JWT_REQUIRED, + &med::MediaService_Stub::CompleteMultipartUpload); + route( + "/service/media/abort_multipart", _media_svc, GatewayAuth::JWT_REQUIRED, + &med::MediaService_Stub::AbortMultipartUpload); + route( "/service/media/speech_recognition", _media_svc, GatewayAuth::JWT_REQUIRED, &med::MediaService_Stub::SpeechRecognition); @@ -521,7 +560,7 @@ class GatewayServerBuilder { auto discovery = std::make_shared(_reg_host, _base, put_cb, del_cb); return std::make_shared( - _http_port, channels, jwt_codec, jwt_store, + _http_port, channels, discovery, jwt_codec, jwt_store, _identity_svc, _relationship_svc, _conversation_svc, _message_svc, _transmite_svc, _media_svc, _presence_svc, _push_svc); diff --git a/identity/CMakeLists.txt b/identity/CMakeLists.txt index d790088..fe113fd 100644 --- a/identity/CMakeLists.txt +++ b/identity/CMakeLists.txt @@ -7,7 +7,7 @@ set(target "identity_server") # 3. 检测并生成框架代码 # 3.1 添加所需的proto映射代码文件名称 set(proto_path ${CMAKE_CURRENT_SOURCE_DIR}/../proto) -set(proto_files common/types.proto common/error.proto common/envelope.proto common/auth/metadata.proto identity/identity_service.proto) +set(proto_files common/types.proto common/error.proto common/envelope.proto common/auth/metadata.proto identity/identity_service.proto media/media_service.proto) # 3.2 检测框架代码文件是否已经生成 set(proto_srcs "") foreach(proto_file ${proto_files}) @@ -60,11 +60,12 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) # 5. 声明目标及依赖 add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lodb-mysql -lodb -lodb-boost - -lredis++ -lhiredis -lcpr -lelasticlient -ljsoncpp + -lredis++ -lhiredis -lcpr -lelasticlient -lcrypt) # FIXME(3.0): identity_client test code is stale (references pre-3.0 APIs: MailRegisterReq, UserService_Stub, etc.) diff --git a/identity/Dockerfile b/identity/Dockerfile index c26649a..7b580e3 100644 --- a/identity/Dockerfile +++ b/identity/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/identity_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y ca-certificates netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/identity_server -flagfile=/im/conf/identity_server.conf diff --git a/identity/source/identity_server.h b/identity/source/identity_server.h index 862a4e7..0f74aa9 100644 --- a/identity/source/identity_server.h +++ b/identity/source/identity_server.h @@ -14,6 +14,9 @@ #include "common/error.pb.h" #include "common/envelope.pb.h" #include "identity/identity_service.pb.h" +#include "media/media_service.pb.h" +#include "auth/forward_auth.hpp" +#include "mq/channel.hpp" #include "auth/auth_context.hpp" #include "auth/jwt_codec.hpp" @@ -43,7 +46,8 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService const std::shared_ptr &mail_client, const std::shared_ptr &jwt_codec, const std::shared_ptr &jwt_store, - const std::string &media_public_url_prefix, + const ServiceManager::ptr &channels, + const std::string &media_service_name, const ESOutbox::ptr &es_outbox, const std::shared_ptr &es_reaper_client) : _mysql_user(std::make_shared(mysql_client)), @@ -53,7 +57,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService _mail_client(mail_client), _jwt_codec(jwt_codec), _jwt_store(jwt_store), - _media_public_url_prefix(media_public_url_prefix), + _channels(channels), _media_service_name(media_service_name), _es_outbox(es_outbox), _es_user_reaper(std::make_shared(es_reaper_client)) { @@ -443,7 +447,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService throw ServiceError(::chatnow::error::kAuthUserNotFound, "user not found: " + uid); } - fill_user_info(response->mutable_user_info(), *user); + fill_user_info(response->mutable_user_info(), *user, cntl); }); } @@ -496,7 +500,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService user->nickname(), user->description(), user->avatar_id()); }); - fill_user_info(response->mutable_user_info(), *user); + fill_user_info(response->mutable_user_info(), *user, cntl); }); } @@ -516,7 +520,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService auto* user_map = response->mutable_users_info(); for (auto& u : users) { ::chatnow::common::UserInfo ui; - fill_user_info(&ui, u); + fill_user_info(&ui, u, cntl); (*user_map)[ui.user_id()] = ui; } }); @@ -532,7 +536,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService HANDLE_RPC(cntl, request, response, { auto users = _es_user->search(request->search_key(), {}, 20); for (auto& u : users) { - fill_user_info(response->add_user_info(), u); + fill_user_info(response->add_user_info(), u, cntl); } }); } @@ -545,7 +549,8 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService std::shared_ptr _mail_client; std::shared_ptr _jwt_codec; std::shared_ptr _jwt_store; - std::string _media_public_url_prefix; + ServiceManager::ptr _channels; + std::string _media_service_name; ESOutbox::ptr _es_outbox; std::shared_ptr _es_user_reaper; std::shared_ptr> _es_reaper_running; @@ -622,18 +627,30 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService // ---- 工具方法 ---- std::string uuid() { return ::chatnow::uuid(); } - std::string make_avatar_url(const std::string &avatar_id) { - if (avatar_id.empty() || _media_public_url_prefix.empty()) return ""; - return _media_public_url_prefix + "/" + avatar_id; + std::string make_avatar_url(const std::string &avatar_id, brpc::Controller* incoming) { + if (avatar_id.empty()) return ""; + auto channel = _channels->choose(_media_service_name); + if (!channel) return ""; + ::chatnow::media::MediaService_Stub stub(channel.get()); + ::chatnow::media::GetFileInfoReq request; + ::chatnow::media::GetFileInfoRsp response; + request.set_request_id(uuid()); + request.set_file_id(avatar_id); + brpc::Controller outgoing; + outgoing.set_timeout_ms(1000); + ::chatnow::auth::forward_auth_metadata(incoming, &outgoing); + stub.GetFileInfo(&outgoing, &request, &response, nullptr); + if (outgoing.Failed() || !response.header().success()) return ""; + return response.file_info().public_url(); } // ---- UserInfo 组装（后续 RPC 共用） ---- - void fill_user_info(::chatnow::common::UserInfo *u, const User &user) { + void fill_user_info(::chatnow::common::UserInfo *u, const User &user, brpc::Controller* incoming) { u->set_user_id(user.user_id()); u->set_nickname(user.nickname()); if (!user.description().empty()) u->set_bio(user.description()); if (!user.phone().empty()) u->set_phone(user.phone()); - u->set_avatar_url(make_avatar_url(user.avatar_id())); + u->set_avatar_url(make_avatar_url(user.avatar_id(), incoming)); } }; @@ -747,11 +764,14 @@ class IdentityServerBuilder void make_discovery_object(const std::string ®_host, const std::string &base_service_name) { - auto put_cb = [](const std::string &name, const std::string &host) { - LOG_INFO("Discovery put: {} @ {}", name, host); + _channels = std::make_shared(); + _media_service_name = base_service_name + "/media_service"; + _channels->declared(_media_service_name); + auto put_cb = [channels = _channels](const std::string &name, const std::string &host) { + channels->onServiceOnline(name, host); }; - auto del_cb = [](const std::string &name, const std::string &host) { - LOG_INFO("Discovery del: {} @ {}", name, host); + auto del_cb = [channels = _channels](const std::string &name, const std::string &host) { + channels->onServiceOffline(name, host); }; _service_discover = std::make_shared(reg_host, base_service_name, put_cb, del_cb); } @@ -793,7 +813,7 @@ class IdentityServerBuilder auto es_reaper_client = ESClientFactory::create(_es_hosts); IdentityServiceImpl *identity_service = new IdentityServiceImpl( _mysql_client, _es_client, _redis_client, _user_info_cache, _mail_client, - _jwt_codec, _jwt_store, _media_public_url_prefix, + _jwt_codec, _jwt_store, _channels, _media_service_name, _es_outbox, es_reaper_client); _service_impl = identity_service; int ret = _rpc_server->AddService(identity_service, brpc::ServiceOwnership::SERVER_OWNS_SERVICE); @@ -841,6 +861,8 @@ class IdentityServerBuilder std::shared_ptr _mail_client; std::string _media_public_url_prefix; + ServiceManager::ptr _channels; + std::string _media_service_name; Discovery::ptr _service_discover; std::shared_ptr<::chatnow::auth::JwtCodec> _jwt_codec; diff --git a/media/CMakeLists.txt b/media/CMakeLists.txt index 38855f8..ac8a52b 100644 --- a/media/CMakeLists.txt +++ b/media/CMakeLists.txt @@ -66,7 +66,7 @@ find_package(jsoncpp CONFIG REQUIRED) add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) target_link_libraries(${target} - -lgflags -lspdlog -lfmt + -lgflags -lspdlog -lfmt -lglog -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lodb-mysql -lodb -lodb-boost diff --git a/media/Dockerfile b/media/Dockerfile index ae3c167..8ed8309 100644 --- a/media/Dockerfile +++ b/media/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/media_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/media_server -flagfile=/im/conf/media_server.conf diff --git a/media/source/download_handler.hpp b/media/source/download_handler.hpp index 637dffd..f694256 100644 --- a/media/source/download_handler.hpp +++ b/media/source/download_handler.hpp @@ -28,8 +28,13 @@ class DownloadHandler { public: DownloadHandler(std::shared_ptr s3, std::shared_ptr files, - int presign_seconds) - : _s3(std::move(s3)), _files(std::move(files)), _presign(presign_seconds) {} + int presign_seconds, + std::string public_bucket, std::string public_url_prefix) + : _s3(std::move(s3)), _files(std::move(files)), _presign(presign_seconds), + _public_bucket(std::move(public_bucket)), _public_url_prefix(std::move(public_url_prefix)) { + while (!_public_url_prefix.empty() && _public_url_prefix.back() == '/') + _public_url_prefix.pop_back(); + } void apply(const std::string& user_id, const ::chatnow::media::ApplyDownloadReq& req, @@ -46,6 +51,8 @@ class DownloadHandler { rsp->set_download_url(url); rsp->set_expires_in_sec(_presign); fill_file_info(*file, rsp->mutable_file_info()); + if (file->bucket() == _public_bucket && !_public_url_prefix.empty()) + rsp->mutable_file_info()->set_public_url(_public_url_prefix + "/" + file->object_key()); LOG_INFO("apply_download user={} file={} size={}", user_id, file->file_id(), file->file_size()); } @@ -58,12 +65,16 @@ class DownloadHandler { throw ServiceError(::chatnow::error::kMediaFileNotFound, "no such file"); } fill_file_info(*file, rsp->mutable_file_info()); + if (file->bucket() == _public_bucket && !_public_url_prefix.empty()) + rsp->mutable_file_info()->set_public_url(_public_url_prefix + "/" + file->object_key()); } private: std::shared_ptr _s3; std::shared_ptr _files; int _presign; + std::string _public_bucket; + std::string _public_url_prefix; }; } // namespace chatnow diff --git a/media/source/media_main.cc b/media/source/media_main.cc index dc9825d..358ef96 100644 --- a/media/source/media_main.cc +++ b/media/source/media_main.cc @@ -54,6 +54,7 @@ struct LoadedMediaConf { chatnow::MediaServiceConfig cfg; std::shared_ptr mime; std::string s3_endpoint; + std::string s3_public_endpoint; std::string s3_region; }; @@ -75,8 +76,13 @@ LoadedMediaConf load_media_conf(const std::string& path) { LoadedMediaConf out; out.s3_endpoint = s3.get("endpoint", "").asString(); + out.s3_public_endpoint = s3.get("public_endpoint", "").asString(); out.s3_region = s3.get("region", "us-east-1").asString(); + if (out.s3_endpoint.empty() || out.s3_public_endpoint.empty()) { + throw std::runtime_error("media_conf: s3_endpoint_invalid"); + } + out.cfg.public_bucket = md.get("public_bucket", "").asString(); out.cfg.private_bucket = md.get("private_bucket", "").asString(); out.cfg.public_url_prefix = md.get("public_url_prefix", "").asString(); @@ -124,7 +130,7 @@ int main(int argc, char* argv[]) { b.set_redis_seeds(FLAGS_redis_seeds); b.make_redis_object(FLAGS_redis_host, FLAGS_redis_port, FLAGS_redis_db, FLAGS_redis_keep_alive); - b.make_s3_object(conf.s3_endpoint, conf.s3_region, + b.make_s3_object(conf.s3_endpoint, conf.s3_public_endpoint, conf.s3_region, s3_access_key, s3_secret_key); b.set_media_config(conf.cfg, conf.mime); b.make_registry_object(FLAGS_registry_host, diff --git a/media/source/media_server.h b/media/source/media_server.h index 6d41a54..18381ef 100644 --- a/media/source/media_server.h +++ b/media/source/media_server.h @@ -72,7 +72,8 @@ class MediaServiceImpl : public ::chatnow::media::MediaService { _multi = std::make_unique( s3, mime, files, blobs, quota, cfg.public_bucket, cfg.private_bucket, cfg.presign_seconds); - _download = std::make_unique(s3, files, cfg.presign_seconds); + _download = std::make_unique(s3, files, cfg.presign_seconds, + cfg.public_bucket, cfg.public_url_prefix); _speech = std::make_unique(cfg.asr_endpoint); } @@ -295,9 +296,11 @@ class MediaServerBuilder { } } - void make_s3_object(const std::string& endpoint, const std::string& region, + void make_s3_object(const std::string& endpoint, const std::string& public_endpoint, + const std::string& region, const std::string& access_key, const std::string& secret_key) { - S3Options o{endpoint, region, access_key, secret_key, /*path_style*/true}; + S3Options o{endpoint, public_endpoint, region, access_key, secret_key, + /*path_style*/true}; _s3 = std::make_shared(o); } diff --git a/media/source/speech_handler.hpp b/media/source/speech_handler.hpp index e2e2609..c18cef2 100644 --- a/media/source/speech_handler.hpp +++ b/media/source/speech_handler.hpp @@ -1,13 +1,7 @@ #pragma once -/** - * SpeechHandler —— 短音频 ASR 占位 - * --- - * P4 v1 仅保留 RPC + bytes 字段，转发到 ASR 引擎的实际实现 - * 在 P7 完成（spec §3.8）。本类做： - * - bytes 长度上限 2MB（speech_content > 2MB 直接拒） - * - 返回空 recognition_result - */ +// The ASR backend is not wired yet. Validate the PCM16 payload and report +// unavailability instead of acknowledging audio that was never processed. #include #include @@ -28,8 +22,13 @@ class SpeechHandler { if (req.speech_content().size() > 2 * 1024 * 1024) { throw ServiceError(::chatnow::error::kMediaFileTooLarge, "speech > 2MB"); } - // P7：调真实 ASR endpoint；当前仅返回空字符串 - rsp->set_recognition_result(""); + if (req.speech_content().empty() || req.speech_content().size() % 2 != 0) { + throw ServiceError(::chatnow::error::kSystemInvalidArgument, + "non-empty PCM16 content required"); + } + // No ASR engine is wired. Do not acknowledge unprocessed audio as success. + throw ServiceError(::chatnow::error::kSystemUnavailable, + "speech recognition unavailable"); } private: diff --git a/message/CMakeLists.txt b/message/CMakeLists.txt index 08688ce..80a7bf7 100644 --- a/message/CMakeLists.txt +++ b/message/CMakeLists.txt @@ -60,12 +60,13 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) # 5. 声明目标及依赖 add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lodb-mysql -lodb -lodb-boost - -lhiredis -lredis++ -lcpr -lelasticlient -ljsoncpp - -lamqpcpp -lev -lmysqlclient) + -lhiredis -lredis++ -lcpr -lelasticlient + -lamqpcpp -lev -lmysqlclient -lglog) set(test_client "message_client") # 4. 获取源码目录下的所有源码文件 @@ -90,4 +91,4 @@ if(test_files) INSTALL(TARGETS ${target} ${test_client} RUNTIME DESTINATION bin) else() INSTALL(TARGETS ${target} RUNTIME DESTINATION bin) -endif() \ No newline at end of file +endif() diff --git a/message/Dockerfile b/message/Dockerfile index f01876a..9ca4b64 100644 --- a/message/Dockerfile +++ b/message/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/message_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/message_server -flagfile=/im/conf/message_server.conf diff --git a/message/source/message_server.cc b/message/source/message_server.cc index b4e4663..58fc9ab 100644 --- a/message/source/message_server.cc +++ b/message/source/message_server.cc @@ -26,7 +26,7 @@ DEFINE_int32(mysql_port, 0, "MySQL服务器访问端口"); DEFINE_int32(mysql_pool_count, 4, "MySQL连接池最大连接数量"); DEFINE_string(mq_user, "root", "消息队列服务器访问用户名"); -DEFINE_string(mq_host, "127.0.0.1:5672", "消息队列服务器访问地址"); +DEFINE_string(mq_host, "127.0.0.1", "RabbitMQ hostname (port is fixed at 5672)"); DEFINE_string(mq_msg_exchange, "chat_msg_exchange", "持久化消息的发布交换机名称"); DEFINE_string(mq_msg_queue_db, "msg_queue_db", "持久化DB消息的发布队列名称"); DEFINE_string(mq_msg_queue_es, "msg_queue_es", "持久化ES消息的发布队列名称"); diff --git a/message/source/message_server.h b/message/source/message_server.h index e3b2a32..9b45ded 100644 --- a/message/source/message_server.h +++ b/message/source/message_server.h @@ -32,6 +32,7 @@ #include "dao/mysql_message.hpp" #include "dao/mysql_user_timeline.hpp" #include "dao/mysql_conversation_member.hpp" +#include "conversation-odb.hxx" #include "dao/mysql_message_reaction.hpp" #include "dao/mysql_message_pin.hpp" #include "dao/data_es.hpp" @@ -559,6 +560,17 @@ class MessageServiceImpl : public chatnow::message::MessageService { if (!timeline_list.empty()) { _mysql_user_timeline->insert(timeline_list); } + // Commit the conversation watermark with the message and timelines. + // A row lock and monotonic update make duplicate/reordered delivery safe. + using conversation_query = odb::query<::chatnow::Conversation>; + std::shared_ptr<::chatnow::Conversation> conversation( + _odb_db->query_one<::chatnow::Conversation>( + (conversation_query::conversation_id == msg_pb.conversation_id()) + " FOR UPDATE")); + if (!conversation) throw std::runtime_error("conversation missing during persistence"); + if (conversation->max_seq() < session_seq) { + conversation->max_seq(session_seq); + _odb_db->update(*conversation); + } if (trans) trans->commit(); mark_idempotency_persisted_(msg_pb.sender_id(), client_msg_id, msg_pb.message_id()); } catch (const odb::object_already_persistent &) { @@ -1000,7 +1012,7 @@ class MessageServerBuilder { LOG_ERROR("Message MQ exchange 不能为空"); abort(); } - std::string amqp_url = "amqp://" + user + ":" + pwd + "@" + host + ":5672/"; + std::string amqp_url = make_amqp_url(user, pwd, host); _mq_client = std::make_shared(amqp_url); _db_queue_settings = { .exchange = exchange_name, diff --git a/presence/CMakeLists.txt b/presence/CMakeLists.txt index 3f8a372..9102652 100644 --- a/presence/CMakeLists.txt +++ b/presence/CMakeLists.txt @@ -38,7 +38,8 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) add_executable(${target} ${src_files} ${proto_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcurl -lhiredis -lredis++ -lpthread -lboost_system -lcpprest) diff --git a/presence/Dockerfile b/presence/Dockerfile index 71af26a..0e1b161 100644 --- a/presence/Dockerfile +++ b/presence/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/presence_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/presence_server -flagfile=/im/conf/presence_server.conf diff --git a/presence/source/presence_server.h b/presence/source/presence_server.h index ef02b5d..5523a75 100644 --- a/presence/source/presence_server.h +++ b/presence/source/presence_server.h @@ -75,8 +75,14 @@ class PresenceAggregator { if (!state_opt) continue; - int state_val = std::stoi(*state_opt); - PresenceState dev_state = static_cast(state_val); + PresenceState dev_state = PresenceState::PRESENCE_UNSPECIFIED; + // Push writes enum names; retain numeric values used by older writers. + if (!PresenceState_Parse(*state_opt, &dev_state)) { + if (state_opt->size() != 1 || (*state_opt)[0] < '0' || (*state_opt)[0] > '5') { + continue; + } + dev_state = static_cast((*state_opt)[0] - '0'); + } // TTL 检查 if (last_opt) { diff --git a/proto/media/media_service.proto b/proto/media/media_service.proto index e04e483..66aea9d 100644 --- a/proto/media/media_service.proto +++ b/proto/media/media_service.proto @@ -36,6 +36,8 @@ message FileInfo { int64 file_size = 3; string mime_type = 4; int64 uploaded_at_ms = 5; + // Canonical unsigned URL for committed public objects; empty for private files. + string public_url = 6; } // ===== 单段三步上传 ===== diff --git a/push/CMakeLists.txt b/push/CMakeLists.txt index 0e326fe..0d830b1 100644 --- a/push/CMakeLists.txt +++ b/push/CMakeLists.txt @@ -30,11 +30,12 @@ set(src_files "") aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) add_executable(${target} ${src_files} ${proto_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lamqpcpp -lev - -lhiredis -lredis++ -ljsoncpp + -lhiredis -lredis++ -lpthread -lboost_system) target_include_directories(${target} PRIVATE diff --git a/push/Dockerfile b/push/Dockerfile index 103eb1c..e554871 100644 --- a/push/Dockerfile +++ b/push/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/push_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/push_server -flagfile=/im/conf/push_server.conf diff --git a/push/source/push_server.cc b/push/source/push_server.cc index 09d3958..5d93690 100644 --- a/push/source/push_server.cc +++ b/push/source/push_server.cc @@ -27,7 +27,7 @@ DEFINE_bool(redis_keep_alive, true, "Redis 长连接"); DEFINE_int32(redis_pool_size, 16, "Redis 连接池大小"); DEFINE_string(mq_user, "root", "MQ 用户"); -DEFINE_string(mq_host, "127.0.0.1:5672", "MQ 地址"); +DEFINE_string(mq_host, "127.0.0.1", "RabbitMQ hostname (port is fixed at 5672)"); DEFINE_string(mq_push_exchange, "chat_push_exchange", "推送交换机"); DEFINE_string(mq_push_queue, "msg_push_queue", "推送队列"); DEFINE_string(mq_push_binding_key, "push", "推送绑定键"); diff --git a/push/source/push_server.h b/push/source/push_server.h index 519d024..6fe9057 100644 --- a/push/source/push_server.h +++ b/push/source/push_server.h @@ -545,7 +545,14 @@ class PushServiceImpl : public PushService } _connections->insert(conn, uid, did, jti); - if (_online_route) _online_route->bind(uid, did, _instance_id); + { + const auto cache_key = key::local_route_cache_key(uid); + auto guard = _inflight_registry ? _inflight_registry->acquire(cache_key) + : InflightRegistry::Guard{}; + std::unique_lock lock(guard.mu ? *guard.mu : _dummy_mu_); + if (_online_route) _online_route->bind(uid, did, _instance_id); + if (_local_route_cache) _local_route_cache->invalidate(cache_key); + } // 写 Presence（Push 为写入端） _write_presence_online_(uid, did); @@ -596,7 +603,7 @@ class PushServiceImpl : public PushService try { std::string k = key::presence_device_key(uid, did); const auto effective_ttl = randomized_ttl(std::chrono::seconds(kPresenceTtlSec)); - auto pipe = _redis->pipeline(); + auto pipe = _redis->pipeline(k); pipe.hset(k, "state", "ONLINE"); pipe.hset(k, "last_active_at_ms", std::to_string( std::chrono::duration_cast( @@ -612,7 +619,7 @@ class PushServiceImpl : public PushService try { std::string k = key::presence_device_key(uid, did); const auto effective_ttl = randomized_ttl(std::chrono::seconds(kPresenceTtlSec)); - auto pipe = _redis->pipeline(); + auto pipe = _redis->pipeline(k); pipe.hset(k, "state", "OFFLINE"); pipe.hset(k, "last_active_at_ms", std::to_string( std::chrono::duration_cast( @@ -725,7 +732,7 @@ class PushServiceImpl : public PushService route.device_ids.push_back(did); route.device_to_instance[did] = inst; } - if (_local_route_cache) { + if (_local_route_cache && !route.device_ids.empty()) { _local_route_cache->set(cache_key, route, randomized_ttl(_route_l1_ttl)); } @@ -1135,7 +1142,7 @@ class PushServerBuilder const std::string &queue, const std::string &binding_key) { - std::string amqp_url = "amqp://" + user + ":" + password + "@" + host + ":5672/"; + std::string amqp_url = make_amqp_url(user, password, host); _mq_client = std::make_shared(amqp_url); _push_settings = { .exchange = exchange, diff --git a/relationship/CMakeLists.txt b/relationship/CMakeLists.txt index d72fcb6..f679fac 100644 --- a/relationship/CMakeLists.txt +++ b/relationship/CMakeLists.txt @@ -10,7 +10,8 @@ set(proto_files common/types.proto common/error.proto common/envelope.proto message/message_types.proto identity/identity_service.proto conversation/conversation_service.proto - relationship/relationship_service.proto) + relationship/relationship_service.proto + push/notify.proto push/push_service.proto) set(proto_srcs "") foreach(proto_file ${proto_files}) string(REPLACE ".proto" ".pb.cc" proto_cc ${proto_file}) @@ -54,11 +55,12 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lodb-mysql -lodb -lodb-boost - -lcpr -lelasticlient -ljsoncpp) + -lcpr -lelasticlient) include_directories(${CMAKE_CURRENT_BINARY_DIR}) include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../common) diff --git a/relationship/Dockerfile b/relationship/Dockerfile index 58f61ad..600d254 100644 --- a/relationship/Dockerfile +++ b/relationship/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/relationship_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/relationship_server -flagfile=/im/conf/relationship_server.conf diff --git a/relationship/source/relationship_server.h b/relationship/source/relationship_server.h index 22f64e8..74bf33b 100644 --- a/relationship/source/relationship_server.h +++ b/relationship/source/relationship_server.h @@ -8,6 +8,7 @@ #include #include "infra/etcd.hpp" #include "mq/channel.hpp" +#include "mq/business_notify.hpp" #include "infra/logger.hpp" #include "utils/utils.hpp" #include "dao/data_es.hpp" @@ -228,6 +229,16 @@ class RelationshipServiceImpl : public ::chatnow::relationship::RelationshipServ throw ServiceError(::chatnow::error::kSystemInternalError, "insert friend_apply failed"); rsp->set_notify_event_id(eid); + ::chatnow::push::NotifyMessage notification; + notification.set_notify_event_id(eid); + notification.set_notify_type(::chatnow::push::FRIEND_ADD_APPLY_NOTIFY); + notification.set_trace_id(auth.trace_id); + auto* applicant = notification.mutable_friend_add_apply()->mutable_user_info(); + applicant->set_user_id(uid); + UserInfoMap users; + if (fetch_users(cntl, req->request_id(), {uid}, users) && users.count(uid)) + applicant->CopyFrom(users.at(uid)); + notify_online_users(_mm_channels, cntl, req->request_id(), {pid}, notification); }); } @@ -366,6 +377,17 @@ class RelationshipServiceImpl : public ::chatnow::relationship::RelationshipServ if (ca.has_conversation()) { rsp->set_new_conversation_id(ca.conversation().conversation_id()); } + ::chatnow::push::NotifyMessage notification; + notification.set_notify_event_id(eid); + notification.set_notify_type(::chatnow::push::FRIEND_ADD_PROCESS_NOTIFY); + notification.set_trace_id(auth.trace_id); + auto* result = notification.mutable_friend_process_result(); + result->set_agree(true); + result->mutable_user_info()->set_user_id(peer_uid); + UserInfoMap users; + if (fetch_users(cntl, req->request_id(), {peer_uid}, users) && users.count(peer_uid)) + result->mutable_user_info()->CopyFrom(users.at(peer_uid)); + notify_online_users(_mm_channels, cntl, req->request_id(), {apply_uid}, notification); }); } @@ -460,6 +482,7 @@ class RelationshipServerBuilder _mm_channels = std::make_shared(); _mm_channels->declared(_identity_service_name); _mm_channels->declared(_conversation_service_name); + _mm_channels->declared(kBusinessPushService); auto put_cb = std::bind(&ServiceManager::onServiceOnline, _mm_channels.get(), std::placeholders::_1, std::placeholders::_2); diff --git a/scripts/collect_test_diagnostics.py b/scripts/collect_test_diagnostics.py new file mode 100644 index 0000000..7367cd5 --- /dev/null +++ b/scripts/collect_test_diagnostics.py @@ -0,0 +1,144 @@ +#!/usr/bin/env python3 +"""Read bounded, allowlisted evidence from a disposable Compose test stack.""" + +import argparse +import base64 +import json +import os +from pathlib import Path +import re +import subprocess +import time +from datetime import datetime, timezone + +SERVICES = ("gateway", "identity", "relationship", "conversation", "transmite", + "message", "media", "presence", "push") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", required=True) + parser.add_argument("--timeout-sec", type=float, default=20) + args = parser.parse_args() + if not 0 < args.timeout_sec <= 60: + parser.error("timeout must be greater than zero and at most 60 seconds") + deadline = time.monotonic() + args.timeout_sec + + def command(arguments): + remaining = deadline - time.monotonic() + if remaining <= 0: + return None, "deadline" + try: + result = subprocess.run(["docker", *arguments], capture_output=True, + timeout=min(2, remaining), check=False) + except subprocess.TimeoutExpired: + return None, "deadline" + except OSError: + return None, "command_unavailable" + if result.returncode: + return None, "command_failed" + return result.stdout.decode("utf-8", errors="replace"), None + + snapshot = {"time": datetime.now(timezone.utc).isoformat(), "services": {}, + "registrations": {}, "events": [], "errors": []} + for service in SERVICES: + name = service + "_server" + ids, error = command(["compose", "ps", "--all", "--quiet", name]) + state = {"available": False} + if not error and re.fullmatch(r"[a-f0-9]{64}\s*", ids or ""): + raw, error = command(["inspect", "--format", + '{"State":{{json .State}},"RestartCount":{{.RestartCount}}}', ids.strip()]) + if not error: + try: + record = json.loads(raw) + source = record["State"] + state = {"available": True, "restart_count": int(record["RestartCount"])} + for key in ("Running", "Paused", "Restarting", "OOMKilled"): + state[key] = source.get(key) is True + for key in ("Pid", "ExitCode"): + state[key] = int(source[key]) + for key in ("StartedAt", "FinishedAt"): + value = source.get(key, "") + if re.fullmatch(r"[0-9TZ:.+-]{1,40}", value): + state[key] = value + except (ValueError, KeyError, TypeError, AttributeError): + state = {"available": False} + error = "invalid_state" + elif not error: + error = "missing_or_multiple_containers" + if error: + state["error"] = error + snapshot["services"][service] = state + + raw, error = command(["compose", "exec", "-T", "etcd", "etcdctl", "get", + "/service/", "--prefix", "--write-out=json"]) + keys = set() + if not error: + try: + for entry in json.loads(raw).get("kvs", []): + keys.add(base64.b64decode(entry["key"], validate=True).decode("utf-8")) + except (ValueError, KeyError, TypeError, UnicodeError, AttributeError): + error = "invalid_registration" + for service in SERVICES[1:]: + snapshot["registrations"][service] = { + "available": error is None, + "registered": (f"/service/{service}_service/instance" in keys) if not error else None, + } + if error: + snapshot["errors"].append({"source": "etcd", "error": error}) + + for service in ("gateway", "message", "transmite", "identity"): + events = [] + raw, error = command(["compose", "exec", "-T", service + "_server", "tail", + "-c", "262144", "/im/logs/" + service + ".log"]) + if error: + snapshot["errors"].append({"source": service + "_log", "error": error}) + continue + for line in raw.splitlines(): + try: + record = json.loads(line) + except ValueError: + continue + if not isinstance(record, dict): + continue + message = record.get("msg", "") + if not isinstance(message, str): + continue + event = {"service": service} + if message.startswith("Gateway RPC"): + event["event"] = "gateway_rpc_failure" + code = re.search(r"err=\[([0-9]{1,5})\]", message) + if code: + event["rpc_code"] = int(code[1]) + elif message.startswith("Gateway forward"): + event["event"] = "gateway_no_backend" + elif message.startswith("rpc_failed code="): + event["event"] = "application_failure" + code = re.match(r"rpc_failed code=([0-9]{1,5})\b", message) + if code: + event["error_code"] = int(code[1]) + elif message.startswith("rpc_exception"): + event["event"] = "application_exception" + else: + continue + for source, target, pattern in (("ts", "time", r"[0-9TZ:.+-]{1,40}"), + ("trace_id", "trace_id", r"[a-f0-9]{32}")): + value = record.get(source, "") + if isinstance(value, str) and re.fullmatch(pattern, value): + event[target] = value + # Never retain msg, fields, identities, request bodies, environment, + # container configuration, registration values, or command stderr. + events.append(event) + # Reserve each service's last events so a noisy Identity error path + # cannot evict all Gateway transport evidence. + snapshot["events"].extend(events[-125:]) + output = Path(args.output) + output.parent.mkdir(parents=True, exist_ok=True) + flags = os.O_WRONLY | os.O_CREAT | os.O_TRUNC | getattr(os, "O_NOFOLLOW", 0) + with os.fdopen(os.open(output, flags, 0o600), "w") as stream: + json.dump(snapshot, stream, indent=2) + stream.write("\n") + + +if __name__ == "__main__": + main() diff --git a/scripts/converge_mysql_users.sh b/scripts/converge_mysql_users.sh new file mode 100755 index 0000000..d6c53ab --- /dev/null +++ b/scripts/converge_mysql_users.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash + +set -euo pipefail + +required=( + MYSQL_ROOT_PASSWORD + CHATNOW_IDENTITY_MYSQL_PASSWORD + CHATNOW_CONVERSATION_MYSQL_PASSWORD + CHATNOW_RELATIONSHIP_MYSQL_PASSWORD + CHATNOW_MESSAGE_MYSQL_PASSWORD + CHATNOW_MEDIA_MYSQL_PASSWORD +) + +for name in "${required[@]}"; do + if [[ -z "${!name:-}" ]]; then + echo "mysql-init: required credential is missing: ${name}" >&2 + exit 1 + fi +done + +mysql_root() { + MYSQL_PWD="${MYSQL_ROOT_PASSWORD}" mysql \ + --host=mysql --protocol=tcp --user=root --batch --skip-column-names "$@" +} + +secret_hex() { + printf '%s' "$1" | od -An -tx1 | tr -d ' \n' +} + +ensure_user() { + local user=$1 + local password_hex + password_hex=$(secret_hex "$2") + mysql_root mysql <&2 + exit 1 +fi + +mysql_root() { + MYSQL_PWD="${MYSQL_ROOT_PASSWORD}" mysql \ + --host=mysql --protocol=tcp --user=root --batch --skip-column-names "$@" +} + +mysql_root <<'SQL' +CREATE DATABASE IF NOT EXISTS `chatnow` CHARACTER SET utf8mb4; +CREATE TABLE IF NOT EXISTS `chatnow`.`schema_migrations` ( + `version` varchar(128) NOT NULL PRIMARY KEY, + `checksum` char(64) NOT NULL, + `applied_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP +) ENGINE=InnoDB; +SQL + +shopt -s nullglob +migrations=(/migrations/V*.sql) +if (( ${#migrations[@]} == 0 )); then + echo "mysql-init: no versioned SQL migrations found" >&2 + exit 1 +fi +mapfile -t migrations < <(printf '%s\n' "${migrations[@]}" | sort -V) + +for migration in "${migrations[@]}"; do + version=$(basename "$migration" .sql) + checksum=$(sha256sum "$migration" | awk '{print $1}') + applied_checksum=$(mysql_root --execute="SELECT checksum FROM chatnow.schema_migrations WHERE version='${version}'") + + if [[ -n "$applied_checksum" ]]; then + if [[ "$applied_checksum" != "$checksum" ]]; then + echo "mysql-init: checksum mismatch for applied migration ${version}; migration changed" >&2 + exit 1 + fi + continue + fi + + echo "mysql-init: applying ${version}" + mysql_root chatnow <"$migration" + mysql_root chatnow --execute="INSERT INTO schema_migrations(version, checksum) VALUES ('${version}', '${checksum}')" +done + +/init/converge_mysql_users.sh +echo "mysql-init: schema and application users are ready" diff --git a/scripts/init_rabbitmq.py b/scripts/init_rabbitmq.py new file mode 100644 index 0000000..6b6128e --- /dev/null +++ b/scripts/init_rabbitmq.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python3 + +import base64 +import json +import os +import time +import urllib.error +import urllib.parse +import urllib.request + + +def required(name: str) -> str: + value = os.environ.get(name, "") + if not value: + raise RuntimeError(f"required credential is missing: {name}") + return value + + +bootstrap_user = required("RABBITMQ_BOOTSTRAP_USER") +bootstrap_password = required("RABBITMQ_BOOTSTRAP_PASSWORD") +max_attempts = int(os.environ.get("RABBITMQ_INIT_MAX_ATTEMPTS", "60")) +authorization = base64.b64encode( + f"{bootstrap_user}:{bootstrap_password}".encode("utf-8") +).decode("ascii") + + +def management_request(method: str, path: str, payload: dict | None = None) -> None: + data = None if payload is None else json.dumps(payload).encode("utf-8") + request = urllib.request.Request( + f"http://rabbitmq:15672{path}", + data=data, + method=method, + headers={ + "Authorization": f"Basic {authorization}", + "Content-Type": "application/json", + }, + ) + try: + with urllib.request.urlopen(request, timeout=3) as response: + if response.status < 200 or response.status >= 300: + raise RuntimeError(f"management request returned status {response.status}") + except urllib.error.HTTPError as error: + raise RuntimeError( + f"management request returned status {error.code} for {path}" + ) from None + + +for attempt in range(1, max_attempts + 1): + try: + management_request("GET", "/api/overview") + break + except (OSError, RuntimeError, urllib.error.URLError): + if attempt == max_attempts: + raise RuntimeError("management API did not become ready") from None + time.sleep(1) + + +users = { + "chatnow_transmite": { + "password": required("CHATNOW_TRANSMITE_MQ_PASSWORD"), + "configure": r"^chat_msg_exchange$", + "write": r"^chat_msg_exchange$", + "read": r"^$", + }, + "chatnow_message": { + "password": required("CHATNOW_MESSAGE_MQ_PASSWORD"), + "configure": r"^(chat_msg_exchange|chat_push_exchange|es_index_exchange|msg_queue_db|msg_queue_es|msg_queue_es_index|msg_push_queue)$", + "write": r"^(chat_push_exchange|es_index_exchange|msg_queue_db|msg_queue_es|msg_queue_es_index|msg_push_queue)$", + "read": r"^(chat_msg_exchange|chat_push_exchange|es_index_exchange|msg_queue_db|msg_queue_es|msg_queue_es_index)$", + }, + "chatnow_push": { + "password": required("CHATNOW_PUSH_MQ_PASSWORD"), + "configure": r"^(chat_push_exchange|msg_push_queue)$", + "write": r"^msg_push_queue$", + "read": r"^(chat_push_exchange|msg_push_queue)$", + }, +} + +for username, settings in users.items(): + encoded_user = urllib.parse.quote(username, safe="") + management_request( + "PUT", + f"/api/users/{encoded_user}", + {"password": settings["password"], "tags": ""}, + ) + management_request( + "PUT", + f"/api/permissions/%2F/{encoded_user}", + { + "configure": settings["configure"], + "write": settings["write"], + "read": settings["read"], + }, + ) + management_request("GET", f"/api/permissions/%2F/{encoded_user}") + +print("rabbitmq-init: application users and permissions are ready") diff --git a/scripts/init_redis_cluster.sh b/scripts/init_redis_cluster.sh new file mode 100755 index 0000000..307f3c9 --- /dev/null +++ b/scripts/init_redis_cluster.sh @@ -0,0 +1,56 @@ +#!/bin/sh + +set -eu + +MAX_ATTEMPTS=${REDIS_INIT_MAX_ATTEMPTS:-60} +NODES="redis-node1:6379 redis-node2:6380 redis-node3:6381 redis-node4:6382 redis-node5:6383 redis-node6:6384" + +attempt=1 +while [ "$attempt" -le "$MAX_ATTEMPTS" ]; do + all_ready=true + for node in $NODES; do + host=${node%:*} + port=${node#*:} + if ! redis-cli -h "$host" -p "$port" ping 2>/dev/null | grep -q '^PONG$'; then + all_ready=false + break + fi + done + if [ "$all_ready" = true ]; then + break + fi + attempt=$((attempt + 1)) + sleep 1 +done + +if [ "$attempt" -gt "$MAX_ATTEMPTS" ]; then + echo "redis-init: nodes did not become ready within ${MAX_ATTEMPTS} attempts" >&2 + exit 1 +fi + +cluster_info=$(redis-cli -h redis-node1 -p 6379 cluster info 2>/dev/null || true) +if ! printf '%s\n' "$cluster_info" | grep -q '^cluster_state:ok'; then + known_nodes=$(printf '%s\n' "$cluster_info" | sed -n 's/^cluster_known_nodes:\([0-9][0-9]*\).*/\1/p') + if [ -n "$known_nodes" ] && [ "$known_nodes" -gt 1 ]; then + echo "redis-init: existing cluster metadata is degraded; refusing to recreate it" >&2 + exit 1 + fi + + redis-cli --cluster create $NODES --cluster-replicas 1 --cluster-yes +fi + +attempt=1 +while [ "$attempt" -le "$MAX_ATTEMPTS" ]; do + cluster_info=$(redis-cli -h redis-node1 -p 6379 cluster info 2>/dev/null || true) + if printf '%s\n' "$cluster_info" | grep -q '^cluster_state:ok' && + printf '%s\n' "$cluster_info" | grep -q '^cluster_slots_assigned:16384'; then + redis-cli --cluster check redis-node1:6379 >/dev/null + echo "redis-init: cluster is ready with all 16384 slots assigned" + exit 0 + fi + attempt=$((attempt + 1)) + sleep 1 +done + +echo "redis-init: cluster did not converge within ${MAX_ATTEMPTS} attempts" >&2 +exit 1 diff --git a/scripts/wait_for_services.sh b/scripts/wait_for_services.sh new file mode 100755 index 0000000..97b3f5c --- /dev/null +++ b/scripts/wait_for_services.sh @@ -0,0 +1,119 @@ +#!/usr/bin/env bash + +set -euo pipefail + +READY_TIMEOUT_SEC=${CHATNOW_READY_TIMEOUT_SEC:-180} +POLL_INTERVAL_SEC=${CHATNOW_READY_POLL_INTERVAL_SEC:-2} +repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +cd "$repo_root" + +deadline=$((SECONDS + READY_TIMEOUT_SEC)) + +compose() { + local remaining=$((deadline - SECONDS)) + (( remaining > 0 )) || return 1 + # A stuck Docker request must not prevent the outer deadline from advancing. + timeout --kill-after=1 "${remaining}s" docker compose "$@" +} + +wait_until() { + local label=$1 + local probe=$2 + while (( SECONDS < deadline )); do + if "$probe"; then + echo "readiness: ${label} ready" + return 0 + fi + sleep "$POLL_INTERVAL_SEC" + done + echo "readiness: timed out waiting for ${label}" >&2 + return 1 +} + +probe_redis_cluster() { + local info + info=$(compose exec -T redis-node1 redis-cli -p 6379 cluster info 2>/dev/null) || return 1 + grep -q '^cluster_state:ok' <<<"$info" && + grep -q '^cluster_slots_assigned:16384' <<<"$info" && + grep -q '^cluster_slots_ok:16384' <<<"$info" && + grep -q '^cluster_known_nodes:6' <<<"$info" +} + +probe_mysql_schema() { + local count users + count=$(compose exec -T mysql sh -ec ' + MYSQL_PWD="$MYSQL_ROOT_PASSWORD" mysql --user=root --batch --skip-column-names \ + --execute="SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = '\''chatnow'\'' AND table_name IN ('\''conversation'\'', '\''conversation_member'\'', '\''friend_apply'\'', '\''media_blob_ref'\'', '\''media_file'\'', '\''media_user_quota'\'', '\''message'\'', '\''message_attachment'\'', '\''message_mention'\'', '\''message_pin'\'', '\''message_reaction'\'', '\''message_read'\'', '\''relation'\'', '\''user'\'', '\''user_block'\'', '\''user_device'\'', '\''user_timeline'\'');" chatnow + ' 2>/dev/null) || return 1 + [[ "$count" == "17" ]] || return 1 + + users=$(compose exec -T mysql sh -ec ' + MYSQL_PWD="$MYSQL_ROOT_PASSWORD" mysql --user=root --batch --skip-column-names \ + --execute="SELECT COUNT(*) FROM mysql.user WHERE user IN ('\''chatnow_identity'\'', '\''chatnow_conversation'\'', '\''chatnow_relationship'\'', '\''chatnow_message'\'', '\''chatnow_media'\'');" + ' 2>/dev/null) || return 1 + [[ "$users" == "5" ]] +} + +probe_rabbitmq() { + compose exec -T rabbitmq rabbitmq-diagnostics -q check_running >/dev/null 2>&1 && + compose exec -T rabbitmq rabbitmq-diagnostics -q check_local_alarms >/dev/null 2>&1 || return 1 + + local users user + users=$(compose exec -T rabbitmq rabbitmqctl -q list_users 2>/dev/null) || return 1 + for user in chatnow_transmite chatnow_message chatnow_push; do + grep -Eq "^${user}[[:space:]]" <<<"$users" || return 1 + compose exec -T rabbitmq rabbitmqctl -q list_user_permissions "$user" 2>/dev/null | + grep -Eq '^/[[:space:]]' || return 1 + done +} + +probe_elasticsearch() { + local health + health=$(curl --fail --silent --show-error --max-time 2 \ + 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=1s') || return 1 + grep -Eq '"status":"(yellow|green)"' <<<"$health" +} + +probe_minio() { + curl --fail --silent --show-error --max-time 2 \ + 'http://127.0.0.1:19000/minio/health/ready' >/dev/null || return 1 + compose run --rm --no-deps --entrypoint /bin/sh minio-init -ec ' + printf "%s\n%s\n" "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" | + mc alias set ready http://minio:9000 --api S3v4 --path on >/dev/null + mc stat ready/chatnow-media-public >/dev/null + mc stat ready/chatnow-media-private >/dev/null + ' >/dev/null 2>&1 +} + +probe_etcd_registrations() { + local keys service count + keys=$(compose exec -T -e ETCDCTL_API=3 etcd \ + etcdctl --endpoints=http://127.0.0.1:2379 get /service --prefix --keys-only \ + 2>/dev/null) || return 1 + for service in identity media transmite message relationship conversation presence push; do + count=$(grep -c "^/service/${service}_service/instance$" <<<"$keys" || true) + [[ "$count" == "1" ]] || return 1 + done + count=$(sed -n 's#^/service/\([^/]*_service\)/instance$#\1#p' <<<"$keys" | sort -u | wc -l) + [[ "${count//[[:space:]]/}" == "8" ]] +} + +probe_gateway() { + curl --fail --silent --show-error --max-time 2 \ + 'http://127.0.0.1:9000/health' >/dev/null +} + +probe_push() { + nc -z -w 2 127.0.0.1 9001 +} + +wait_until "Redis Cluster" probe_redis_cluster +wait_until "MySQL schema and application users" probe_mysql_schema +wait_until "RabbitMQ" probe_rabbitmq +wait_until "Elasticsearch" probe_elasticsearch +wait_until "MinIO buckets" probe_minio +wait_until "eight exact etcd service registrations" probe_etcd_registrations +wait_until "Gateway HTTP health" probe_gateway +wait_until "Push WebSocket port 9001" probe_push + +echo "readiness: full stack is ready" diff --git a/sql/V1__core.sql b/sql/V1__core.sql new file mode 100644 index 0000000..ca60c3d --- /dev/null +++ b/sql/V1__core.sql @@ -0,0 +1,325 @@ +/* This file was generated by ODB, object-relational mapping (ORM) + * compiler for C++. + */ + +CREATE TABLE IF NOT EXISTS `conversation` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `conversation_id` varchar(48) NOT NULL, + `conversation_name` varchar(64) NULL, + `conversation_type` tinyint unsigned NOT NULL, + `create_time` DATETIME(3) NULL, + `member_count` int unsigned NOT NULL, + `status` tinyint unsigned NOT NULL, + `avatar_id` varchar(64) NULL, + `owner_id` varchar(32) NULL, + `peer_user_id` varchar(32) NULL, + `description` varchar(255) NULL, + `announcement` varchar(1024) NULL, + `muted_all` tinyint(1) NOT NULL, + `max_seq` bigint unsigned NOT NULL, + `update_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `conversation_id_i` + ON `conversation` (`conversation_id`); + +CREATE TABLE IF NOT EXISTS `conversation_member` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `conversation_id` varchar(48) NOT NULL, + `user_id` varchar(32) NOT NULL, + `last_read_seq` bigint unsigned NOT NULL, + `last_ack_seq` bigint unsigned NOT NULL, + `muted` tinyint(1) NOT NULL, + `visible` tinyint(1) NOT NULL, + `pin_time` DATETIME(3) NULL, + `role` tinyint unsigned NOT NULL, + `alias` varchar(64) NULL, + `inviter_id` varchar(32) NULL, + `join_source` tinyint unsigned NOT NULL, + `join_time` DATETIME(3) NULL, + `mute_until` DATETIME(3) NULL, + `is_quit` tinyint(1) NOT NULL, + `quit_time` DATETIME(3) NULL, + `draft` text NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_conv_user` + ON `conversation_member` ( + `conversation_id`, + `user_id`); + +CREATE INDEX `idx_user_conv` + ON `conversation_member` ( + `user_id`, + `is_quit`, + `conversation_id`); + +CREATE TABLE IF NOT EXISTS `friend_apply` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `event_id` varchar(32) NOT NULL, + `user_id` varchar(32) NOT NULL, + `peer_id` varchar(32) NOT NULL, + `status` tinyint unsigned NOT NULL, + `source` tinyint unsigned NOT NULL, + `greeting` varchar(255) NULL, + `remark` varchar(64) NULL, + `create_time` DATETIME(3) NULL, + `handle_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `event_id_i` + ON `friend_apply` (`event_id`); + +CREATE INDEX `idx_peer_status_time` + ON `friend_apply` ( + `peer_id`, + `status`, + `create_time`); + +CREATE INDEX `idx_user_peer_time` + ON `friend_apply` ( + `user_id`, + `peer_id`, + `create_time`); + +CREATE TABLE IF NOT EXISTS `message` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `message_id` bigint unsigned NOT NULL, + `seq_id` bigint unsigned NOT NULL, + `session_id` varchar(48) NOT NULL, + `user_id` varchar(32) NOT NULL, + `message_type` tinyint unsigned NOT NULL, + `create_time` DATETIME(3) NULL, + `content` text NULL, + `client_msg_id` varchar(64) NULL, + `file_id` varchar(64) NULL, + `file_name` varchar(255) NULL, + `file_size` bigint unsigned NULL, + `reply_to_msg_id` bigint unsigned NULL, + `status` tinyint unsigned NOT NULL, + `revoke_time` DATETIME(3) NULL, + `revoke_by` varchar(32) NULL, + `edit_time` DATETIME(3) NULL, + `forward_from_uid` varchar(32) NULL, + `forward_at` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `message_id_i` + ON `message` (`message_id`); + +CREATE UNIQUE INDEX `uk_session_seq` + ON `message` ( + `session_id`, + `seq_id`); + +CREATE UNIQUE INDEX `uk_client_msg` + ON `message` ( + `user_id`, + `client_msg_id`); + +CREATE TABLE IF NOT EXISTS `message_attachment` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `message_id` bigint unsigned NOT NULL, + `order_idx` int NOT NULL, + `att_type` tinyint unsigned NOT NULL, + `file_id` varchar(64) NOT NULL, + `file_name` varchar(255) NULL, + `file_size` bigint unsigned NOT NULL, + `mime_type` varchar(64) NULL, + `thumb_file_id` varchar(64) NULL, + `width` int unsigned NULL, + `height` int unsigned NULL, + `duration_ms` int unsigned NULL, + `asr_text` text NULL, + `create_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE INDEX `idx_message` + ON `message_attachment` ( + `message_id`, + `order_idx`); + +CREATE TABLE IF NOT EXISTS `message_mention` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `message_id` bigint unsigned NOT NULL, + `user_id` varchar(32) NOT NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_msg_user` + ON `message_mention` ( + `message_id`, + `user_id`); + +CREATE INDEX `idx_user_msg` + ON `message_mention` ( + `user_id`, + `message_id`); + +CREATE TABLE IF NOT EXISTS `message_pin` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `session_id` varchar(48) NOT NULL, + `message_id` bigint unsigned NOT NULL, + `pinned_by` varchar(32) NOT NULL, + `pinned_at` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_conv_msg` + ON `message_pin` ( + `session_id`, + `message_id`); + +CREATE INDEX `idx_conv` + ON `message_pin` (`session_id`); + +CREATE TABLE IF NOT EXISTS `message_reaction` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `message_id` bigint unsigned NOT NULL, + `user_id` varchar(32) NOT NULL, + `emoji` varchar(16) NOT NULL, + `create_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_msg_user_emoji` + ON `message_reaction` ( + `message_id`, + `user_id`, + `emoji`); + +CREATE INDEX `idx_msg` + ON `message_reaction` (`message_id`); + +CREATE TABLE IF NOT EXISTS `message_read` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `message_id` bigint unsigned NOT NULL, + `user_id` varchar(32) NOT NULL, + `read_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_msg_user` + ON `message_read` ( + `message_id`, + `user_id`); + +CREATE INDEX `idx_msg_time` + ON `message_read` ( + `message_id`, + `read_time`); + +CREATE TABLE IF NOT EXISTS `relation` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `user_id` varchar(32) NOT NULL, + `peer_id` varchar(32) NOT NULL, + `remark` varchar(64) NULL, + `group_name` varchar(32) NULL, + `status` tinyint unsigned NOT NULL, + `starred` tinyint(1) NOT NULL, + `create_time` DATETIME(3) NULL, + `update_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_user_peer` + ON `relation` ( + `user_id`, + `peer_id`); + +CREATE INDEX `idx_peer_user` + ON `relation` ( + `peer_id`, + `user_id`); + +CREATE TABLE IF NOT EXISTS `user` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `user_id` varchar(32) NOT NULL, + `nickname` varchar(64) NULL, + `description` varchar(255) NULL, + `password` varchar(256) NULL, + `password_salt` varchar(64) NULL, + `mail` varchar(128) NULL, + `phone` varchar(20) NULL, + `avatar_id` varchar(64) NULL, + `gender` tinyint unsigned NOT NULL, + `region` varchar(64) NULL, + `status` tinyint unsigned NOT NULL, + `register_time` DATETIME(3) NULL, + `last_login_time` DATETIME(3) NULL, + `last_login_ip` varchar(45) NULL, + `update_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `user_id_i` + ON `user` (`user_id`); + +CREATE INDEX `nickname_i` + ON `user` (`nickname`); + +CREATE UNIQUE INDEX `mail_i` + ON `user` (`mail`); + +CREATE UNIQUE INDEX `phone_i` + ON `user` (`phone`); + +CREATE TABLE IF NOT EXISTS `user_block` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `blocker_id` varchar(32) NOT NULL, + `blocked_id` varchar(32) NOT NULL, + `create_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `uk_blocker_blocked` + ON `user_block` ( + `blocker_id`, + `blocked_id`); + +CREATE INDEX `idx_blocked` + ON `user_block` (`blocked_id`); + +CREATE TABLE IF NOT EXISTS `user_device` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `user_id` varchar(32) NOT NULL, + `device_id` varchar(128) NOT NULL, + `device_type` tinyint unsigned NOT NULL, + `device_name` varchar(64) NULL, + `app_version` varchar(32) NULL, + `os_version` varchar(32) NULL, + `login_session_id` varchar(64) NOT NULL, + `push_token` varchar(255) NULL, + `login_ip` varchar(45) NULL, + `online_status` tinyint unsigned NOT NULL, + `last_active_time` DATETIME(3) NULL, + `login_time` DATETIME(3) NULL, + `expire_time` DATETIME(3) NULL, + `update_time` DATETIME(3) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `login_session_id_i` + ON `user_device` (`login_session_id`); + +CREATE UNIQUE INDEX `uk_user_device` + ON `user_device` ( + `user_id`, + `device_id`); + +CREATE UNIQUE INDEX `uk_push_token` + ON `user_device` (`push_token`); + +CREATE TABLE IF NOT EXISTS `user_timeline` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `user_id` varchar(32) NOT NULL, + `user_seq` bigint unsigned NOT NULL, + `session_id` varchar(48) NOT NULL, + `session_seq` bigint unsigned NOT NULL, + `message_id` bigint unsigned NOT NULL, + `message_time` DATETIME(3) NULL, + `deliver_status` tinyint unsigned NOT NULL) + ENGINE=InnoDB; + +CREATE INDEX `idx_user_seq` + ON `user_timeline` ( + `user_id`, + `user_seq`); + +CREATE INDEX `idx_user_session_seq` + ON `user_timeline` ( + `user_id`, + `session_id`, + `session_seq`); diff --git a/sql/V4__media.sql b/sql/V4__media.sql index eddd1ef..3c66754 100644 --- a/sql/V4__media.sql +++ b/sql/V4__media.sql @@ -1,56 +1,58 @@ --- =========================================================================== --- V4__media.sql —— 媒体子系统三张表 DDL --- --------------------------------------------------------------------------- --- 注意：本仓的权威 schema 由 ODB 通过 odb/media_*.hxx 的 --- `--generate-schema` 自动生成（见 file/CMakeLists.txt）。 --- 本文件仅作部署参考与 code review 友好性；与 ODB 输出冲突时以 ODB 为准。 --- --------------------------------------------------------------------------- --- 表关系： --- media_file 元数据 + 状态机（pending / committed / deleted / quarantined） --- media_blob_ref 按 content_hash 去重的物理对象引用计数 --- media_user_quota 用户级配额（默认 5 GB） --- =========================================================================== - --- 媒体文件元数据 -CREATE TABLE IF NOT EXISTS media_file ( - id BIGINT NOT NULL AUTO_INCREMENT, - file_id VARCHAR(20) NOT NULL, -- 业务唯一 ID（snowflake Next() → 16 hex） - content_hash VARCHAR(72) NOT NULL, -- "sha256:<64hex>" - bucket VARCHAR(64) NOT NULL, - object_key VARCHAR(255) NOT NULL, - file_name VARCHAR(255) NOT NULL DEFAULT '', - file_size BIGINT NOT NULL, - mime_type VARCHAR(128) NOT NULL, - purpose TINYINT UNSIGNED NOT NULL, -- MediaPurpose enum - owner_id VARCHAR(32) NOT NULL, -- user_id - uploaded_at DATETIME(3) NOT NULL, - status TINYINT UNSIGNED NOT NULL, -- 0 pending / 1 committed / 2 deleted / 3 quarantined - upload_id VARCHAR(128) NULL, -- multipart upload_id (S3) - PRIMARY KEY (id), - UNIQUE KEY uq_file_id (file_id), - KEY idx_hash (content_hash), - KEY idx_owner_uploaded (owner_id, uploaded_at), - KEY idx_status_uploaded (status, uploaded_at), - KEY idx_upload_id (upload_id) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; - --- 物理 blob 的引用计数（按 content_hash 去重） -CREATE TABLE IF NOT EXISTS media_blob_ref ( - content_hash VARCHAR(72) NOT NULL, - bucket VARCHAR(64) NOT NULL, - object_key VARCHAR(255) NOT NULL, - ref_count INT NOT NULL DEFAULT 0, - total_size BIGINT NOT NULL, - last_decremented_at DATETIME(3) NOT NULL, - PRIMARY KEY (content_hash), - KEY idx_refcount_decremented (ref_count, last_decremented_at) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; - --- 用户级配额 -CREATE TABLE IF NOT EXISTS media_user_quota ( - user_id VARCHAR(32) NOT NULL, - used_bytes BIGINT NOT NULL DEFAULT 0, - quota_bytes BIGINT NOT NULL DEFAULT 5368709120, -- 5 GB - updated_at DATETIME(3) NOT NULL, - PRIMARY KEY (user_id) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; +/* This file was generated by ODB, object-relational mapping (ORM) + * compiler for C++. + */ + +CREATE TABLE IF NOT EXISTS `media_blob_ref` ( + `content_hash` varchar(72) NOT NULL PRIMARY KEY, + `bucket` varchar(64) NOT NULL, + `object_key` varchar(255) NOT NULL, + `ref_count` int NOT NULL, + `total_size` bigint NOT NULL, + `last_decremented_at` DATETIME(3) NULL) + ENGINE=InnoDB; +CREATE INDEX `idx_refcount_decremented` + ON `media_blob_ref` ( + `ref_count`, + `last_decremented_at`); + +CREATE TABLE IF NOT EXISTS `media_file` ( + `id` BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT, + `file_id` varchar(20) NOT NULL, + `content_hash` varchar(72) NOT NULL, + `bucket` varchar(64) NOT NULL, + `object_key` varchar(255) NOT NULL, + `file_name` varchar(255) NOT NULL, + `file_size` bigint NOT NULL, + `mime_type` varchar(128) NOT NULL, + `purpose` tinyint unsigned NOT NULL, + `owner_id` varchar(32) NOT NULL, + `uploaded_at` DATETIME(3) NULL, + `status` tinyint unsigned NOT NULL, + `upload_id` varchar(128) NULL) + ENGINE=InnoDB; + +CREATE UNIQUE INDEX `file_id_i` + ON `media_file` (`file_id`); + +CREATE INDEX `idx_hash` + ON `media_file` (`content_hash`); + +CREATE INDEX `idx_owner_uploaded` + ON `media_file` ( + `owner_id`, + `uploaded_at`); + +CREATE INDEX `idx_status_uploaded` + ON `media_file` ( + `status`, + `uploaded_at`); + +CREATE INDEX `idx_upload_id` + ON `media_file` (`upload_id`); + +CREATE TABLE IF NOT EXISTS `media_user_quota` ( + `user_id` varchar(32) NOT NULL PRIMARY KEY, + `used_bytes` bigint NOT NULL, + `quota_bytes` bigint NOT NULL, + `updated_at` DATETIME(3) NULL) + ENGINE=InnoDB; diff --git a/tests/Makefile b/tests/Makefile index 619f47a..b4c4640 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -14,19 +14,19 @@ PF09_EXPECTED_TRANSMITE_INSTANCES ?= 1 # push_service.proto defines PushToUserReq/PushBatchReq etc. Both are needed. proto: @echo "Generating protobuf..." - PROTO_BASE=../proto OUT_BASE=./proto; \ + set -e; PROTO_BASE=../proto OUT_BASE=./proto; \ GO_OPTS="module=chatnow-tests/proto"; \ - for p in $$PROTO_BASE/*/*.proto; do \ + for p in $$PROTO_BASE/*/*.proto $$PROTO_BASE/common/auth/*.proto; do \ [ -f "$$p" ] || continue; \ rel=$${p#$$PROTO_BASE/}; \ dir=$$(dirname $$rel); \ GO_OPTS="$$GO_OPTS,M$$rel=chatnow-tests/proto/chatnow/$$dir"; \ done; \ - for dir in common identity relationship conversation message transmite media presence push; do \ + for dir in common common/auth identity relationship conversation message transmite media presence push; do \ mkdir -p "$$OUT_BASE/chatnow/$$dir"; \ for f in "$$PROTO_BASE/$$dir"/*.proto; do \ [ -f "$$f" ] || continue; \ - protoc --proto_path="$$PROTO_BASE" --go_out="$$OUT_BASE" --go_opt=$$GO_OPTS "$$f"; \ + protoc --experimental_allow_proto3_optional --proto_path="$$PROTO_BASE" --go_out="$$OUT_BASE" --go_opt=$$GO_OPTS "$$f"; \ done; \ done diff --git a/tests/bvt/conversation_test.go b/tests/bvt/conversation_test.go index 61630c2..77429cf 100644 --- a/tests/bvt/conversation_test.go +++ b/tests/bvt/conversation_test.go @@ -43,7 +43,7 @@ func TestBVT_AddMembers_Success(t *testing.T) { m3, _, _ := fixture.RegisterAndLogin(t, HTTP) fixture.AddMembers(t, owner, convID, []string{m3.UserID}) - // 验证成员数 = 3（owner + 2 初始 + 1 新增） + // Verify the owner, both original members, and the newly added member. listReq := &conversation.ListMembersReq{ RequestId: client.NewRequestID(), ConversationId: convID, @@ -52,9 +52,11 @@ func TestBVT_AddMembers_Success(t *testing.T) { err := owner.DoAuth("/service/conversation/list_members", listReq, listRsp) require.NoError(t, err) assert.True(t, listRsp.Header.Success) - assert.Len(t, listRsp.Members, 3) - - _ = members + actual := make([]string, 0, len(listRsp.Members)) + for _, member := range listRsp.Members { + actual = append(actual, member.GetUserInfo().GetUserId()) + } + assert.ElementsMatch(t, []string{owner.UserID, members[0].UserID, members[1].UserID, m3.UserID}, actual) } // BVT-014 | P0 | 会话链路 | 列出会话，包含刚建的群 diff --git a/tests/bvt/media_test.go b/tests/bvt/media_test.go index e15b72c..a38f080 100644 --- a/tests/bvt/media_test.go +++ b/tests/bvt/media_test.go @@ -5,6 +5,7 @@ package bvt_test import ( "bytes" "crypto/sha256" + "encoding/xml" "fmt" "io" "net/http" @@ -78,7 +79,15 @@ func TestBVT_CompleteUpload_Success(t *testing.T) { } putResp, err := http.DefaultClient.Do(httpReq) require.NoError(t, err) - require.Equal(t, 200, putResp.StatusCode, "PUT 到 MinIO 失败") + if putResp.StatusCode != http.StatusOK { + var detail struct { + Code string `xml:"Code"` + Message string `xml:"Message"` + } + _ = xml.NewDecoder(io.LimitReader(putResp.Body, 4096)).Decode(&detail) + putResp.Body.Close() + t.Fatalf("S3 PUT failed: status=%d code=%s message=%s", putResp.StatusCode, detail.Code, detail.Message) + } putResp.Body.Close() // Step 3: CompleteUpload diff --git a/tests/bvt/presence_test.go b/tests/bvt/presence_test.go index 2d9baa3..a4f4d16 100644 --- a/tests/bvt/presence_test.go +++ b/tests/bvt/presence_test.go @@ -16,8 +16,10 @@ import ( // BVT-018 | P0 | presence 链路 | 查询在线状态，返回 online func TestBVT_GetPresence_Success(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) + ws := fixture.ConnectWS(t, authed) + t.Cleanup(func() { ws.Close() }) - // 登录后 presence 应为 ONLINE + // Presence is ONLINE after an authenticated WebSocket becomes active. req := &presence.GetPresenceReq{ RequestId: client.NewRequestID(), UserId: authed.UserID, diff --git a/tests/bvt/setup_test.go b/tests/bvt/setup_test.go index e69c9e6..3bc25f2 100644 --- a/tests/bvt/setup_test.go +++ b/tests/bvt/setup_test.go @@ -14,7 +14,7 @@ var HTTP *client.HTTPClient var Cfg *client.Config func TestMain(m *testing.M) { - Cfg = client.LoadConfig("") + Cfg = client.LoadConfig("../config.yaml") HTTP = client.NewHTTPClient(Cfg) if err := cleanup.WaitForStackReady(Cfg, 120*1e9); err != nil { panic(err) diff --git a/tests/compose/reliability.yml b/tests/compose/reliability.yml new file mode 100644 index 0000000..0da7e3c --- /dev/null +++ b/tests/compose/reliability.yml @@ -0,0 +1,7 @@ +# Use only with an isolated disposable Reliability stack. Explicit IPAM keeps +# endpoint move/restore operations supported across Docker Engine versions. +networks: + default: + ipam: + config: + - subnet: ${CHATNOW_RELIABILITY_SUBNET:-172.30.97.0/24} diff --git a/tests/func/auth_middleware_test.go b/tests/func/auth_middleware_test.go index 0a14107..8291163 100644 --- a/tests/func/auth_middleware_test.go +++ b/tests/func/auth_middleware_test.go @@ -69,7 +69,7 @@ func TestJWTRequired_GetProfile_NoToken(t *testing.T) { require.Error(t, err) } -func TestJWTRequired_ExpiredToken(t *testing.T) { +func TestJWTRequired_InvalidToken(t *testing.T) { expiredClient := client.NewHTTPClient(HTTP.Config()) expiredClient.AccessToken = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U" req := &identity.GetProfileReq{RequestId: client.NewRequestID()} diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go index e5195c9..1631441 100644 --- a/tests/func/cache_test.go +++ b/tests/func/cache_test.go @@ -416,30 +416,55 @@ func TestFN_CA_UnackedSameUserSeqLatestPayloadAndAck(t *testing.T) { // FN-CA-05 | healthy Redis applies the distributed message rate limit. func TestFN_CA_RateLimit(t *testing.T) { - user, peer, convID := fixture.MakeFriends(t, HTTP) - _ = peer - - rateLimited := 0 - for i := 0; i < 650; i++ { - rsp := &transmite.SendMessageRsp{} - err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ - RequestId: client.NewRequestID(), - ConversationId: convID, - Content: &msg.MessageContent{ - Type: msg.MessageType_TEXT, - Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: fmt.Sprintf("rate-limit-%d", i)}}, - }, - ClientMsgId: client.NewRequestID(), - }, rsp) + user, _, convID := fixture.MakeFriends(t, HTTP) + // Start with an exhausted, user-owned bucket so this correctness test does + // not depend on load-generator throughput exceeding the refill rate. + key := "im:rl:user:" + user.UserID + verify.RedisCLI(t, "HSET", key, "tokens", "0", "ts", strconv.FormatInt(time.Now().UnixMilli(), 10)) + verify.RedisCLI(t, "PEXPIRE", key, "120000") + t.Cleanup(func() { verify.RedisCLI(t, "DEL", key) }) + const workers, requests = 8, 24 + jobs := make(chan int, requests) + results := make(chan *transmite.SendMessageRsp, requests) + errors := make(chan error, requests) + var wg sync.WaitGroup + for i := 0; i < workers; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for index := range jobs { + rsp := &transmite.SendMessageRsp{} + err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ + RequestId: client.NewRequestID(), ConversationId: convID, + Content: &msg.MessageContent{Type: msg.MessageType_TEXT, + Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: fmt.Sprintf("rate-limit-%d", index)}}}, + ClientMsgId: client.NewRequestID(), + }, rsp) + errors <- err + results <- rsp + } + }() + } + for i := 0; i < requests; i++ { + jobs <- i + } + close(jobs) + wg.Wait() + close(errors) + close(results) + for err := range errors { require.NoError(t, err) + } + rateLimited := 0 + for rsp := range results { if rsp.GetHeader().GetErrorMessage() == "rate_limited" { rateLimited++ + } else { + require.True(t, rsp.GetHeader().GetSuccess(), rsp.GetHeader().GetErrorMessage()) } } - require.Greater(t, rateLimited, 0, "healthy Redis must enforce the distributed rate limit") } - func TestFN_CA_UserInfoL2AvoidsRepeatedRPC(t *testing.T) { user, _, convID := fixture.MakeFriends(t, HTTP) verify.RedisCLI(t, "DEL", userInfoRedisKey(user.UserID)) diff --git a/tests/func/concurrency_test.go b/tests/func/concurrency_test.go index 6e84061..fef5444 100644 --- a/tests/func/concurrency_test.go +++ b/tests/func/concurrency_test.go @@ -7,6 +7,7 @@ import ( "fmt" "sync" "testing" + "time" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -14,6 +15,7 @@ import ( "chatnow-tests/pkg/client" "chatnow-tests/pkg/fixture" "chatnow-tests/pkg/verify" + conversation "chatnow-tests/proto/chatnow/conversation" media "chatnow-tests/proto/chatnow/media" msg "chatnow-tests/proto/chatnow/message" relationship "chatnow-tests/proto/chatnow/relationship" @@ -76,7 +78,7 @@ func TestFN_CC_SendMessage_SameClientMsgId(t *testing.T) { // FN-CC-02 | P1 | concurrency | 10 goroutine 并发发消息，全部落库，seq 不重复 func TestFN_CC_SendMessage_DifferentMsgId(t *testing.T) { - a, _, convID := setupConv(t) + a, recipient, convID := setupConv(t) var wg sync.WaitGroup msgIDs := make([]int64, 10) @@ -142,7 +144,23 @@ func TestFN_CC_SendMessage_DifferentMsgId(t *testing.T) { // 直查 DB：message 表有 10 条 dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) defer dbV.Close() + for _, id := range msgIDs { + dbV.WaitMessageExists(t, id, 10*time.Second) + } dbV.MessageCount(t, convID, 10) + // Every persisted concurrent message contributes to the conversation watermark. + list := &conversation.ListConversationsRsp{} + require.NoError(t, recipient.DoAuth("/service/conversation/list", + &conversation.ListConversationsReq{RequestId: client.NewRequestID()}, list)) + require.True(t, list.GetHeader().GetSuccess()) + found := false + for _, item := range list.Conversations { + if item.ConversationId == convID { + found = true + require.Equal(t, uint64(10), item.GetSelf().GetUnreadCount()) + } + } + require.True(t, found) } // FN-CC-03 | P1 | concurrency | 好友通过瞬间并发发消息，不丢 @@ -171,6 +189,7 @@ func TestFN_CC_FriendAccept_ThenSend(t *testing.T) { // 并发发 5 条消息 var wg sync.WaitGroup errs := make([]error, 5) + messageIDs := make([]int64, 5) for i := 0; i < 5; i++ { wg.Add(1) go func(idx int) { @@ -189,6 +208,7 @@ func TestFN_CC_FriendAccept_ThenSend(t *testing.T) { if errs[idx] == nil && !rsp.Header.Success { errs[idx] = fmt.Errorf("send failed: %s", rsp.Header.ErrorMessage) } + messageIDs[idx] = rsp.GetMessage().GetMessageId() }(i) } wg.Wait() @@ -201,6 +221,10 @@ func TestFN_CC_FriendAccept_ThenSend(t *testing.T) { // 直查 DB：5 条消息全部落库 dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) defer dbV.Close() + for _, id := range messageIDs { + require.NotZero(t, id) + dbV.WaitMessageExists(t, id, 10*time.Second) + } dbV.MessageCount(t, convID, 5) } @@ -218,6 +242,7 @@ func TestFN_CC_MediaUpload_SameHash(t *testing.T) { // 5 goroutine 并发 ApplyUpload 相同 hash（已存在） var wg sync.WaitGroup fileIDs := make([]string, 5) + deduplicated := make([]bool, 5) errs := make([]error, 5) for i := 0; i < 5; i++ { wg.Add(1) @@ -235,6 +260,7 @@ func TestFN_CC_MediaUpload_SameHash(t *testing.T) { errs[idx] = authed.DoAuth("/service/media/apply_upload", req, rsp) if errs[idx] == nil && rsp.Header.Success { fileIDs[idx] = rsp.FileId + deduplicated[idx] = rsp.AlreadyExists && rsp.UploadUrl == "" } }(i) } @@ -246,10 +272,21 @@ func TestFN_CC_MediaUpload_SameHash(t *testing.T) { require.NotEmpty(t, fileIDs[i], "goroutine %d 的 file_id 为空", i) } - // 验证所有返回的 file_id 相同（dedup 正确） - require.Equal(t, existingID, fileIDs[0], "dedup 应返回已存在的 file_id") + // Dedup shares stored bytes while each upload owns a distinct metadata row. + dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer dbV.Close() + existing := dbV.MediaFile(t, existingID) + seen := map[string]bool{existingID: true} for i, id := range fileIDs { - assert.Equal(t, fileIDs[0], id, "goroutine %d 的 file_id 应一致（dedup）", i) + require.True(t, deduplicated[i]) + require.False(t, seen[id], "each upload needs a distinct file reference") + seen[id] = true + complete := &media.CompleteUploadRsp{} + require.NoError(t, authed.DoAuth("/service/media/complete_upload", &media.CompleteUploadReq{RequestId: client.NewRequestID(), FileId: id}, complete)) + require.True(t, complete.GetHeader().GetSuccess()) + record := dbV.MediaFile(t, id) + require.Equal(t, existing.ObjectKey, record.ObjectKey) + require.Equal(t, existing.Bucket, record.Bucket) } } diff --git a/tests/func/conversation_test.go b/tests/func/conversation_test.go index 56423f5..4421be9 100644 --- a/tests/func/conversation_test.go +++ b/tests/func/conversation_test.go @@ -4,6 +4,7 @@ package func_test import ( "testing" + "time" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -437,26 +438,24 @@ func TestSaveDraft_Success(t *testing.T) { func TestSearchConversations_Success(t *testing.T) { owner, _, _ := fixture.RegisterAndLogin(t, HTTP) member, _, _ := fixture.RegisterAndLogin(t, HTTP) - convID := fixture.CreateGroupWithMembers(t, owner, []*client.HTTPClient{member}, "test_group") - - // Search by conversation name (partial match via ik_max_word analyzer). - searchKey := "test_group" + searchKey := "searchgroup" + client.NewRequestID() + convID := fixture.CreateGroupWithMembers(t, owner, []*client.HTTPClient{member}, searchKey) req := &conversation.SearchConversationsReq{ RequestId: client.NewRequestID(), SearchKey: searchKey, } - rsp := &conversation.SearchConversationsRsp{} - err := owner.DoAuth("/service/conversation/search", req, rsp) - require.NoError(t, err) - assert.True(t, rsp.Header.Success) - found := false - for _, c := range rsp.Conversations { - if c.ConversationId == convID { - found = true - break + require.Eventually(t, func() bool { + rsp := &conversation.SearchConversationsRsp{} + if err := owner.DoAuth("/service/conversation/search", req, rsp); err != nil || !rsp.GetHeader().GetSuccess() { + return false } - } - assert.True(t, found, "search by partial convID should find the conversation") + for _, c := range rsp.Conversations { + if c.ConversationId == convID { + return true + } + } + return false + }, 5*time.Second, 100*time.Millisecond, "search must find the indexed conversation by name") } // --------------------------------------------------------------------------- diff --git a/tests/func/idempotent_response_test.go b/tests/func/idempotent_response_test.go new file mode 100644 index 0000000..90fc40f --- /dev/null +++ b/tests/func/idempotent_response_test.go @@ -0,0 +1,41 @@ +//go:build func + +package func_test + +import ( + "testing" + + "github.com/stretchr/testify/require" + + "chatnow-tests/pkg/client" + "chatnow-tests/pkg/fixture" + "chatnow-tests/pkg/verify" + msg "chatnow-tests/proto/chatnow/message" + transmite "chatnow-tests/proto/chatnow/transmite" +) + +// FN-TM-02 | P0 | Existing accepted records cannot produce partial success. +func TestFN_TM_IdempotentRecordWithoutReadableMessage(t *testing.T) { + user, _, convID := fixture.MakeFriends(t, HTTP) + db := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer db.Close() + for _, value := range []string{"accepted:42", "persisted:42"} { + t.Run(value, func(t *testing.T) { + clientID := client.NewRequestID() + key := "im:msg:idem:" + user.UserID + ":" + clientID + t.Cleanup(func() { verify.RedisCLI(t, "DEL", key) }) + verify.RedisCLI(t, "SET", key, value, "EX", "86400") + rsp := &transmite.SendMessageRsp{} + require.NoError(t, user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ + RequestId: client.NewRequestID(), ConversationId: convID, ClientMsgId: clientID, + Content: &msg.MessageContent{Type: msg.MessageType_TEXT, + Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "unreadable-original"}}}, + }, rsp)) + require.False(t, rsp.GetHeader().GetSuccess(), "an unreadable original must not become partial success") + require.EqualValues(t, 9002, rsp.GetHeader().GetErrorCode()) + require.Nil(t, rsp.GetMessage()) + require.Equal(t, value, verify.RedisCLI(t, "GET", key), "do not remove a prior acceptance guard") + db.MessageCount(t, convID, 0) + }) + } +} diff --git a/tests/func/identity_test.go b/tests/func/identity_test.go index 5fcfe2d..66465f7 100644 --- a/tests/func/identity_test.go +++ b/tests/func/identity_test.go @@ -6,7 +6,6 @@ import ( "fmt" "math/rand" "os" - "strings" "testing" "github.com/stretchr/testify/assert" @@ -14,6 +13,7 @@ import ( "chatnow-tests/pkg/client" "chatnow-tests/pkg/fixture" + "chatnow-tests/pkg/verify" identity "chatnow-tests/proto/chatnow/identity" media "chatnow-tests/proto/chatnow/media" ) @@ -506,10 +506,10 @@ func TestFN_ID_UpdateProfileAvatarUpload(t *testing.T) { } rsp := &identity.UpdateProfileRsp{} require.NoError(t, authed.DoAuth("/service/identity/update_profile", req, rsp)) - require.True(t, rsp.Header.Success) + require.True(t, rsp.Header.Success, "update profile: code=%d message=%s", rsp.Header.ErrorCode, rsp.Header.ErrorMessage) require.NotNil(t, rsp.UserInfo) require.NotEmpty(t, rsp.UserInfo.AvatarUrl) - assert.True(t, strings.HasSuffix(rsp.UserInfo.AvatarUrl, "/avatar/"+fileID)) + verify.FileURLContentEquals(t, rsp.UserInfo.AvatarUrl, content) getRsp := &identity.GetProfileRsp{} require.NoError(t, authed.DoAuth( diff --git a/tests/func/jwt_expiry_test.go b/tests/func/jwt_expiry_test.go new file mode 100644 index 0000000..c3d7bad --- /dev/null +++ b/tests/func/jwt_expiry_test.go @@ -0,0 +1,69 @@ +//go:build func + +package func_test + +import ( + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "chatnow-tests/pkg/client" + "chatnow-tests/pkg/fixture" + identity "chatnow-tests/proto/chatnow/identity" +) + +// FN-AM-07 | P1 | Signed expired JWTs remain distinguishable from invalid JWTs. +func TestFN_AM_ExpiredJWTClassification(t *testing.T) { + authed, _, _ := fixture.RegisterAndLogin(t, HTTP) + now := time.Now() + cases := []struct { + name string + variant fixture.JWTVariant + code int32 + }{ + {"expired_30_seconds", fixture.JWTVariant{ExpiresAt: now.Add(-30 * time.Second), IssuedAt: now.Add(-time.Hour)}, 1002}, + {"expired_120_seconds", fixture.JWTVariant{ExpiresAt: now.Add(-120 * time.Second), IssuedAt: now.Add(-time.Hour)}, 1002}, + {"expired_one_day", fixture.JWTVariant{ExpiresAt: now.Add(-24 * time.Hour), IssuedAt: now.Add(-48 * time.Hour)}, 1002}, + {"expired_wrong_signature", fixture.JWTVariant{ExpiresAt: now.Add(-24 * time.Hour), IssuedAt: now.Add(-48 * time.Hour), WrongSigningKey: true}, 1003}, + {"expired_unknown_key", fixture.JWTVariant{ExpiresAt: now.Add(-24 * time.Hour), IssuedAt: now.Add(-48 * time.Hour), UnknownKeyID: true}, 1003}, + {"future_issued_at", fixture.JWTVariant{ExpiresAt: now.Add(time.Hour), IssuedAt: now.Add(5 * time.Minute)}, 1003}, + {"future_not_before", fixture.JWTVariant{ExpiresAt: now.Add(time.Hour), IssuedAt: now.Add(-time.Hour), NotBefore: now.Add(5 * time.Minute)}, 1003}, + {"expired_future_issued_at", fixture.JWTVariant{ExpiresAt: now.Add(-24 * time.Hour), IssuedAt: now.Add(5 * time.Minute)}, 1003}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + refresh := fixture.SignJWTVariant(t, authed.RefreshToken, tc.variant) + rsp := &identity.RefreshTokenRsp{} + err := HTTP.DoNoAuth("/service/identity/refresh_token", &identity.RefreshTokenReq{ + RequestId: client.NewRequestID(), RefreshToken: refresh, + }, rsp) + require.NoError(t, err) + require.NotNil(t, rsp.Header) + assert.False(t, rsp.Header.Success) + assert.Equal(t, tc.code, rsp.Header.ErrorCode, "Identity must distinguish expiration from invalid verification") + assert.True(t, rsp.Tokens == nil, "a rejected token must never produce new credentials") + + access := fixture.SignJWTVariant(t, authed.AccessToken, tc.variant) + profile := &identity.GetProfileRsp{} + err = HTTP.Do("/service/identity/get_profile", &identity.GetProfileReq{RequestId: client.NewRequestID()}, profile, access) + require.Error(t, err, "Gateway must reject the invalid token") + assert.ErrorContains(t, err, "http status 401:") + if tc.code == 1002 { + assert.ErrorContains(t, err, "token expired", "Gateway must preserve the expiration reason") + } + }) + } + t.Run("valid_tokens_still_work", func(t *testing.T) { + profile := &identity.GetProfileRsp{} + require.NoError(t, authed.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{RequestId: client.NewRequestID()}, profile)) + require.True(t, profile.Header.Success) + rsp := &identity.RefreshTokenRsp{} + require.NoError(t, HTTP.DoNoAuth("/service/identity/refresh_token", &identity.RefreshTokenReq{ + RequestId: client.NewRequestID(), RefreshToken: authed.RefreshToken, + }, rsp)) + require.NotNil(t, rsp.Header) + require.True(t, rsp.Header.Success) + }) +} diff --git a/tests/func/media_test.go b/tests/func/media_test.go index 8d14757..ce4090a 100644 --- a/tests/func/media_test.go +++ b/tests/func/media_test.go @@ -8,6 +8,7 @@ import ( "fmt" "io" "net/http" + "net/url" "os" "path" "regexp" @@ -17,6 +18,7 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "google.golang.org/protobuf/proto" "chatnow-tests/pkg/client" "chatnow-tests/pkg/fixture" @@ -92,17 +94,6 @@ func TestGetFileInfo_NotFound(t *testing.T) { assert.False(t, rsp.Header.Success) } -func TestSpeechRecognition_Success(t *testing.T) { - authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - req := &media.SpeechRecognitionReq{ - RequestId: client.NewRequestID(), SpeechContent: []byte("fake-audio-data"), - } - rsp := &media.SpeechRecognitionRsp{} - err := authed.DoAuth("/service/media/speech_recognition", req, rsp) - require.NoError(t, err) - assert.True(t, rsp.Header.Success) -} - // FN-MD-01 | P0 | happy path | 三步上传全链路：apply -> PUT -> complete，验证 file_id 可用 + MinIO 落对象 func TestFN_MD_CompleteUpload_Success(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) @@ -117,6 +108,11 @@ func TestFN_MD_CompleteUpload_Success(t *testing.T) { require.NoError(t, authed.DoAuth("/service/media/get_file_info", infoReq, infoRsp)) require.True(t, infoRsp.Header.Success) require.Equal(t, int64(len(content)), infoRsp.FileInfo.FileSize) + download := &media.ApplyDownloadRsp{} + require.NoError(t, authed.DoAuth("/service/media/apply_download", + &media.ApplyDownloadReq{RequestId: client.NewRequestID(), FileId: fileID}, download)) + require.True(t, download.GetHeader().GetSuccess()) + verify.FileURLContentEquals(t, download.DownloadUrl, content) } // FN-MD-02 | P0 | error path | 未 PUT 到 MinIO 就 complete，应失败 @@ -159,11 +155,11 @@ func TestFN_MD_CompleteUpload_AlreadyCompleted(t *testing.T) { // FN-MD-04 | P0 | happy path | 大文件 InitMultipart，返回 upload_id + 推荐 part_size func TestFN_MD_InitMultipart_Success(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - content := make([]byte, 3*1024*1024) // 3MB + content := fixture.MultipartPDFContent(3 * 1024 * 1024) // 3MB hash := sha256.Sum256(content) req := &media.InitMultipartReq{ RequestId: client.NewRequestID(), FileName: "big.bin", - FileSize: int64(len(content)), MimeType: "application/octet-stream", + FileSize: int64(len(content)), MimeType: "application/pdf", ContentHash: fmt.Sprintf("sha256:%x", hash), Purpose: media.MediaPurpose_CHAT, } rsp := &media.InitMultipartRsp{} @@ -194,11 +190,11 @@ func TestFN_MD_InitMultipart_FileTooLarge(t *testing.T) { // FN-MD-06 | P0 | happy path | ApplyPartUpload 获取分片 presigned URL func TestFN_MD_ApplyPartUpload_Success(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - content := make([]byte, 3*1024*1024) + content := fixture.MultipartPDFContent(3 * 1024 * 1024) hash := sha256.Sum256(content) initReq := &media.InitMultipartReq{ RequestId: client.NewRequestID(), FileName: "parts.bin", - FileSize: int64(len(content)), MimeType: "application/octet-stream", + FileSize: int64(len(content)), MimeType: "application/pdf", ContentHash: fmt.Sprintf("sha256:%x", hash), Purpose: media.MediaPurpose_CHAT, } initRsp := &media.InitMultipartRsp{} @@ -214,14 +210,11 @@ func TestFN_MD_ApplyPartUpload_Success(t *testing.T) { assert.NotEmpty(t, rsp.UploadUrl) } -// FN-MD-07 | P0 | happy path | init -> upload 3 parts -> complete，验证合并后 file_id 可查 +// FN-MD-07 | P0 | happy path | init -> upload 2 parts -> complete，验证合并后 file_id 可查 func TestFN_MD_CompleteMultipart_FullFlow(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - content := make([]byte, 6*1024*1024) // 6MB -> 3 parts @ 2MB - for i := range content { - content[i] = byte(i % 256) - } - fileID := fixture.UploadLargeFile(t, authed, content, "application/octet-stream", 2*1024*1024) + content := fixture.MultipartPDFContent(6 * 1024 * 1024) // 6MB -> 2 parts @ 5MB + 1MB + fileID := fixture.UploadLargeFile(t, authed, content, "application/pdf", 5*1024*1024) require.NotEmpty(t, fileID) // 验证 file_info @@ -230,16 +223,21 @@ func TestFN_MD_CompleteMultipart_FullFlow(t *testing.T) { require.NoError(t, authed.DoAuth("/service/media/get_file_info", infoReq, infoRsp)) require.True(t, infoRsp.Header.Success) require.Equal(t, int64(len(content)), infoRsp.FileInfo.FileSize) + download := &media.ApplyDownloadRsp{} + require.NoError(t, authed.DoAuth("/service/media/apply_download", + &media.ApplyDownloadReq{RequestId: client.NewRequestID(), FileId: fileID}, download)) + require.True(t, download.GetHeader().GetSuccess()) + verify.FileURLContentEquals(t, download.DownloadUrl, content) } // FN-MD-08 | P1 | error path | 缺少某个 part number，CompleteMultipart 拒绝 func TestFN_MD_CompleteMultipart_MissingPart(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - content := make([]byte, 6*1024*1024) + content := fixture.MultipartPDFContent(6 * 1024 * 1024) hash := sha256.Sum256(content) initReq := &media.InitMultipartReq{ RequestId: client.NewRequestID(), FileName: "missing.bin", - FileSize: int64(len(content)), MimeType: "application/octet-stream", + FileSize: int64(len(content)), MimeType: "application/pdf", ContentHash: fmt.Sprintf("sha256:%x", hash), Purpose: media.MediaPurpose_CHAT, } initRsp := &media.InitMultipartRsp{} @@ -253,7 +251,7 @@ func TestFN_MD_CompleteMultipart_MissingPart(t *testing.T) { partRsp := &media.ApplyPartRsp{} require.NoError(t, authed.DoAuth("/service/media/apply_part_upload", partReq, partRsp)) - partContent := content[:2*1024*1024] + partContent := content[:5*1024*1024] httpReq, err := http.NewRequest("PUT", partRsp.UploadUrl, bytes.NewReader(partContent)) require.NoError(t, err) putResp, err := http.DefaultClient.Do(httpReq) @@ -273,11 +271,11 @@ func TestFN_MD_CompleteMultipart_MissingPart(t *testing.T) { // FN-MD-09 | P1 | happy path | init -> abort，验证 upload_id 失效 func TestFN_MD_AbortMultipart_Success(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - content := make([]byte, 3*1024*1024) + content := fixture.MultipartPDFContent(3 * 1024 * 1024) hash := sha256.Sum256(content) initReq := &media.InitMultipartReq{ RequestId: client.NewRequestID(), FileName: "abort.bin", - FileSize: int64(len(content)), MimeType: "application/octet-stream", + FileSize: int64(len(content)), MimeType: "application/pdf", ContentHash: fmt.Sprintf("sha256:%x", hash), Purpose: media.MediaPurpose_CHAT, } initRsp := &media.InitMultipartRsp{} @@ -299,11 +297,11 @@ func TestFN_MD_AbortMultipart_Success(t *testing.T) { // FN-MD-10 | P2 | idempotent | 重复 abort 幂等 func TestFN_MD_AbortMultipart_AlreadyAborted(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - content := make([]byte, 3*1024*1024) + content := fixture.MultipartPDFContent(3 * 1024 * 1024) hash := sha256.Sum256(content) initReq := &media.InitMultipartReq{ RequestId: client.NewRequestID(), FileName: "abort2.bin", - FileSize: int64(len(content)), MimeType: "application/octet-stream", + FileSize: int64(len(content)), MimeType: "application/pdf", ContentHash: fmt.Sprintf("sha256:%x", hash), Purpose: media.MediaPurpose_CHAT, } initRsp := &media.InitMultipartRsp{} @@ -321,7 +319,7 @@ func TestFN_MD_AbortMultipart_AlreadyAborted(t *testing.T) { _ = abortRsp2.Header.Success } -// FN-MD-11 | P0 | dedup | 相同 content_hash，第二次 apply 返回 already_exists=true + 相同 file_id +// FN-MD-11 | P0 | dedup | same hash reuses bytes with a distinct file reference. func TestFN_MD_ApplyUpload_Dedup_SameHash(t *testing.T) { authed, _, _ := fixture.RegisterAndLogin(t, HTTP) content := []byte("md-dedup-same-hash") @@ -341,8 +339,12 @@ func TestFN_MD_ApplyUpload_Dedup_SameHash(t *testing.T) { require.NoError(t, authed.DoAuth("/service/media/apply_upload", applyReq, applyRsp)) require.True(t, applyRsp.Header.Success) assert.True(t, applyRsp.AlreadyExists, "相同 hash 应返回 already_exists=true") - assert.Equal(t, fileID, applyRsp.FileId, "dedup 应返回相同 file_id") + require.NotEmpty(t, applyRsp.FileId) + assert.NotEqual(t, fileID, applyRsp.FileId, "dedup creates a new file reference") assert.Empty(t, applyRsp.UploadUrl, "dedup 时不应返回 upload_url") + dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer dbV.Close() + assert.Equal(t, dbV.MediaFile(t, fileID).ObjectKey, dbV.MediaFile(t, applyRsp.FileId).ObjectKey) } // FN-MD-12 | P0 | quota | 超用户配额拒绝（默认 5GB，此处用大文件触发） @@ -379,6 +381,11 @@ func TestFN_MD_ApplyUpload_QuotaRemaining(t *testing.T) { require.NoError(t, authed.DoAuth("/service/media/get_file_info", infoReq, infoRsp)) require.True(t, infoRsp.Header.Success) require.Equal(t, int64(len(content)), infoRsp.FileInfo.FileSize) + download := &media.ApplyDownloadRsp{} + require.NoError(t, authed.DoAuth("/service/media/apply_download", + &media.ApplyDownloadReq{RequestId: client.NewRequestID(), FileId: fileID}, download)) + require.True(t, download.GetHeader().GetSuccess()) + verify.FileURLContentEquals(t, download.DownloadUrl, content) } // FN-MD-14 | P0 | happy path | 上传后下载，验证内容一致 @@ -507,3 +514,72 @@ func TestFN_MD_MagicMismatchQuarantined(t *testing.T) { assert.False(t, rsp.Header.Success) assert.Equal(t, int32(5008), rsp.Header.ErrorCode) } + +// FN-MD-21 | P0 | Multipart signatures bind the part number as well as the upload ID. +func TestFN_MD_MultipartSignatureBindsPart(t *testing.T) { + authed, _, _ := fixture.RegisterAndLogin(t, HTTP) + content := fixture.MultipartPDFContent(128) + hash := sha256.Sum256(content) + init := &media.InitMultipartRsp{} + require.NoError(t, authed.DoAuth("/service/media/init_multipart", &media.InitMultipartReq{ + RequestId: client.NewRequestID(), FileName: "signed.pdf", FileSize: int64(len(content)), + MimeType: "application/pdf", ContentHash: fmt.Sprintf("sha256:%x", hash), Purpose: media.MediaPurpose_CHAT, + }, init)) + require.True(t, init.GetHeader().GetSuccess()) + t.Cleanup(func() { + _ = authed.DoAuth("/service/media/abort_multipart", &media.AbortMultipartReq{ + RequestId: client.NewRequestID(), UploadId: init.UploadId, + }, &media.AbortMultipartRsp{}) + }) + part := &media.ApplyPartRsp{} + require.NoError(t, authed.DoAuth("/service/media/apply_part_upload", &media.ApplyPartReq{ + RequestId: client.NewRequestID(), UploadId: init.UploadId, PartNumber: 1, + }, part)) + require.True(t, part.GetHeader().GetSuccess()) + parsed, err := url.Parse(part.UploadUrl) + require.NoError(t, err) + query := parsed.Query() + query.Set("partNumber", "2") + parsed.RawQuery = query.Encode() + put := func(target string) int { + request, err := http.NewRequest(http.MethodPut, target, bytes.NewReader(content)) + require.NoError(t, err) + response, err := (&http.Client{Timeout: 10 * time.Second}).Do(request) + if err != nil { + t.Fatal("multipart PUT transport failed") + } + defer response.Body.Close() + return response.StatusCode + } + require.Equal(t, http.StatusForbidden, put(parsed.String())) + require.Equal(t, http.StatusOK, put(part.UploadUrl)) +} + +// FN-MD-22 | P0 | Every multipart route rejects missing authentication before RPC dispatch. +func TestFN_MD_MultipartRequiresAuthentication(t *testing.T) { + for _, test := range []struct { + path string + request, response proto.Message + }{ + {"init_multipart", &media.InitMultipartReq{}, &media.InitMultipartRsp{}}, + {"apply_part_upload", &media.ApplyPartReq{}, &media.ApplyPartRsp{}}, + {"complete_multipart", &media.CompleteMultipartReq{}, &media.CompleteMultipartRsp{}}, + {"abort_multipart", &media.AbortMultipartReq{}, &media.AbortMultipartRsp{}}, + } { + t.Run(test.path, func(t *testing.T) { + err := HTTP.DoNoAuth("/service/media/"+test.path, test.request, test.response) + require.ErrorContains(t, err, "http status 401") + }) + } +} + +// FN-MD-23 | P1 | Valid PCM16 must report unavailable until an ASR engine processes it. +func TestFN_MD_SpeechRecognition_BackendUnavailable(t *testing.T) { + authed, _, _ := fixture.RegisterAndLogin(t, HTTP) + response := &media.SpeechRecognitionRsp{} + require.NoError(t, authed.DoAuth("/service/media/speech_recognition", &media.SpeechRecognitionReq{ + RequestId: client.NewRequestID(), SpeechContent: []byte{0, 0}, + }, response)) + require.False(t, response.GetHeader().GetSuccess()) + require.Equal(t, "speech recognition unavailable", response.GetHeader().GetErrorMessage()) +} diff --git a/tests/func/message_test.go b/tests/func/message_test.go index de0ec4f..6c8da98 100644 --- a/tests/func/message_test.go +++ b/tests/func/message_test.go @@ -13,26 +13,12 @@ import ( "chatnow-tests/pkg/fixture" "chatnow-tests/pkg/verify" msg "chatnow-tests/proto/chatnow/message" - transmite "chatnow-tests/proto/chatnow/transmite" ) -// Helper: sends a text message and returns the message_id and seq_id. +// Message API tests require the shared fixture's durable-message boundary. func sendMsg(t *testing.T, c *client.HTTPClient, convID string, text string) (msgID int64, seqID uint64) { t.Helper() - req := &transmite.SendMessageReq{ - RequestId: client.NewRequestID(), - ConversationId: convID, - Content: &msg.MessageContent{ - Type: msg.MessageType_TEXT, - Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: text}}, - }, - ClientMsgId: client.NewRequestID(), - } - rsp := &transmite.SendMessageRsp{} - require.NoError(t, c.DoAuth("/service/transmite/send", req, rsp)) - require.True(t, rsp.GetHeader().GetSuccess()) - require.NotNil(t, rsp.GetMessage()) - return rsp.GetMessage().GetMessageId(), rsp.GetMessage().GetSeqId() + return fixture.SendTextMessage(t, c, convID, text) } // --------------------------------------------------------------------------- @@ -105,19 +91,22 @@ func TestGetMessagesById_Success(t *testing.T) { func TestSearchMessages_Success(t *testing.T) { a, _, convID := setupConv(t) - sendMsg(t, a, convID, "unique search term zebra42") + keyword := "search" + client.NewRequestID() + messageID, _ := sendMsg(t, a, convID, keyword) + verify.NewESVerifier(Cfg.Database.ESURL).MessageIndexed(t, messageID, keyword) req := &msg.SearchMessagesReq{ RequestId: client.NewRequestID(), ConversationId: convID, - Keyword: "zebra42", + Keyword: keyword, Limit: 20, } rsp := &msg.SearchMessagesRsp{} err := a.DoAuth("/service/message/search", req, rsp) require.NoError(t, err) - // Search may return empty if ES is not available; verify response is valid. - assert.True(t, rsp.GetHeader().GetSuccess() || !rsp.GetHeader().GetSuccess(), "response received") + require.True(t, rsp.GetHeader().GetSuccess(), "search failed with code %d", rsp.GetHeader().GetErrorCode()) + require.Len(t, rsp.GetMessages(), 1) + assert.Equal(t, messageID, rsp.GetMessages()[0].GetMessageId()) } // --------------------------------------------------------------------------- @@ -379,6 +368,9 @@ func TestFN_MS_SyncMessages_NotMember(t *testing.T) { func TestFN_MS_RecallMessage_ByNonAuthor(t *testing.T) { alice, bob, convID := fixture.MakeFriends(t, HTTP) msgID, _ := fixture.SendTextMessage(t, alice, convID, "will-try-recall") + verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer verifier.Close() + verifier.WaitMessageExists(t, msgID, 10*time.Second) // bob（非发送者）尝试撤回 alice 的消息 req := &msg.RecallMessageReq{ @@ -393,12 +385,17 @@ func TestFN_MS_RecallMessage_ByNonAuthor(t *testing.T) { assert.Equal(t, int32(3003), rsp.Header.ErrorCode, "错误码应为 CONVERSATION_NO_PERMISSION(3003)") } -// FN-MS-10 | P0 | error path | 删除他人消息应失败 -func TestFN_MS_DeleteMessages_NotOwned(t *testing.T) { +// FN-MS-10 | P0 | isolation | deleting a received message affects only the caller. +func TestFN_MS_DeleteMessages_OnlyOwnTimeline(t *testing.T) { alice, bob, convID := fixture.MakeFriends(t, HTTP) msgID, _ := fixture.SendTextMessage(t, alice, convID, "will-try-delete") + verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer verifier.Close() + verifier.WaitMessageExists(t, msgID, 10*time.Second) + verifier.UserTimelineExists(t, alice.UserID, convID, 1) + verifier.UserTimelineExists(t, bob.UserID, convID, 1) - // bob 尝试删除 alice 的消息 + // Bob removes his own view of Alice's message. req := &msg.DeleteMessagesReq{ RequestId: client.NewRequestID(), ConversationId: convID, @@ -407,15 +404,28 @@ func TestFN_MS_DeleteMessages_NotOwned(t *testing.T) { rsp := &msg.DeleteMessagesRsp{} err := bob.DoAuth("/service/message/delete", req, rsp) require.NoError(t, err) - require.False(t, rsp.Header.Success, "删除他人消息应失败") - assert.Equal(t, int32(3003), rsp.Header.ErrorCode, "错误码应为 CONVERSATION_NO_PERMISSION(3003)") + require.True(t, rsp.GetHeader().GetSuccess()) + verifier.UserTimelineExists(t, bob.UserID, convID, 0) + verifier.UserTimelineExists(t, alice.UserID, convID, 1) + verifier.MessageExists(t, msgID) + verifier.MessageStatus(t, msgID, 0) + outsider, _, _ := fixture.RegisterAndLogin(t, HTTP) + denied := &msg.DeleteMessagesRsp{} + require.NoError(t, outsider.DoAuth("/service/message/delete", req, denied)) + require.False(t, denied.GetHeader().GetSuccess()) + require.Equal(t, int32(3002), denied.GetHeader().GetErrorCode()) + verifier.UserTimelineExists(t, alice.UserID, convID, 1) } // FN-MS (untested) | P0 | SelectByClientMsgId 查询存在 func TestFN_MS_SelectByClientMsgId_Found(t *testing.T) { alice, _, convID := fixture.MakeFriends(t, HTTP) clientMsgID := client.NewRequestID() - msgID, _, _ := fixture.SendTextMessageWithClientMsgId(t, alice, convID, "select-by-client-msg-id", clientMsgID) + msgID, _, accepted := fixture.SendTextMessageWithClientMsgId(t, alice, convID, "select-by-client-msg-id", clientMsgID) + require.True(t, accepted) + verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer verifier.Close() + verifier.WaitMessageExists(t, msgID, 10*time.Second) req := &msg.SelectByClientMsgIdReq{ RequestId: client.NewRequestID(), diff --git a/tests/func/presence_test.go b/tests/func/presence_test.go index 82072ec..0337932 100644 --- a/tests/func/presence_test.go +++ b/tests/func/presence_test.go @@ -120,25 +120,30 @@ func TestSendTyping_GroupChat_Success(t *testing.T) { // FN-PR-01 | P1 | state transition | 同用户多设备在线，presence 为 online func TestFN_PR_GetPresence_MultiDevice(t *testing.T) { - authed, _, _ := fixture.RegisterAndLogin(t, HTTP) - - // 设备 A 连接 WS - wsA, err := client.NewWSClient(HTTP.Config(), authed.AccessToken, authed.UserID, "device-A") - require.NoError(t, err) - defer wsA.Close() - - // 设备 B 连接 WS（同用户不同设备） - wsB, err := client.NewWSClient(HTTP.Config(), authed.AccessToken, authed.UserID, "device-B") - require.NoError(t, err) - defer wsB.Close() - - // 查询 presence，应为 ONLINE - req := &presence.GetPresenceReq{RequestId: client.NewRequestID(), UserId: authed.UserID} - rsp := &presence.GetPresenceRsp{} - require.NoError(t, authed.DoAuth("/service/presence/get", req, rsp)) - require.True(t, rsp.Header.Success) - assert.Equal(t, presence.PresenceState_ONLINE, rsp.Presence.AggregatedState) - assert.GreaterOrEqual(t, len(rsp.Presence.Devices), 2, "多设备应列出 >=2 个 device") + _, username, password := fixture.RegisterAndLogin(t, HTTP) + // Each device needs its own Identity-issued JWT; changing the WS payload + // alone cannot override the authoritative device claim. + deviceA := fixture.LoginUser(t, HTTP, username, password) + deviceB := fixture.LoginUser(t, HTTP, username, password) + require.NotEqual(t, deviceA.DeviceID, deviceB.DeviceID) + fixture.ConnectWS(t, deviceA) + fixture.ConnectWS(t, deviceB) + require.Eventually(t, func() bool { + rsp := &presence.GetPresenceRsp{} + err := deviceA.DoAuth("/service/presence/get", &presence.GetPresenceReq{ + RequestId: client.NewRequestID(), UserId: deviceA.UserID, + }, rsp) + if err != nil || !rsp.GetHeader().GetSuccess() { + return false + } + devices := map[string]bool{} + for _, device := range rsp.GetPresence().GetDevices() { + if device.GetState() == presence.PresenceState_ONLINE { + devices[device.GetDeviceId()] = true + } + } + return devices[deviceA.DeviceID] && devices[deviceB.DeviceID] + }, 5*time.Second, 50*time.Millisecond, "both authenticated devices must be online") } // FN-PR-02 | P1 | state transition | 心跳续期，TTL 刷新 diff --git a/tests/func/scenarios_test.go b/tests/func/scenarios_test.go index 63823af..49fa93e 100644 --- a/tests/func/scenarios_test.go +++ b/tests/func/scenarios_test.go @@ -380,7 +380,7 @@ func TestScenario_MediaUploadFullFlow(t *testing.T) { dlResp.Body.Close() assert.Equal(t, content, body, "下载内容与上传不一致") - // Step 5: 重复 ApplyUpload（相同 hash）-> dedup 返回相同 file_id + // Step 5: duplicate content shares bytes through a distinct file reference. applyReq2 := &media.ApplyUploadReq{ RequestId: client.NewRequestID(), FileName: "sc05-dup.txt", FileSize: int64(len(content)), MimeType: "text/plain", @@ -390,14 +390,16 @@ func TestScenario_MediaUploadFullFlow(t *testing.T) { require.NoError(t, user.DoAuth("/service/media/apply_upload", applyReq2, applyRsp2)) require.True(t, applyRsp2.Header.Success) assert.True(t, applyRsp2.AlreadyExists, "相同 hash 应返回 already_exists=true") - assert.Equal(t, fileID, applyRsp2.FileId, "dedup 应返回相同 file_id") - - // Step 6: 大文件 multipart（6MB -> 3 parts @ 2MB） - bigContent := make([]byte, 6*1024*1024) - for i := range bigContent { - bigContent[i] = byte(i % 256) - } - bigFileID := fixture.UploadLargeFile(t, user, bigContent, "application/octet-stream", 2*1024*1024) + require.NotEmpty(t, applyRsp2.FileId) + assert.NotEqual(t, fileID, applyRsp2.FileId) + assert.Empty(t, applyRsp2.UploadUrl) + mediaDB := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer mediaDB.Close() + assert.Equal(t, mediaDB.MediaFile(t, fileID).ObjectKey, mediaDB.MediaFile(t, applyRsp2.FileId).ObjectKey) + + // Step 6: Multipart PDF, 6 MiB split into 5 MiB + 1 MiB. + bigContent := fixture.MultipartPDFContent(6 * 1024 * 1024) + bigFileID := fixture.UploadLargeFile(t, user, bigContent, "application/pdf", 5*1024*1024) require.NotEmpty(t, bigFileID) // 下载大文件验证 @@ -443,6 +445,7 @@ func TestScenario_MessageReliability(t *testing.T) { msgID1, seq1, success1 := fixture.SendTextMessageWithClientMsgId(t, alice, convID, "reliability-test", clientMsgID) require.True(t, success1, "第一次发送应成功") require.NotZero(t, msgID1) + require.NotZero(t, seq1) // Step 2: 用相同 client_msg_id 重发（模拟网络重传） msgID2, seq2, success2 := fixture.SendTextMessageWithClientMsgId(t, alice, convID, "reliability-test", clientMsgID) @@ -452,6 +455,15 @@ func TestScenario_MessageReliability(t *testing.T) { assert.Equal(t, msgID1, msgID2, "相同 client_msg_id 应返回相同 message_id") assert.Equal(t, seq1, seq2, "相同 client_msg_id 应返回相同 seq_id") } + // A retry during asynchronous persistence may be unavailable. Recovery + // must still produce the complete original result, not merely avoid success. + dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer dbV.Close() + dbV.WaitMessageExists(t, msgID1, 10*time.Second) + recoveredID, recoveredSeq, recovered := fixture.SendTextMessageWithClientMsgId(t, alice, convID, "reliability-test", clientMsgID) + require.True(t, recovered, "retry must succeed after the original is persisted") + require.Equal(t, msgID1, recoveredID) + require.Equal(t, seq1, recoveredSeq) // Step 3: bob sync 验证收到该消息（仅 1 条） syncReq := &msg.SyncMessagesReq{ @@ -479,50 +491,40 @@ func TestScenario_MessageReliability(t *testing.T) { assert.Equal(t, msgID1, selectRsp.Message.MessageId) // Step 5: 数据一致性 - DB 仅 1 条（不重复） - dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) - defer dbV.Close() dbV.MessageCount(t, convID, 1) dbV.MessageByClientMsgId(t, clientMsgID, true) } // --------------------------------------------------------------------------- -// Scenario 7: Multi-Device Login Kick -// SC-07 | P1 | scenario | 多设备登录：设备 A 登录 -> 设备 B 登录 -> A 被踢 -> A token 失效 +// Scenario 7: Multi-device coexistence +// SC-07 | P1 | scenario | both devices remain authenticated and receive messages. // --------------------------------------------------------------------------- func TestScenario_MultiDeviceLogin(t *testing.T) { - // 先注册用户（LoginUser 要求用户已存在） - username := "sc07_user_" + client.NewRequestID()[:8] - password := "Sc07@123456" - - regReq := &identity.RegisterReq{ - RequestId: client.NewRequestID(), - Credential: &identity.RegisterReq_UsernamePwd{ - UsernamePwd: &identity.UsernamePassword{Username: username, Password: password}, - }, - Nickname: username, - } - require.NoError(t, HTTP.DoNoAuth("/service/identity/register", regReq, &identity.RegisterRsp{})) - - // 设备 A 登录 + _, username, password := fixture.RegisterAndLogin(t, HTTP) deviceA := fixture.LoginUser(t, HTTP, username, password) - require.NotEmpty(t, deviceA.AccessToken) - - // 验证 A 能调 API - profileReq := &identity.GetProfileReq{RequestId: client.NewRequestID()} - require.NoError(t, deviceA.DoAuth("/service/identity/get_profile", profileReq, &identity.GetProfileRsp{})) - - // 设备 B 登录同用户 + wsA := fixture.ConnectWS(t, deviceA) deviceB := fixture.LoginUser(t, HTTP, username, password) - require.NotEmpty(t, deviceB.AccessToken) - require.NotEqual(t, deviceA.AccessToken, deviceB.AccessToken, "B 的 token 应不同于 A") - - // 设备 A 的 token 应失效（被踢） - err := deviceA.DoAuth("/service/identity/get_profile", profileReq, &identity.GetProfileRsp{}) - assert.Error(t, err, "设备 A 被踢后 token 应失效") - - // 设备 B 仍可调 API - require.NoError(t, deviceB.DoAuth("/service/identity/get_profile", profileReq, &identity.GetProfileRsp{})) + wsB := fixture.ConnectWS(t, deviceB) + require.NotEqual(t, deviceA.AccessToken, deviceB.AccessToken) + require.NotEqual(t, deviceA.DeviceID, deviceB.DeviceID) + for _, device := range []*client.HTTPClient{deviceA, deviceB} { + rsp := &identity.GetProfileRsp{} + require.NoError(t, device.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{RequestId: client.NewRequestID()}, rsp)) + require.True(t, rsp.GetHeader().GetSuccess()) + require.Equal(t, deviceA.UserID, rsp.GetUserInfo().GetUserId()) + } + peer, _, _ := fixture.RegisterAndLogin(t, HTTP) + convID := fixture.CreateGroupWithMembers(t, deviceA, []*client.HTTPClient{peer}, "multi-device-delivery") + marker := "multi-device-" + client.NewRequestID() + fixture.SendTextMessage(t, peer, convID, marker) + for _, ws := range []*client.WSClient{wsA, wsB} { + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + notify, err := ws.WaitForNotify(ctx, int32(push.NotifyType_CHAT_MESSAGE_NOTIFY)) + cancel() + require.NoError(t, err, "each authenticated device must receive the message") + require.Equal(t, marker, notify.GetNewMessageInfo().GetMessageInfo().GetContent().GetText().GetText()) + } } // --------------------------------------------------------------------------- @@ -596,17 +598,18 @@ func TestScenario_UnreadCountConsistency(t *testing.T) { // Step 2: b ListConversations，验证 unread_count=3 listReq := &conversation.ListConversationsReq{RequestId: client.NewRequestID()} - listRsp := &conversation.ListConversationsRsp{} - require.NoError(t, b.DoAuth("/service/conversation/list", listReq, listRsp)) - var bobConv *conversation.Conversation - for _, c := range listRsp.Conversations { - if c.ConversationId == convID { - bobConv = c - break + require.Eventually(t, func() bool { + listRsp := &conversation.ListConversationsRsp{} + if err := b.DoAuth("/service/conversation/list", listReq, listRsp); err != nil || !listRsp.GetHeader().GetSuccess() { + return false } - } - require.NotNil(t, bobConv, "b 的会话列表中应包含 convID") - assert.Equal(t, uint64(3), bobConv.Self.UnreadCount, "b 未读数应为 3") + for _, c := range listRsp.Conversations { + if c.ConversationId == convID { + return c.GetSelf().GetUnreadCount() == 3 + } + } + return false + }, 5*time.Second, 100*time.Millisecond, "recipient unread count must converge to three") // Step 3: 数据一致性 - DB unread_count=3 dbV := verify.NewDBVerifier(Cfg.Database.MySQLDSN) diff --git a/tests/func/setup_test.go b/tests/func/setup_test.go index 862f991..1c3fdbe 100644 --- a/tests/func/setup_test.go +++ b/tests/func/setup_test.go @@ -15,7 +15,7 @@ var HTTP *client.HTTPClient var Cfg *client.Config func TestMain(m *testing.M) { - Cfg = client.LoadConfig("") + Cfg = client.LoadConfig("../config.yaml") HTTP = client.NewHTTPClient(Cfg) if err := cleanup.WaitForStackReady(Cfg, 120*1e9); err != nil { panic(err) diff --git a/tests/func/transmite_test.go b/tests/func/transmite_test.go index 4866dfc..9d90b81 100644 --- a/tests/func/transmite_test.go +++ b/tests/func/transmite_test.go @@ -5,6 +5,7 @@ package func_test import ( "fmt" "testing" + "time" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -359,6 +360,9 @@ func TestSendMessage_Idempotent(t *testing.T) { require.NotNil(t, rsp1.Message) // Second send with same client_msg_id (new request_id). + db := verify.NewDBVerifier(Cfg.Database.MySQLDSN) + defer db.Close() + db.WaitMessageExists(t, rsp1.Message.MessageId, 10*time.Second) req2 := &transmite.SendMessageReq{ RequestId: client.NewRequestID(), ConversationId: convID, @@ -374,6 +378,8 @@ func TestSendMessage_Idempotent(t *testing.T) { require.True(t, rsp2.Header.Success) require.NotNil(t, rsp2.Message) assert.Equal(t, rsp1.Message.MessageId, rsp2.Message.MessageId, "same client_msg_id should return same message_id") + require.NotZero(t, rsp1.Message.SeqId) + assert.Equal(t, rsp1.Message.SeqId, rsp2.Message.SeqId) } // --------------------------------------------------------------------------- diff --git a/tests/func/ws_notify_test.go b/tests/func/ws_notify_test.go index 39f6c16..e78435d 100644 --- a/tests/func/ws_notify_test.go +++ b/tests/func/ws_notify_test.go @@ -9,9 +9,11 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "google.golang.org/protobuf/proto" "chatnow-tests/pkg/client" "chatnow-tests/pkg/fixture" + conversation "chatnow-tests/proto/chatnow/conversation" identity "chatnow-tests/proto/chatnow/identity" msg "chatnow-tests/proto/chatnow/message" presence "chatnow-tests/proto/chatnow/presence" @@ -126,9 +128,12 @@ func TestFN_WS_ConversationCreateNotify(t *testing.T) { // member 应收到 CONVERSATION_CREATE_NOTIFY ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() - _, err := wsMember.WaitForNotify(ctx, int32(push.NotifyType_CONVERSATION_CREATE_NOTIFY)) + notification, err := wsMember.WaitForNotify(ctx, int32(push.NotifyType_CONVERSATION_CREATE_NOTIFY)) require.NoError(t, err, "member 应收到会话创建通知") - _ = convID + created := &conversation.Conversation{} + require.NoError(t, proto.Unmarshal(notification.GetNewConversationInfo().GetConversationPayload(), created)) + require.Equal(t, convID, created.ConversationId) + require.Nil(t, created.Self, "broadcast payload must not expose the creator's personal membership state") } // FN-WS-05 | P1 | websocket | 订阅的用户上线/离线，WS 收到通知 diff --git a/tests/pkg/chaos/address.go b/tests/pkg/chaos/address.go new file mode 100644 index 0000000..6fffc6f --- /dev/null +++ b/tests/pkg/chaos/address.go @@ -0,0 +1,240 @@ +package chaos + +import ( + "context" + "encoding/json" + "fmt" + "net/netip" + "os/exec" + "slices" + "strings" + "testing" + "time" + + "chatnow-tests/pkg/client" +) + +type containerAddress struct { + ID, StartedAt, Project, Image string + Running bool + Networks map[string]struct { + IPAddress, NetworkID string + Aliases []string + } +} + +func dockerAddress(cfg *client.Config, args ...string) ([]byte, error) { + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "docker", args...) + cmd.Dir = cfg.Infra.ComposeDir + out, err := cmd.CombinedOutput() + if err != nil { + return nil, fmt.Errorf("docker %v: %w: %s", args, err, out) + } + return out, nil +} + +func inspectAddress(cfg *client.Config, id string) (containerAddress, error) { + // Select only network/process metadata; never collect container credentials. + format := `{"ID":{{json .Id}},"Image":{{json .Config.Image}},"StartedAt":{{json .State.StartedAt}},"Running":{{json .State.Running}},"Project":{{json (index .Config.Labels "com.docker.compose.project")}},"Networks":{{json .NetworkSettings.Networks}}}` + out, err := dockerAddress(cfg, "inspect", "--format", format, id) + var result containerAddress + if err == nil { + err = json.Unmarshal(out, &result) + } + return result, err +} + +// ChangeIdentityAddress changes only the selected Compose project's Identity +// endpoint and restarts Identity to reconnect its outgoing clients at the new IP. +// Cleanup restores its original address and aliases, including on RED. +// The returned check proves Gateway and Transmite were not restarted to recover. +func ChangeIdentityAddress(t testing.TB, cfg *client.Config) func() { + t.Helper() + services := []string{"identity_server", "gateway_server", "transmite_server"} + before := make([]containerAddress, len(services)) + for i, service := range services { + out, err := dockerAddress(cfg, "compose", "ps", "-q", service) + if err != nil || len(strings.Fields(string(out))) != 1 { + t.Fatalf("expected one running Compose %s container: %v", service, err) + } + before[i], err = inspectAddress(cfg, strings.TrimSpace(string(out))) + if err != nil || !before[i].Running || before[i].Project == "" { + t.Fatalf("inspect running Compose %s: %v", service, err) + } + if before[i].Project != before[0].Project { + t.Fatal("fault endpoints must belong to one Compose project") + } + } + identity := before[0] + if len(identity.Networks) != 1 { + t.Fatal("address fault requires exactly one Identity network") + } + var network, networkID, original string + var aliases []string + for name, endpoint := range identity.Networks { + network, networkID, original, aliases = name, endpoint.NetworkID, endpoint.IPAddress, endpoint.Aliases + } + var networks []struct { + Labels map[string]string + IPAM struct { + Config []struct{ Subnet, Gateway string } + } + Containers map[string]struct{ IPv4Address string } + } + out, err := dockerAddress(cfg, "network", "inspect", networkID) + if err != nil || json.Unmarshal(out, &networks) != nil || len(networks) != 1 { + t.Fatalf("inspect fault network: %v", err) + } + n := networks[0] + if n.Labels["com.docker.compose.project"] != identity.Project { + t.Fatal("refusing to change an external/shared network") + } + oldIP, err := netip.ParseAddr(original) + if err != nil || !oldIP.Is4() { + t.Fatal("address fault requires a Compose IPv4 endpoint") + } + used := map[netip.Addr]bool{oldIP: true} + var subnet netip.Prefix + for _, c := range n.IPAM.Config { + p, e := netip.ParsePrefix(c.Subnet) + if e == nil && p.Contains(oldIP) { + subnet = p.Masked() + } + if gateway, e := netip.ParseAddr(c.Gateway); e == nil { + used[gateway] = true + } + } + for _, c := range n.Containers { + if p, e := netip.ParsePrefix(c.IPv4Address); e == nil { + used[p.Addr()] = true + } + } + if !subnet.IsValid() || !subnet.Addr().Is4() { + t.Fatal("no IPv4 subnet covers the isolated Identity endpoint") + } + last := subnet.Addr().As4() + for bit := subnet.Bits(); bit < 32; bit++ { + last[bit/8] |= 1 << (7 - bit%8) + } + // Prefer the high end, away from normal low-address Compose allocation. + // Network inspection can omit reservations held by stopped containers, so + // Docker's actual allocation probe remains authoritative. + candidate := netip.AddrFrom4(last).Prev() + var replacement netip.Addr + // Older Docker engines reject explicit IPs on auto-allocated subnets. Prove + // support with an inert, test-owned endpoint before touching the live service. + // No credentials or application entrypoint are copied into the probe. + for attempt := 0; attempt < 16; attempt++ { + for subnet.Contains(candidate) && candidate != subnet.Addr() && used[candidate] { + candidate = candidate.Prev() + } + if !subnet.Contains(candidate) || candidate == subnet.Addr() { + break + } + used[candidate] = true + out, probeErr := dockerAddress(cfg, "create", "--network", networkID, "--ip", candidate.String(), + "--entrypoint", "/bin/true", identity.Image) + if probeErr == nil { + probeID := strings.TrimSpace(string(out)) + probeRemoved := false + t.Cleanup(func() { + if !probeRemoved { + if _, e := dockerAddress(cfg, "rm", "--force", probeID); e != nil { + t.Errorf("remove address capability probe: %v", e) + } + } + }) + _, probeErr = dockerAddress(cfg, "start", "--attach", probeID) + if _, e := dockerAddress(cfg, "rm", "--force", probeID); e != nil { + t.Fatalf("release address capability probe: %v", e) + } + probeRemoved = true + } + if probeErr == nil { + replacement = candidate + break + } + if !strings.Contains(strings.ToLower(probeErr.Error()), "address already in use") { + t.Fatalf("address fault needs a user configured subnet (tests/compose/reliability.yml), candidate %s: %v", candidate, probeErr) + } + t.Logf("Docker reserves candidate %s; probing another address", candidate) + } + if !replacement.IsValid() { + t.Fatal("no reservable address found within 16 probes of the isolated Compose network") + } + connect := func(ip string) error { + args := []string{"network", "connect", "--ip", ip} + for _, alias := range aliases { + args = append(args, "--alias", alias) + } + _, e := dockerAddress(cfg, append(args, networkID, identity.ID)...) + return e + } + aliasesMatch := func(got []string) bool { + want := slices.Clone(aliases) + got = slices.Clone(got) + slices.Sort(want) + slices.Sort(got) + return slices.Equal(got, want) + } + check := func() { + t.Helper() + for i, b := range before[1:] { + after, e := inspectAddress(cfg, b.ID) + if e != nil || !after.Running || after.StartedAt != b.StartedAt { + t.Errorf("%s process changed during address recovery: %v", services[i+1], e) + } + } + } + // Arm restoration before the first mutation, including uncertain CLI exits. + t.Cleanup(func() { + after, e := inspectAddress(cfg, identity.ID) + if e != nil { + t.Errorf("inspect Identity before network restoration: %v", e) + return + } + if endpoint, connected := after.Networks[network]; connected { + if endpoint.IPAddress == original && aliasesMatch(endpoint.Aliases) { + check() + return + } + if _, e = dockerAddress(cfg, "network", "disconnect", networkID, identity.ID); e != nil { + t.Errorf("disconnect changed Identity address: %v", e) + return + } + } + if e = connect(original); e != nil { + t.Errorf("restore original Identity address: %v", e) + return + } + if _, e = dockerAddress(cfg, "restart", "--time", "1", identity.ID); e != nil { + t.Errorf("restart restored Identity endpoint: %v", e) + } + restored, e := inspectAddress(cfg, identity.ID) + if e != nil || restored.Networks[network].IPAddress != original || !aliasesMatch(restored.Networks[network].Aliases) { + t.Errorf("original Identity address and aliases were not restored: %v", e) + } + t.Log("original Identity address and aliases restored") + check() + }) + if _, err = dockerAddress(cfg, "network", "disconnect", networkID, identity.ID); err != nil { + t.Fatal(err) + } + if err = connect(replacement.String()); err != nil { + t.Fatal(err) + } + // Moving an interface breaks existing outbound sockets as well. Restart only + // the moved service to renew its registry lease and datastore connections; + // otherwise lease expiration would confound the caller DNS recovery assertion. + if _, err = dockerAddress(cfg, "restart", "--time", "1", identity.ID); err != nil { + t.Fatal(err) + } + after, err := inspectAddress(cfg, identity.ID) + if err != nil || after.Networks[network].IPAddress != replacement.String() { + t.Fatalf("Identity did not acquire the replacement address: %v", err) + } + t.Logf("Identity address changed from %s to %s without restarting callers", original, replacement) + return check +} diff --git a/tests/pkg/chaos/address_test.go b/tests/pkg/chaos/address_test.go new file mode 100644 index 0000000..a228783 --- /dev/null +++ b/tests/pkg/chaos/address_test.go @@ -0,0 +1,96 @@ +package chaos + +import ( + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "testing" + + "chatnow-tests/pkg/client" +) + +func TestAddressFaultRejectsUnsupportedNetworkBeforeDisconnect(t *testing.T) { + if os.Getenv("CHATNOW_ADDRESS_PROBE_CHILD") == "1" { + ChangeIdentityAddress(t, &client.Config{Infra: client.InfraConfig{ComposeDir: os.Getenv("CHATNOW_ADDRESS_PROBE_DIR")}}) + return + } + if runtime.GOOS == "windows" { + t.Skip("Docker command-boundary fixture requires the supported Linux test host") + } + for _, mode := range []string{"create", "start", "reserved"} { + t.Run(mode, func(t *testing.T) { + checkUnsupportedAddressNetwork(t, mode) + }) + } +} + +func checkUnsupportedAddressNetwork(t *testing.T, mode string) { + t.Helper() + dir := t.TempDir() + // The real fault controller executes this process boundary. It represents an + // engine that rejects explicit addresses on an automatically allocated subnet. + script := `#!/bin/sh +printf '%s\n' "$*" >> "$CHATNOW_ADDRESS_CALLS" +case "$1 $2" in + "compose ps") echo "$4" ;; + "inspect --format") + printf '{"ID":"%s","Image":"fixture-image","StartedAt":"same","Running":true,"Project":"fixture","Networks":{"fixture_default":{"IPAddress":"172.30.97.20","NetworkID":"fixture-network","Aliases":["identity_server"]}}}\n' "$4" ;; + "network inspect") + echo '[{"Labels":{"com.docker.compose.project":"fixture"},"IPAM":{"Config":[{"Subnet":"172.30.97.0/24","Gateway":"172.30.97.1"}]},"Containers":{}}]' ;; + "network disconnect") exit 0 ;; + "create --network") + if [ "$CHATNOW_ADDRESS_PROBE_MODE" != create ]; then + echo fixture-probe; exit 0 + fi + echo 'user specified IP address requires a user configured subnet' >&2 + exit 1 ;; + "start --attach") + if [ "$CHATNOW_ADDRESS_PROBE_MODE" = reserved ] && [ "$(grep -c '^start ' "$CHATNOW_ADDRESS_CALLS")" = 1 ]; then + echo 'failed to set up container networking: Address already in use' >&2 + exit 1 + fi + echo 'user specified IP address requires a user configured subnet' >&2 + exit 1 ;; + "rm --force") exit 0 ;; + "network connect") + echo 'user specified IP address requires a user configured subnet' >&2 + exit 1 ;; + *) echo 'unexpected Docker command' >&2; exit 2 ;; +esac +` + if err := os.WriteFile(filepath.Join(dir, "docker"), []byte(script), 0755); err != nil { + t.Fatal(err) + } + log := filepath.Join(dir, "calls.log") + cmd := exec.Command(os.Args[0], "-test.run=^TestAddressFaultRejectsUnsupportedNetworkBeforeDisconnect$", "-test.v") + cmd.Env = append(os.Environ(), "CHATNOW_ADDRESS_PROBE_CHILD=1", "CHATNOW_ADDRESS_PROBE_DIR="+dir, + "CHATNOW_ADDRESS_PROBE_MODE="+mode, "CHATNOW_ADDRESS_CALLS="+log, "PATH="+dir+string(os.PathListSeparator)+os.Getenv("PATH")) + out, err := cmd.CombinedOutput() + if err == nil { + t.Fatal("unsupported network must be rejected") + } + if !strings.Contains(string(out), "user configured subnet") { + t.Fatalf("child did not reach the intended network rejection: %s", out) + } + calls, err := os.ReadFile(log) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(calls), "network disconnect") { + t.Fatalf("unsupported network must be rejected before disconnecting Identity; calls:\n%s", calls) + } + if mode == "reserved" { + candidates := make(map[string]bool) + for _, line := range strings.Split(string(calls), "\n") { + args := strings.Fields(line) + if len(args) > 4 && args[0] == "create" { + candidates[args[4]] = true + } + } + if len(candidates) < 2 || strings.Count(string(calls), "rm --force") < 2 { + t.Fatalf("a reserved candidate must be released and replaced before rejecting the network; calls:\n%s", calls) + } + } +} diff --git a/tests/pkg/chaos/lease.go b/tests/pkg/chaos/lease.go new file mode 100644 index 0000000..9de891a --- /dev/null +++ b/tests/pkg/chaos/lease.go @@ -0,0 +1,87 @@ +package chaos + +import ( + "encoding/json" + "fmt" + "strings" + "testing" + "time" + + "chatnow-tests/pkg/client" +) + +const identityRegistryKey = "/service/identity_service/instance" + +// IdentityRegistered reads only the exact synthetic-stack registration key. +func IdentityRegistered(cfg *client.Config) (bool, error) { + out, err := dockerAddress(cfg, "compose", "exec", "-T", "etcd", "etcdctl", "get", identityRegistryKey, "--write-out=json") + if err != nil { + return false, err + } + var response struct { + Count int64 `json:"count"` + } + if err := json.Unmarshal(out, &response); err != nil { + return false, fmt.Errorf("decode scoped registration: %w", err) + } + return response.Count == 1, nil +} + +// ExpireIdentityLease suspends Identity until its registration expires, then +// resumes the same process. Cleanup repairs a failed baseline by restarting only +// Identity after the test's process-preservation assertion has finished. +func ExpireIdentityLease(t testing.TB, cfg *client.Config) func() { + t.Helper() + services := []string{"identity_server", "gateway_server", "transmite_server"} + before := make([]containerAddress, len(services)) + for i, service := range services { + out, err := dockerAddress(cfg, "compose", "ps", "-q", service) + if err != nil || len(strings.Fields(string(out))) != 1 { + t.Fatalf("expected one running %s container: %v", service, err) + } + before[i], err = inspectAddress(cfg, strings.TrimSpace(string(out))) + if err != nil || !before[i].Running || before[i].Project == "" || before[i].Project != before[0].Project { + t.Fatalf("lease fault requires one isolated Compose project: %v", err) + } + } + registered, err := IdentityRegistered(cfg) + if err != nil || !registered { + t.Fatalf("Identity must be registered before lease fault: %v", err) + } + paused := true + resume := func() { + if paused { + compose(t, cfg, "unpause", "identity_server") + paused = false + } + } + t.Cleanup(func() { + resume() + registered, err := IdentityRegistered(cfg) + if err != nil || !registered { + compose(t, cfg, "restart", "identity_server") + } + }) + compose(t, cfg, "pause", "identity_server") + deadline := time.Now().Add(45 * time.Second) + for time.Now().Before(deadline) { + registered, err = IdentityRegistered(cfg) + if err != nil { + t.Fatal(err) + } + if !registered { + resume() + return func() { + for i, service := range services { + after, err := inspectAddress(cfg, before[i].ID) + if err != nil || !after.Running || after.StartedAt != before[i].StartedAt { + t.Fatalf("lease recovery restarted %s: %v", service, err) + } + } + } + } + time.Sleep(500 * time.Millisecond) + } + t.Fatal("Identity registration did not expire while renewal was suspended") + return nil +} diff --git a/tests/pkg/chaos/message.go b/tests/pkg/chaos/message.go new file mode 100644 index 0000000..e90c8db --- /dev/null +++ b/tests/pkg/chaos/message.go @@ -0,0 +1,23 @@ +package chaos + +import ( + "testing" + + "chatnow-tests/pkg/client" +) + +// StopMessage delays persistence and RPC lookup in a disposable Compose stack. +// The returned restore function is idempotent and also runs on assertion failure. +func StopMessage(t testing.TB, cfg *client.Config) func() { + t.Helper() + stopped := true + restore := func() { + if stopped { + compose(t, cfg, "start", "message_server") + stopped = false + } + } + t.Cleanup(restore) + compose(t, cfg, "stop", "--timeout", "5", "message_server") + return restore +} diff --git a/tests/pkg/chaos/redis.go b/tests/pkg/chaos/redis.go index d136862..845bcea 100644 --- a/tests/pkg/chaos/redis.go +++ b/tests/pkg/chaos/redis.go @@ -1,6 +1,7 @@ package chaos import ( + "context" "os/exec" "strings" "testing" @@ -16,7 +17,9 @@ var redisServices = []string{ func compose(t testing.TB, cfg *client.Config, args ...string) { t.Helper() - cmd := exec.Command("docker", append([]string{"compose"}, args...)...) + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "docker", append([]string{"compose"}, args...)...) cmd.Dir = cfg.Infra.ComposeDir if out, err := cmd.CombinedOutput(); err != nil { t.Fatalf("docker compose %v: %v: %s", args, err, out) @@ -31,6 +34,16 @@ func StartRedisCluster(t testing.TB, cfg *client.Config) { compose(t, cfg, append([]string{"start"}, redisServices...)...) } +// Pause preserves the container network and DNS while Redis stops answering. +// This isolates socket timeout/circuit behavior from Docker DNS withdrawal. +func PauseRedisCluster(t testing.TB, cfg *client.Config) { + compose(t, cfg, append([]string{"pause"}, redisServices...)...) +} + +func UnpauseRedisCluster(t testing.TB, cfg *client.Config) { + compose(t, cfg, append([]string{"unpause"}, redisServices...)...) +} + func WaitRedisCluster(t testing.TB, cfg *client.Config, timeout time.Duration) { t.Helper() deadline := time.Now().Add(timeout) diff --git a/tests/pkg/chaos/worker.go b/tests/pkg/chaos/worker.go new file mode 100644 index 0000000..91910c1 --- /dev/null +++ b/tests/pkg/chaos/worker.go @@ -0,0 +1,152 @@ +package chaos + +import ( + "encoding/json" + "fmt" + "strconv" + "strings" + "testing" + "time" + + "chatnow-tests/pkg/client" +) + +// WorkerExit records only process diagnostics from the selected test container. +type WorkerExit struct { + Running, OOMKilled bool + ExitCode int + StartedAt, FinishedAt string + ProtectionLogged bool +} + +// RevokeWorkerLease revokes the sole worker lease in an isolated Compose stack. +// It temporarily disables restart so the original exit status remains observable. +// Cleanup restores the exact restart policy and starts Transmite even on RED. +func RevokeWorkerLease(t testing.TB, cfg *client.Config) (func() (WorkerExit, error), func()) { + t.Helper() + var containers []containerAddress + for _, service := range []string{"transmite_server", "etcd"} { + out, err := dockerAddress(cfg, "compose", "ps", "-q", service) + if err != nil || len(strings.Fields(string(out))) != 1 { + t.Fatalf("expected one running %s: %v", service, err) + } + state, err := inspectAddress(cfg, strings.TrimSpace(string(out))) + if err != nil || !state.Running || state.Project == "" { + t.Fatalf("inspect %s: %v", service, err) + } + containers = append(containers, state) + } + if containers[0].Project != containers[1].Project { + t.Fatal("worker fault requires one isolated Compose project") + } + out, err := dockerAddress(cfg, "compose", "exec", "-T", "etcd", "etcdctl", "get", "/chatnow/snowflake/worker/", "--prefix", "--write-out=json") + var leases struct { + KVs []struct { + Lease int64 `json:"lease"` + } `json:"kvs"` + } + if err != nil || json.Unmarshal(out, &leases) != nil || len(leases.KVs) != 1 || leases.KVs[0].Lease == 0 { + t.Fatalf("expected exactly one leased worker slot: %v", err) + } + id := containers[0].ID + out, err = dockerAddress(cfg, "inspect", "--format", "{{json .HostConfig.RestartPolicy}}", id) + var policy struct { + Name string + MaximumRetryCount int + } + if err != nil || json.Unmarshal(out, &policy) != nil || policy.Name == "" { + t.Fatalf("read worker restart policy: %v", err) + } + restart := policy.Name + if restart == "on-failure" && policy.MaximumRetryCount > 0 { + restart += ":" + strconv.Itoa(policy.MaximumRetryCount) + } + restored := false + restore := func() { + if restored { + return + } + if _, e := dockerAddress(cfg, "update", "--restart="+restart, id); e != nil { + t.Errorf("restore worker restart policy: %v", e) + return + } + if _, e := dockerAddress(cfg, "start", id); e != nil { + t.Errorf("restore worker process: %v", e) + return + } + restored = true + } + t.Cleanup(restore) + if _, err := dockerAddress(cfg, "update", "--restart=no", id); err != nil { + t.Fatal(err) + } + lease := strconv.FormatInt(leases.KVs[0].Lease, 16) + if _, err := dockerAddress(cfg, "compose", "exec", "-T", "etcd", "etcdctl", "lease", "revoke", lease); err != nil { + t.Fatal(err) + } + t.Logf("revoked worker lease=%s original_start=%s", lease, containers[0].StartedAt) + return func() (WorkerExit, error) { + out, err := dockerAddress(cfg, "inspect", "--format", `{"Running":{{.State.Running}},"OOMKilled":{{.State.OOMKilled}},"ExitCode":{{.State.ExitCode}},"StartedAt":{{json .State.StartedAt}},"FinishedAt":{{json .State.FinishedAt}}}`, id) + var state WorkerExit + if err == nil { + err = json.Unmarshal(out, &state) + } + if err == nil && state.StartedAt != containers[0].StartedAt { + err = fmt.Errorf("worker restarted before its exit was recorded") + } + if err == nil && !state.Running { + // Keep raw logs in memory; only expose the fixed diagnostic's presence. + logs, logErr := dockerAddress(cfg, "logs", "--since", state.StartedAt, "--tail", "100", id) + if logErr != nil { + return state, fmt.Errorf("read worker exit diagnostic") + } + state.ProtectionLogged = strings.Contains(string(logs), "worker_lease_lost action=exit code=1") + } + return state, err + }, restore +} + +// ObserveWorkerRecovery rejects a delayed second exit after initial readiness. +// Seven seconds cover the five one-second TTL probes and the watchdog interval. +func ObserveWorkerRecovery(t testing.TB, cfg *client.Config) { + t.Helper() + out, err := dockerAddress(cfg, "compose", "ps", "-q", "transmite_server") + if err != nil || len(strings.Fields(string(out))) != 1 { + t.Fatalf("find recovered worker: %v", err) + } + id := strings.TrimSpace(string(out)) + before, err := inspectAddress(cfg, id) + if err != nil || !before.Running { + t.Fatalf("inspect recovered worker: %v", err) + } + deadline := time.Now().Add(7 * time.Second) + for { + after, err := inspectAddress(cfg, id) + if err != nil || !after.Running || after.StartedAt != before.StartedAt { + t.Fatalf("worker exited again after initial recovery: %v", err) + } + if time.Now().After(deadline) { + return + } + time.Sleep(200 * time.Millisecond) + } +} + +// WaitWorkerExit keeps observing the original process across watchdog cycles. +func WaitWorkerExit(t testing.TB, inspect func() (WorkerExit, error)) WorkerExit { + t.Helper() + deadline := time.Now().Add(20 * time.Second) + for time.Now().Before(deadline) { + state, err := inspect() + if err != nil { + t.Fatal(err) + } + if !state.Running { + t.Logf("worker exit=%+v", state) + return state + } + time.Sleep(200 * time.Millisecond) + } + t.Fatal("worker did not fence its process after confirmed lease loss") + return WorkerExit{} +} diff --git a/tests/pkg/cleanup/cleanup.go b/tests/pkg/cleanup/cleanup.go index 52650bb..4bd58d0 100644 --- a/tests/pkg/cleanup/cleanup.go +++ b/tests/pkg/cleanup/cleanup.go @@ -1,6 +1,7 @@ package cleanup import ( + "bufio" "database/sql" "encoding/base64" "encoding/json" @@ -134,12 +135,15 @@ func flushRedisNode(addr string) error { if err != nil { return err } - resp := make([]byte, 64) - n, err := conn.Read(resp) + s, err := bufio.NewReader(conn).ReadString('\n') if err != nil { return fmt.Errorf("read FLUSHALL response: %w", err) } - s := string(resp[:n]) + // Replica contents converge from the primaries flushed by the same suite. + // Authentication, loading, and other failures still abort cleanup. + if strings.HasPrefix(s, "-READONLY ") { + return nil + } if !strings.HasPrefix(s, "+OK") { return fmt.Errorf("FLUSHALL failed: %s", strings.TrimSpace(s)) } diff --git a/tests/pkg/cleanup/redis_test.go b/tests/pkg/cleanup/redis_test.go new file mode 100644 index 0000000..b4239b6 --- /dev/null +++ b/tests/pkg/cleanup/redis_test.go @@ -0,0 +1,48 @@ +package cleanup + +import ( + "io" + "net" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestFlushRedisNodeHandlesClusterReplicas(t *testing.T) { + for _, tc := range []struct { + name, response string + wantError bool + }{ + {"primary", "+OK\r\n", false}, + {"replica", "-READONLY You can't write against a read only replica.\r\n", false}, + {"unauthorized", "-NOAUTH Authentication required.\r\n", true}, + {"unavailable", "-LOADING Redis is loading the dataset in memory\r\n", true}, + } { + t.Run(tc.name, func(t *testing.T) { + listener, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + t.Cleanup(func() { listener.Close() }) + done := make(chan struct{}) + go func() { + defer close(done) + conn, err := listener.Accept() + if err != nil { + return + } + defer conn.Close() + request := make([]byte, len("*1\r\n$8\r\nFLUSHALL\r\n")) + if _, err := io.ReadFull(conn, request); err != nil { + return + } + _, _ = io.WriteString(conn, tc.response) + }() + err = flushRedisNode(listener.Addr().String()) + if tc.wantError { + require.Error(t, err) + } else { + require.NoError(t, err) + } + <-done + }) + } +} diff --git a/tests/pkg/client/config.go b/tests/pkg/client/config.go index 394e486..beec928 100644 --- a/tests/pkg/client/config.go +++ b/tests/pkg/client/config.go @@ -2,6 +2,7 @@ package client import ( "os" + "path/filepath" "strconv" "gopkg.in/yaml.v3" @@ -56,6 +57,9 @@ func LoadConfig(path string) *Config { if err := yaml.Unmarshal(data, cfg); err != nil { panic("failed to parse config: " + err.Error()) } + if cfg.Infra.ComposeDir != "" && !filepath.IsAbs(cfg.Infra.ComposeDir) { + cfg.Infra.ComposeDir = filepath.Clean(filepath.Join(filepath.Dir(path), cfg.Infra.ComposeDir)) + } // Env overrides for CI if v := os.Getenv("GATEWAY_ADDR"); v != "" { cfg.Target.GatewayAddr = v diff --git a/tests/pkg/client/config_test.go b/tests/pkg/client/config_test.go new file mode 100644 index 0000000..607daff --- /dev/null +++ b/tests/pkg/client/config_test.go @@ -0,0 +1,19 @@ +package client + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestComposeDirectoryIsRelativeToConfigFile(t *testing.T) { + t.Setenv("COMPOSE_DIR", "") + root := t.TempDir() + configDir := filepath.Join(root, "tests") + require.NoError(t, os.Mkdir(configDir, 0700)) + path := filepath.Join(configDir, "config.yaml") + require.NoError(t, os.WriteFile(path, []byte("infra:\n compose_dir: ..\n"), 0600)) + require.Equal(t, root, LoadConfig(path).Infra.ComposeDir) +} diff --git a/tests/pkg/client/diagnostics.go b/tests/pkg/client/diagnostics.go new file mode 100644 index 0000000..37132f7 --- /dev/null +++ b/tests/pkg/client/diagnostics.go @@ -0,0 +1,60 @@ +package client + +import ( + "encoding/json" + "os" + "regexp" + "strconv" + "time" + + common "chatnow-tests/proto/chatnow/common" + "google.golang.org/protobuf/proto" +) + +var rpcErrorCodes = regexp.MustCompile(`\[E([0-9]{1,5})\]`) + +// recordResponseFailure preserves the four header fields as safe diagnostics. +// Free-form server text and request IDs are untrusted and never copied to logs. +func recordResponseFailure(response proto.Message, generatedTrace string) { + provider, ok := response.(interface{ GetHeader() *common.ResponseHeader }) + if !ok { + return + } + header := provider.GetHeader() + if header != nil && header.GetSuccess() { + return + } + classification := "application" + codes := []int{} + text := header.GetErrorMessage() + if len(text) > 16384 { + text = text[:16384] + } + for _, match := range rpcErrorCodes.FindAllStringSubmatch(text, 8) { + value, _ := strconv.Atoi(match[1]) + codes = append(codes, value) + } + if header == nil { + classification = "missing_header" + } else if text == "no backend available" { + classification = "no_backend" + } else if len(codes) > 0 { + classification = "rpc_transport" + } + event := struct { + Event string `json:"event"` + Time string `json:"time"` + Trace string `json:"trace_id,omitempty"` + HeaderPresent bool `json:"header_present"` + RequestIDPresent bool `json:"request_id_present"` + Success bool `json:"success"` + ErrorCode int32 `json:"error_code"` + ErrorMessage string `json:"error_message"` + Classification string `json:"classification"` + RPCCodes []int `json:"rpc_codes"` + }{"protobuf_response_failed", time.Now().UTC().Format(time.RFC3339Nano), generatedTrace, + header != nil, header.GetRequestId() != "", false, header.GetErrorCode(), "omitted", + classification, codes} + // A failed diagnostic write cannot change the request result or trigger a retry. + _ = json.NewEncoder(os.Stderr).Encode(event) +} diff --git a/tests/pkg/client/http.go b/tests/pkg/client/http.go index 167c39a..f558d67 100644 --- a/tests/pkg/client/http.go +++ b/tests/pkg/client/http.go @@ -7,6 +7,7 @@ import ( "fmt" "io" "net/http" + "os" "time" "google.golang.org/protobuf/proto" @@ -68,6 +69,11 @@ func (c *HTTPClient) do(path string, req proto.Message, resp proto.Message, acce } func (c *HTTPClient) doURL(endpoint string, req proto.Message, resp proto.Message, accessToken, traceID string) (http.Header, error) { + diagnosticTrace := "" + if os.Getenv("CHATNOW_TEST_DIAGNOSTICS") == "1" && traceID == "" { + diagnosticTrace = NewRequestID() + traceID = diagnosticTrace + } body, err := proto.Marshal(req) if err != nil { return nil, fmt.Errorf("marshal request: %w", err) @@ -104,6 +110,9 @@ func (c *HTTPClient) doURL(endpoint string, req proto.Message, resp proto.Messag if err := proto.Unmarshal(respBody, resp); err != nil { return headers, fmt.Errorf("unmarshal response: %w", err) } + if os.Getenv("CHATNOW_TEST_DIAGNOSTICS") == "1" { + recordResponseFailure(resp, diagnosticTrace) + } return headers, nil } diff --git a/tests/pkg/client/http_diagnostics_test.go b/tests/pkg/client/http_diagnostics_test.go new file mode 100644 index 0000000..41ad443 --- /dev/null +++ b/tests/pkg/client/http_diagnostics_test.go @@ -0,0 +1,75 @@ +package client + +import ( + "io" + "net/http" + "net/http/httptest" + "os" + "strings" + "testing" + + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/proto" + + common "chatnow-tests/proto/chatnow/common" + msg "chatnow-tests/proto/chatnow/message" +) + +// Client boundary | P0 | Failed responses retain classification without secrets. +func TestHTTPFailureDiagnostics(t *testing.T) { + t.Setenv("CHATNOW_TEST_DIAGNOSTICS", "1") + const secret = "synthetic-private-value" + for _, tc := range []struct { + name string + code int32 + message string + class string + }{ + {"transport", 9002, "[E112]not connected to endpoint: " + secret, "rpc_transport"}, + {"discovery", 9002, "no backend available", "no_backend"}, + {"application", 4001, "mid not found", "application"}, + {"untrusted", 9001, secret + "\nBearer " + secret, "application"}, + } { + t.Run(tc.name, func(t *testing.T) { + calls := 0 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + calls++ + response, err := proto.Marshal(&msg.RecallMessageRsp{Header: &common.ResponseHeader{ + RequestId: secret, ErrorCode: tc.code, ErrorMessage: tc.message, + }}) + require.NoError(t, err) + _, _ = w.Write(response) + })) + t.Cleanup(server.Close) + output, err := os.CreateTemp(t.TempDir(), "diagnostic") + require.NoError(t, err) + original := os.Stderr + os.Stderr = output + t.Cleanup(func() { os.Stderr = original; output.Close() }) + c := NewHTTPClient(&Config{}) + c.baseURL = server.URL + response := &msg.RecallMessageRsp{} + err = c.Do("/service/message/recall", &msg.RecallMessageReq{RequestId: NewRequestID()}, response, secret) + os.Stderr = original + require.NoError(t, err, "diagnostics must preserve application responses") + require.Equal(t, tc.code, response.GetHeader().GetErrorCode()) + require.Equal(t, tc.message, response.GetHeader().GetErrorMessage()) + require.Equal(t, 1, calls, "diagnostics must never retry a failed request") + _, err = output.Seek(0, 0) + require.NoError(t, err) + data, err := io.ReadAll(output) + require.NoError(t, err) + text := string(data) + require.Contains(t, text, `"event":"protobuf_response_failed"`) + require.Contains(t, text, `"classification":"`+tc.class+`"`) + require.Contains(t, text, `"success":false`) + require.Contains(t, text, `"error_code":`) + require.NotContains(t, text, secret) + require.NotContains(t, text, "Bearer") + require.Equal(t, 1, strings.Count(text, "\n"), "one bounded JSON event per failed response") + if tc.name == "transport" { + require.Contains(t, text, `"rpc_codes":[112]`) + } + }) + } +} diff --git a/tests/pkg/client/internal_rpc.go b/tests/pkg/client/internal_rpc.go new file mode 100644 index 0000000..32ad6a1 --- /dev/null +++ b/tests/pkg/client/internal_rpc.go @@ -0,0 +1,150 @@ +package client + +import ( + "bytes" + "encoding/binary" + "fmt" + "io" + "net" + "net/url" + "time" + + "google.golang.org/protobuf/encoding/protowire" + "google.golang.org/protobuf/proto" +) + +// DoInternalRPC makes one uncompressed baidu_std call on a disposable stack. +// It forwards fixture metadata across the existing trusted internal boundary; +// it does not authenticate callers or implement a public client protocol. +// Wire reference: apache/brpc 041cec5, src/brpc/policy/baidu_rpc_meta.proto +// and baidu_rpc_protocol.cpp. One connection owns one correlation ID. +func (c *HTTPClient) DoInternalRPC(endpoint, service, method string, req, resp proto.Message, attachment []byte) error { + u, err := url.Parse(endpoint) + if err != nil || u.Scheme != "http" || u.Host == "" || u.Port() == "" || u.User != nil || + (u.Path != "" && u.Path != "/") || u.RawQuery != "" || u.Fragment != "" { + return fmt.Errorf("internal RPC requires an explicit test endpoint host and port") + } + if service == "" || method == "" { + return fmt.Errorf("internal RPC service and method required") + } + payload, err := proto.Marshal(req) + if err != nil { + return fmt.Errorf("encode RPC request: %w", err) + } + requestMeta := rpcBytes(nil, 1, []byte(service)) + requestMeta = rpcBytes(requestMeta, 2, []byte(method)) + meta := rpcBytes(nil, 1, requestMeta) + meta = rpcUint(meta, 4, 1) + meta = rpcUint(meta, 5, uint64(len(attachment))) + const maxFrame = 1 << 20 + if len(meta)+len(payload)+len(attachment) > maxFrame { + return fmt.Errorf("RPC request frame size exceeds limit") + } + header := make([]byte, 12) + copy(header, "PRPC") + binary.BigEndian.PutUint32(header[4:8], uint32(len(meta)+len(payload)+len(attachment))) + binary.BigEndian.PutUint32(header[8:12], uint32(len(meta))) + frame := append(append(append(header, meta...), payload...), attachment...) + timeout := time.Duration(c.cfg.Timeout.HTTPRequestSec) * time.Second + if timeout <= 0 { + timeout = 10 * time.Second + } + deadline := time.Now().Add(timeout) + conn, err := net.DialTimeout("tcp", u.Host, timeout) + if err != nil { + return fmt.Errorf("connect internal RPC: %w", err) + } + defer conn.Close() + if err := conn.SetDeadline(deadline); err != nil { + return err + } + if _, err := io.Copy(conn, bytes.NewReader(frame)); err != nil { + return fmt.Errorf("write RPC request: %w", err) + } + if _, err := io.ReadFull(conn, header); err != nil { + return fmt.Errorf("read RPC response header: %w", err) + } + size, metaSize := binary.BigEndian.Uint32(header[4:8]), binary.BigEndian.Uint32(header[8:12]) + if string(header[:4]) != "PRPC" || size > maxFrame || metaSize > size { + return fmt.Errorf("invalid RPC response frame size or magic") + } + body := make([]byte, size) + if _, err := io.ReadFull(conn, body); err != nil { + return fmt.Errorf("read RPC response body: %w", err) + } + fields, err := rpcFields(body[:metaSize]) + if err != nil { + return err + } + if fields[4].kind != protowire.VarintType || fields[4].number != 1 { + return fmt.Errorf("RPC response correlation mismatch") + } + for _, number := range []protowire.Number{3, 5} { + if field, exists := fields[number]; exists && field.kind != protowire.VarintType { + return fmt.Errorf("invalid RPC metadata type") + } + } + if fields[3].number != 0 { + return fmt.Errorf("compressed RPC response unsupported") + } + if fields[2].kind != protowire.BytesType { + return fmt.Errorf("RPC response metadata missing") + } + status, err := rpcFields(fields[2].data) + if err != nil { + return err + } + if field, exists := status[1]; exists && field.kind != protowire.VarintType { + return fmt.Errorf("invalid RPC metadata type") + } + if status[1].number != 0 { + return fmt.Errorf("RPC error %d", status[1].number) + } + attached := fields[5].number + if attached > uint64(size-metaSize) { + return fmt.Errorf("invalid RPC response attachment size") + } + if err := proto.Unmarshal(body[metaSize:uint64(size)-attached], resp); err != nil { + return fmt.Errorf("decode RPC response: %w", err) + } + return nil +} + +func rpcBytes(dst []byte, field protowire.Number, value []byte) []byte { + return protowire.AppendBytes(protowire.AppendTag(dst, field, protowire.BytesType), value) +} + +func rpcUint(dst []byte, field protowire.Number, value uint64) []byte { + return protowire.AppendVarint(protowire.AppendTag(dst, field, protowire.VarintType), value) +} + +type rpcField struct { + kind protowire.Type + number uint64 + data []byte +} + +func rpcFields(data []byte) (map[protowire.Number]rpcField, error) { + fields := make(map[protowire.Number]rpcField) + for len(data) > 0 { + num, kind, n := protowire.ConsumeTag(data) + if n < 0 { + return nil, fmt.Errorf("invalid RPC metadata tag") + } + data = data[n:] + field := rpcField{kind: kind} + switch kind { + case protowire.VarintType: + field.number, n = protowire.ConsumeVarint(data) + case protowire.BytesType: + field.data, n = protowire.ConsumeBytes(data) + default: + n = protowire.ConsumeFieldValue(num, kind, data) + } + if n < 0 { + return nil, fmt.Errorf("invalid RPC metadata value") + } + fields[num], data = field, data[n:] + } + return fields, nil +} diff --git a/tests/pkg/client/internal_rpc_test.go b/tests/pkg/client/internal_rpc_test.go new file mode 100644 index 0000000..b0d85cd --- /dev/null +++ b/tests/pkg/client/internal_rpc_test.go @@ -0,0 +1,92 @@ +package client + +import ( + "encoding/binary" + "io" + "net" + "testing" + "time" + + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/types/known/wrapperspb" +) + +// The peer consumes the actual TCP frame. These fixtures are independent wire +// bytes for brpc's pinned baidu_std protocol, not another implementation. +func TestInternalRPCTransport(t *testing.T) { + for _, tc := range []struct { + name string + response []byte + wantError string + }{ + {"success", []byte{0x12, 2, 8, 0, 0x20, 1, 0x0a, 2, 'o', 'k'}, ""}, + {"server_error", []byte{0x12, 2, 8, 5, 0x20, 1}, "RPC error 5"}, + {"wrong_correlation", []byte{0x12, 2, 8, 0, 0x20, 2}, "correlation"}, + {"invalid_status_type", []byte{0x12, 2, 0x0a, 0, 0x20, 1}, "metadata type"}, + {"oversized_body", nil, "frame size"}, + {"truncated_body", []byte{0x12}, "response body"}, + {"stalled_response", nil, "timeout"}, + } { + t.Run(tc.name, func(t *testing.T) { + listener, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + peerDone := make(chan struct{}) + t.Cleanup(func() { _ = listener.Close(); <-peerDone }) + observed := make(chan []byte, 1) + go func() { + defer close(peerDone) + conn, err := listener.Accept() + if err != nil { + observed <- nil + return + } + defer conn.Close() + _ = conn.SetDeadline(time.Now().Add(2 * time.Second)) + header := make([]byte, 12) + if _, err := io.ReadFull(conn, header); err != nil { + observed <- nil + return + } + size := binary.BigEndian.Uint32(header[4:8]) + if size > 1024 { + observed <- nil + return + } + body := make([]byte, size) + _, _ = io.ReadFull(conn, body) + observed <- append(header, body...) + if tc.name == "stalled_response" { + time.Sleep(1500 * time.Millisecond) + return + } + copy(header, "PRPC") + binary.BigEndian.PutUint32(header[4:8], uint32(len(tc.response))) + binary.BigEndian.PutUint32(header[8:12], 6) + if tc.name == "oversized_body" { + binary.BigEndian.PutUint32(header[4:8], 2<<20) + } + if tc.name == "truncated_body" { + binary.BigEndian.PutUint32(header[4:8], 10) + } + _, _ = conn.Write(append(header, tc.response...)) + }() + cfg := &Config{Timeout: TimeoutConfig{HTTPRequestSec: 1}} + c := NewHTTPClient(cfg) + resp := &wrapperspb.StringValue{} + err = c.DoInternalRPC("http://"+listener.Addr().String(), "S", "M", wrapperspb.String("in"), resp, []byte{7, 8}) + if tc.wantError == "" { + require.NoError(t, err) + require.Equal(t, "ok", resp.Value) + } else { + require.ErrorContains(t, err, tc.wantError) + } + select { + case frame := <-observed: + require.Equal(t, []byte{'P', 'R', 'P', 'C', 0, 0, 0, 18, 0, 0, 0, 12, + 0x0a, 6, 0x0a, 1, 'S', 0x12, 1, 'M', 0x20, 1, 0x28, 2, 0x0a, 2, 'i', 'n', 7, 8}, frame) + case <-time.After(2 * time.Second): + t.Fatal("peer did not receive request") + } + }) + } +} diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go index d303fbd..ec8e127 100644 --- a/tests/pkg/contracts/ci_gates_test.go +++ b/tests/pkg/contracts/ci_gates_test.go @@ -202,6 +202,9 @@ func TestCIGates(t *testing.T) { require.Equal(t, "service-artifacts", reliability.Needs) require.Equal(t, "github.event_name == 'pull_request' || github.event_name == 'schedule'", reliability.If) + require.Equal(t, "8", gateEnv(t, reliability, "TRANSMITE_RATE_LIMIT_USER_MAX")) + require.Equal(t, "40", gateEnv(t, reliability, "TRANSMITE_RATE_LIMIT_SESSION_MAX")) + perfCache, ok := workflow.Jobs["perf-cache"] require.True(t, ok, "PF-09 must have a dedicated perf-cache job") require.Equal(t, "service-artifacts", perfCache.Needs) diff --git a/tests/pkg/contracts/compose_runtime_test.go b/tests/pkg/contracts/compose_runtime_test.go new file mode 100644 index 0000000..44ab9ca --- /dev/null +++ b/tests/pkg/contracts/compose_runtime_test.go @@ -0,0 +1,582 @@ +package contracts + +import ( + "encoding/json" + "fmt" + "net" + "net/url" + "os" + "path/filepath" + "regexp" + "sort" + "strings" + "testing" + + "github.com/stretchr/testify/require" + "gopkg.in/yaml.v3" +) + +type composeRuntimeDocument struct { + Services map[string]composeRuntimeService `yaml:"services"` +} + +type composeRuntimeService struct { + Build any `yaml:"build"` + Command any `yaml:"command"` + Entrypoint any `yaml:"entrypoint"` + DependsOn any `yaml:"depends_on"` + Environment any `yaml:"environment"` + Healthcheck any `yaml:"healthcheck"` + Hostname string `yaml:"hostname"` + Ports []any `yaml:"ports"` + Restart any `yaml:"restart"` + Volumes []any `yaml:"volumes"` +} + +// Repository contract | P0 | The root Compose topology includes usable object storage. +func TestComposeRuntimeTopologyContract(t *testing.T) { + root := repositoryRoot(t) + compose := readRuntimeCompose(t, root) + + t.Run("root stack defines MinIO", func(t *testing.T) { + minio, exists := compose.Services["minio"] + require.True(t, exists, "root docker-compose.yml must define minio") + require.NotNil(t, minio.Healthcheck, "minio must expose a healthcheck for its initializer") + }) + + t.Run("MinIO initializer is bounded and one-shot", func(t *testing.T) { + initializer, exists := compose.Services["minio-init"] + require.True(t, exists, "root docker-compose.yml must define minio-init") + require.Equal(t, "no", strings.ToLower(fmt.Sprint(initializer.Restart)), + "minio-init must be a one-shot service") + require.Equal(t, "service_healthy", dependencyCondition(initializer.DependsOn, "minio"), + "minio-init must wait for MinIO health") + initializerText := serviceCommandText(initializer) + require.NotRegexp(t, regexp.MustCompile(`(?i)\btail\s+-f\b`), initializerText, + "minio-init must exit after bucket initialization") + require.Contains(t, fmt.Sprint(initializer.Build), "docker/minio-init", + "the shell-based initializer must use the repository init image, not the shell-less mc image") + dockerfile, err := os.ReadFile(filepath.Join(root, "docker/minio-init/Dockerfile")) + require.NoError(t, err) + require.Regexp(t, regexp.MustCompile(`(?im)^FROM\s+(?:busybox|alpine):[^\s]+\s*$`), string(dockerfile), + "the final MinIO init stage must provide /bin/sh") + require.Regexp(t, regexp.MustCompile(`(?im)^COPY\s+--from=mc\s+/usr/bin/mc\s+/usr/bin/mc\s*$`), string(dockerfile), + "the init image must copy the pinned official mc binary") + entrypoint, err := os.ReadFile(filepath.Join(root, "docker/minio-init/entrypoint.sh")) + require.NoError(t, err) + entrypointSource := string(entrypoint) + require.Contains(t, entrypointSource, "MC_CONFIG_DIR", + "temporary mc state must be isolated and removed") + require.NotRegexp(t, + regexp.MustCompile(`(?m)^\s*mc\s+alias\s+set[^\n]*(?:MINIO_ROOT_PASSWORD|\$\{?MINIO_ROOT_PASSWORD)`), + entrypointSource, "the MinIO root password must not enter process arguments") + require.NotRegexp(t, + regexp.MustCompile(`(?m)^\s*mc\s+admin\s+user\s+add[^\n]*(?:MINIO_APP_SECRET_KEY|\$\{?MINIO_APP_SECRET_KEY)`), + entrypointSource, "the MinIO application secret must not enter process arguments") + }) + + t.Run("Media waits for bucket initialization", func(t *testing.T) { + media, exists := compose.Services["media_server"] + require.True(t, exists, "root docker-compose.yml must define media_server") + require.Equal(t, "service_completed_successfully", + dependencyCondition(media.DependsOn, "minio-init"), + "Media must not start before the one-shot bucket initializer succeeds") + }) + + t.Run("Media uses service DNS for S3", func(t *testing.T) { + media := compose.Services["media_server"] + configSource, readOnly, found := findVolumeMount(media.Volumes, "/im/conf/media.json") + require.True(t, found, "Media must mount its S3 configuration") + configSource = strings.TrimPrefix(filepath.ToSlash(configSource), "./") + mediaConfigBytes, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(configSource))) + require.NoError(t, err) + var mediaConfig struct { + S3 struct { + Endpoint string `json:"endpoint"` + PublicEndpoint string `json:"public_endpoint"` + } `json:"s3"` + } + require.NoError(t, json.Unmarshal(mediaConfigBytes, &mediaConfig)) + endpoint, err := url.Parse(mediaConfig.S3.Endpoint) + require.NoError(t, err) + require.Equal(t, "minio", endpoint.Hostname(), + "Media S3 endpoint must resolve through the root Compose MinIO service") + require.Equal(t, "9000", endpoint.Port()) + require.False(t, isLoopbackHostname(endpoint.Hostname()), + "a container dependency endpoint must not use loopback") + + publicEndpoint, err := url.Parse(mediaConfig.S3.PublicEndpoint) + require.NoError(t, err) + require.True(t, isLoopbackHostname(publicEndpoint.Hostname()), + "local/CI presigned URLs must be reachable from the host client") + require.Equal(t, "19000", publicEndpoint.Port()) + require.NotEqual(t, endpoint.Host, publicEndpoint.Host, + "internal S3 requests and client-facing presigned URLs need distinct endpoints") + + s3Client, err := os.ReadFile(filepath.Join(root, "common/infra/s3_client.hpp")) + require.NoError(t, err) + require.Contains(t, string(s3Client), "public_endpoint", + "S3Client must support a client-facing presign endpoint") + require.Contains(t, string(s3Client), "_presign_client", + "presigning must not reuse the internal service-DNS client") + require.True(t, readOnly, "Media configuration must be mounted read-only") + + conversationConfig, err := os.ReadFile(filepath.Join(root, "conf/docker/conversation_server.conf")) + require.NoError(t, err) + require.Contains(t, string(conversationConfig), + "-public_url_prefix=http://127.0.0.1:19000/chatnow-media-public", + "Conversation-generated public media URLs must use the client-facing MinIO endpoint") + conversationSource, err := os.ReadFile(filepath.Join(root, "conversation/source/conversation_server.cc")) + require.NoError(t, err) + require.Contains(t, string(conversationSource), + `"http://127.0.0.1:19000/chatnow-media-public"`) + }) + + t.Run("published host ports are unique", func(t *testing.T) { + owners := make(map[string]string) + for serviceName, service := range compose.Services { + for _, rawPort := range service.Ports { + hostPort, published := publishedHostPort(rawPort) + if !published { + continue + } + if previous, duplicate := owners[hostPort]; duplicate { + t.Errorf("host port %s is published by both %s and %s", hostPort, previous, serviceName) + continue + } + owners[hostPort] = serviceName + } + } + }) + + t.Run("Redis cluster initialization has no fixed sleep", func(t *testing.T) { + initializer, exists := compose.Services["redis-cluster-init"] + require.True(t, exists, "root docker-compose.yml must define redis-cluster-init") + initializerText := serviceCommandText(initializer) + require.NotRegexp(t, regexp.MustCompile(`(?i)\bsleep\s+[0-9]+(?:s)?\b`), initializerText, + "a fixed sleep is not Redis Cluster readiness evidence") + }) + + t.Run("Redis cluster initializer exits", func(t *testing.T) { + initializer, exists := compose.Services["redis-cluster-init"] + require.True(t, exists, "root docker-compose.yml must define redis-cluster-init") + initializerText := serviceCommandText(initializer) + require.NotRegexp(t, regexp.MustCompile(`(?i)\btail\s+-f\b`), initializerText, + "redis-cluster-init must exit after convergence") + }) + + t.Run("Redis cluster initialization is bounded", func(t *testing.T) { + initializer, exists := compose.Services["redis-cluster-init"] + require.True(t, exists, "root docker-compose.yml must define redis-cluster-init") + initializerText := serviceCommandText(initializer) + initializerScript, err := os.ReadFile(filepath.Join(root, "scripts/init_redis_cluster.sh")) + require.NoError(t, err) + initializerText += "\n" + string(initializerScript) + require.True(t, declaresBoundedDeadline(initializerText), + "redis-cluster-init must declare and enforce a bounded deadline or attempt limit") + }) + + t.Run("Redis Cluster advertises stable service DNS", func(t *testing.T) { + for index := 1; index <= 6; index++ { + serviceName := fmt.Sprintf("redis-node%d", index) + node, exists := compose.Services[serviceName] + require.True(t, exists) + command := serviceCommandText(node) + require.Contains(t, command, "--cluster-announce-hostname "+serviceName) + require.Contains(t, command, "--cluster-preferred-endpoint-type hostname") + } + }) + + t.Run("RabbitMQ persistent node identity is stable", func(t *testing.T) { + rabbit, exists := compose.Services["rabbitmq"] + require.True(t, exists) + require.Equal(t, "rabbitmq", rabbit.Hostname, + "RabbitMQ must keep a stable node identity across container recreation") + nodeName, declared := environmentValue(rabbit.Environment, "RABBITMQ_NODENAME") + require.True(t, declared) + require.Equal(t, "rabbit@rabbitmq", nodeName) + }) + +} + +// Repository contract | P0 | RabbitMQ hosts are not expanded into invalid double-port URLs. +func TestRabbitMQRuntimeAddressContract(t *testing.T) { + root := repositoryRoot(t) + for _, relativePath := range []string{ + "conf/docker/message_server.conf", + "conf/docker/push_server.conf", + "conf/docker/transmite_server.conf", + "conf/local/message_server.conf", + "conf/local/push_server.conf", + "conf/local/transmite_server.conf", + "conf/transmite_server.conf.example", + "message/source/message_server.cc", + "push/source/push_server.cc", + "transmite/source/transmite_server.cc", + } { + content, err := os.ReadFile(filepath.Join(root, relativePath)) + require.NoError(t, err) + mqDefault := regexp.MustCompile(`DEFINE_string\s*\(\s*mq_host\s*,\s*"([^"]+)"`) + for _, line := range strings.Split(string(content), "\n") { + trimmed := strings.TrimSpace(line) + host := "" + if strings.HasPrefix(trimmed, "-mq_host=") { + host = strings.TrimSpace(strings.TrimPrefix(trimmed, "-mq_host=")) + } else if match := mqDefault.FindStringSubmatch(line); len(match) == 2 { + host = match[1] + } + if host == "" { + continue + } + require.NotContains(t, host, ":", + "%s must contain a host only; RabbitMQ builders append port 5672", relativePath) + } + } +} + +// Repository contract | P0 | RabbitMQ credentials cannot change AMQP URI structure. +func TestRabbitMQRuntimeCredentialEncodingContract(t *testing.T) { + root := repositoryRoot(t) + helper, err := os.ReadFile(filepath.Join(root, "common/mq/amqp_url.hpp")) + require.NoError(t, err, "a shared AMQP URI builder must encode credentials") + helperSource := string(helper) + require.Contains(t, helperSource, "percent_encode_userinfo") + require.Contains(t, helperSource, "make_amqp_url") + + for _, relativePath := range []string{ + "message/source/message_server.h", + "push/source/push_server.h", + "transmite/source/transmite_server.h", + } { + content, err := os.ReadFile(filepath.Join(root, relativePath)) + require.NoError(t, err) + require.Contains(t, string(content), "make_amqp_url(", + "%s must not concatenate credentials into an AMQP URI", relativePath) + } +} + +// Repository contract | P0 | A clean MySQL volume creates every current ODB object table. +func TestComposeDatabaseBootstrapContract(t *testing.T) { + root := repositoryRoot(t) + compose := readRuntimeCompose(t, root) + mysql, exists := compose.Services["mysql"] + require.True(t, exists, "root docker-compose.yml must define mysql") + + t.Run("database name is declared", func(t *testing.T) { + databaseName, declared := environmentValue(mysql.Environment, "MYSQL_DATABASE") + require.True(t, declared, "MySQL must declare MYSQL_DATABASE") + require.True(t, databaseName == "chatnow" || strings.HasSuffix(databaseName, ":-chatnow}"), + "MYSQL_DATABASE must resolve to chatnow, got %q", databaseName) + }) + + t.Run("migration mount is read-only", func(t *testing.T) { + initializer, exists := compose.Services["mysql-init"] + require.True(t, exists, "root docker-compose.yml must define mysql-init") + require.Equal(t, "service_healthy", dependencyCondition(initializer.DependsOn, "mysql"), + "mysql-init must wait for MySQL health") + require.Equal(t, "no", strings.ToLower(fmt.Sprint(initializer.Restart)), + "mysql-init must be a one-shot service") + _, readOnly, found := findVolumeMount(initializer.Volumes, "/migrations") + require.True(t, found, "mysql-init must mount the versioned sql directory") + require.True(t, readOnly, "the migration mount must be read-only") + }) + + t.Run("application services wait for schema convergence", func(t *testing.T) { + for _, serviceName := range []string{ + "identity_server", "conversation_server", "relationship_server", + "message_server", "media_server", + } { + service, exists := compose.Services[serviceName] + require.True(t, exists) + require.Equal(t, "service_completed_successfully", + dependencyCondition(service.DependsOn, "mysql-init"), + "%s must not start before schema and grants converge", serviceName) + } + }) + + t.Run("migration runner rejects changed applied versions", func(t *testing.T) { + path := filepath.Join(root, "scripts/init_mysql.sh") + info, err := os.Stat(path) + require.NoError(t, err, "a repeatable MySQL migration runner must exist") + require.NotZero(t, info.Mode().Perm()&0o111, "MySQL migration runner must be executable") + content, err := os.ReadFile(path) + require.NoError(t, err) + source := strings.ToLower(string(content)) + require.Contains(t, source, "schema_migrations") + require.Contains(t, source, "sha256") + require.Contains(t, source, "checksum") + require.Regexp(t, regexp.MustCompile(`(?is)checksum.{0,500}(?:mismatch|changed).{0,500}exit\s+1`), source, + "an applied migration whose checksum changes must fail closed") + }) + + t.Run("migrations cover current ODB object tables idempotently", func(t *testing.T) { + objectTables := odbObjectTables(t, root) + require.NotEmpty(t, objectTables) + migrationTables := migrationTableContracts(t, root) + + var missing []string + var nonIdempotent []string + for _, table := range objectTables { + idempotent, present := migrationTables[table] + if !present { + missing = append(missing, table) + continue + } + if !idempotent { + nonIdempotent = append(nonIdempotent, table) + } + } + require.Empty(t, missing, "versioned migrations are missing current ODB object tables") + require.Empty(t, nonIdempotent, + "cold-start CREATE TABLE statements must use IF NOT EXISTS") + }) +} + +// Repository contract | P0 | CI readiness has a bounded semantic helper. +func TestRuntimeReadinessHelperContract(t *testing.T) { + root := repositoryRoot(t) + path := filepath.Join(root, "scripts/wait_for_services.sh") + info, err := os.Stat(path) + require.NoError(t, err, "scripts/wait_for_services.sh must exist") + require.True(t, info.Mode().IsRegular()) + require.NotZero(t, info.Mode().Perm()&0o111, + "scripts/wait_for_services.sh must be executable") + + content, err := os.ReadFile(path) + require.NoError(t, err) + source := string(content) + require.True(t, declaresBoundedDeadline(source), + "readiness must declare and enforce a bounded deadline or attempt limit") + + probes := map[string]*regexp.Regexp{ + "Redis Cluster state": regexp.MustCompile( + `(?is)redis-cli.{0,300}cluster\s+info.{0,300}cluster_state\s*:\s*ok`), + "MySQL schema": regexp.MustCompile( + `(?is)(?:mysql|mariadb).{0,300}(?:information_schema\.tables|show\s+tables)`), + "RabbitMQ readiness": regexp.MustCompile( + `(?is)(?:rabbitmq-diagnostics.{0,160}(?:ping|check_running)|/api/health/checks/(?:ready-to-serve-clients|alarms|local-alarms))`), + "Elasticsearch health": regexp.MustCompile(`(?is)_cluster/health`), + "MinIO health": regexp.MustCompile(`(?is)/minio/health/(?:ready|live)`), + "public MinIO bucket": regexp.MustCompile( + `(?is)(?:mc|mcli)\s+(?:stat|ls).{0,240}chatnow-media-public`), + "private MinIO bucket": regexp.MustCompile( + `(?is)(?:mc|mcli)\s+(?:stat|ls).{0,240}chatnow-media-private`), + "etcd service registrations": regexp.MustCompile( + `(?is)(?:etcdctl.{0,240}get.{0,240}(?:--prefix.{0,120}/service|/service.{0,120}--prefix)|/v3/kv/range)`), + "Gateway HTTP readiness": regexp.MustCompile( + `(?is)(?:curl|wget).{0,240}(?:gateway|127\.0\.0\.1|localhost).{0,160}/health`), + "Push service readiness": regexp.MustCompile( + `(?is)(?:nc|curl|wget|websocat).{0,240}(?:push|127\.0\.0\.1|localhost).{0,160}(?:9001|health|ready)`), + } + for name, pattern := range probes { + require.True(t, pattern.MatchString(source), "readiness helper must probe %s", name) + } +} + +// Repository contract | P0 | Gateway health is dependency-aware, not liveness-only. +func TestGatewayHealthReadinessContract(t *testing.T) { + root := repositoryRoot(t) + header, err := os.ReadFile(filepath.Join(root, "gateway/source/gateway_server.h")) + require.NoError(t, err) + implementation, err := os.ReadFile(filepath.Join(root, "gateway/source/gateway_server.cc")) + require.NoError(t, err) + source := string(header) + "\n" + string(implementation) + + registration := regexp.MustCompile(`(?s)_http_server\s*\.\s*Get\s*\(\s*"/health"`) + require.True(t, registration.MatchString(source), "Gateway must register GET /health") + + healthOffset := strings.Index(source, `"/health"`) + require.GreaterOrEqual(t, healthOffset, 0) + healthEnd := healthOffset + 3000 + if healthEnd > len(source) { + healthEnd = len(source) + } + healthContract := source[healthOffset:healthEnd] + require.True(t, + regexp.MustCompile(`(?i)(ready|readiness|dependenc|service_manager|_channels|redis|etcd)`).MatchString(healthContract), + "Gateway health must inspect dependencies") + require.True(t, + regexp.MustCompile(`(?i)(status\s*=\s*50[023]|service_unavailable|unavailable)`).MatchString(healthContract), + "Gateway health must return a non-200 result when dependencies are unavailable") +} + +func readRuntimeCompose(t testing.TB, root string) composeRuntimeDocument { + t.Helper() + content, err := os.ReadFile(filepath.Join(root, "docker-compose.yml")) + require.NoError(t, err) + var compose composeRuntimeDocument + require.NoError(t, yaml.Unmarshal(content, &compose), "docker-compose.yml must be valid YAML") + require.NotEmpty(t, compose.Services) + return compose +} + +func serviceCommandText(service composeRuntimeService) string { + return strings.TrimSpace(stringifyYAMLValue(service.Entrypoint) + "\n" + stringifyYAMLValue(service.Command)) +} + +func stringifyYAMLValue(value any) string { + switch typed := value.(type) { + case nil: + return "" + case string: + return typed + case []any: + parts := make([]string, 0, len(typed)) + for _, item := range typed { + parts = append(parts, stringifyYAMLValue(item)) + } + return strings.Join(parts, " ") + default: + return fmt.Sprint(typed) + } +} + +func dependencyCondition(raw any, dependency string) string { + switch typed := raw.(type) { + case []any: + for _, candidate := range typed { + if fmt.Sprint(candidate) == dependency { + return "service_started" + } + } + case map[string]any: + value, exists := typed[dependency] + if !exists { + return "" + } + if details, ok := value.(map[string]any); ok { + return fmt.Sprint(details["condition"]) + } + return "service_started" + } + return "" +} + +func publishedHostPort(raw any) (string, bool) { + if details, ok := raw.(map[string]any); ok { + published := strings.TrimSpace(fmt.Sprint(details["published"])) + return published, published != "" && published != "" + } + short := strings.TrimSpace(fmt.Sprint(raw)) + short = strings.TrimSuffix(strings.TrimSuffix(short, "/tcp"), "/udp") + lastColon := strings.LastIndex(short, ":") + if lastColon < 0 { + return "", false + } + hostSide := short[:lastColon] + if strings.HasPrefix(hostSide, "${") { + return hostSide, true + } + if hostColon := strings.LastIndex(hostSide, ":"); hostColon >= 0 { + hostSide = hostSide[hostColon+1:] + } + hostSide = strings.Trim(hostSide, "[]") + return hostSide, hostSide != "" +} + +func findVolumeMount(rawMounts []any, target string) (source string, readOnly bool, found bool) { + normalizedTarget := strings.TrimRight(target, "/") + for _, raw := range rawMounts { + if details, ok := raw.(map[string]any); ok { + if strings.TrimRight(fmt.Sprint(details["target"]), "/") != normalizedTarget { + continue + } + readOnly, _ = details["read_only"].(bool) + return fmt.Sprint(details["source"]), readOnly, true + } + + short := fmt.Sprint(raw) + parts := strings.Split(short, ":") + if len(parts) < 2 || strings.TrimRight(parts[1], "/") != normalizedTarget { + continue + } + for _, option := range parts[2:] { + if option == "ro" { + readOnly = true + } + } + return parts[0], readOnly, true + } + return "", false, false +} + +func environmentValue(raw any, name string) (string, bool) { + switch typed := raw.(type) { + case map[string]any: + value, exists := typed[name] + if !exists { + return "", false + } + return fmt.Sprint(value), true + case []any: + prefix := name + "=" + for _, item := range typed { + candidate := fmt.Sprint(item) + if strings.HasPrefix(candidate, prefix) { + return strings.TrimPrefix(candidate, prefix), true + } + } + } + return "", false +} + +func odbObjectTables(t testing.TB, root string) []string { + t.Helper() + paths, err := filepath.Glob(filepath.Join(root, "odb/*.hxx")) + require.NoError(t, err) + require.NotEmpty(t, paths) + objectPragma := regexp.MustCompile(`#pragma\s+db\s+object\s+table\("([A-Za-z_][A-Za-z0-9_]*)"\)`) + set := make(map[string]struct{}) + for _, path := range paths { + content, err := os.ReadFile(path) + require.NoError(t, err) + for _, match := range objectPragma.FindAllStringSubmatch(string(content), -1) { + set[match[1]] = struct{}{} + } + } + tables := make([]string, 0, len(set)) + for table := range set { + tables = append(tables, table) + } + sort.Strings(tables) + return tables +} + +func migrationTableContracts(t testing.TB, root string) map[string]bool { + t.Helper() + paths, err := filepath.Glob(filepath.Join(root, "sql/*.sql")) + require.NoError(t, err) + require.NotEmpty(t, paths, "sql/ must contain versioned migrations") + versionedName := regexp.MustCompile(`^V[0-9]+__[A-Za-z0-9_.-]+\.sql$`) + createTable := regexp.MustCompile("(?i)CREATE\\s+TABLE\\s+(IF\\s+NOT\\s+EXISTS\\s+)?(?:\\x60?[A-Za-z_][A-Za-z0-9_]*\\x60?\\.)?\\x60?([A-Za-z_][A-Za-z0-9_]*)\\x60?") + contracts := make(map[string]bool) + for _, path := range paths { + require.Regexp(t, versionedName, filepath.Base(path), + "migration filenames must be versioned") + content, err := os.ReadFile(path) + require.NoError(t, err) + for _, match := range createTable.FindAllStringSubmatch(string(content), -1) { + idempotent := strings.TrimSpace(match[1]) != "" + if previous, exists := contracts[match[2]]; exists { + contracts[match[2]] = previous && idempotent + } else { + contracts[match[2]] = idempotent + } + } + } + return contracts +} + +func declaresBoundedDeadline(source string) bool { + boundDeclaration := regexp.MustCompile(`(?im)\b[A-Z][A-Z0-9_]*(?:TIMEOUT|DEADLINE|MAX_ATTEMPTS|MAX_RETRIES)[A-Z0-9_]*\s*=`) + boundEnforcement := regexp.MustCompile(`(?im)(date\s+\+%s|\bSECONDS\b|\btimeout\b|\battempts?\b|\bretr(?:y|ies)\b)`) + return boundDeclaration.MatchString(source) && boundEnforcement.MatchString(source) +} + +func isLoopbackHostname(host string) bool { + if strings.EqualFold(host, "localhost") { + return true + } + ip := net.ParseIP(host) + return ip != nil && ip.IsLoopback() +} diff --git a/tests/pkg/contracts/diagnostics_test.go b/tests/pkg/contracts/diagnostics_test.go new file mode 100644 index 0000000..ade21f3 --- /dev/null +++ b/tests/pkg/contracts/diagnostics_test.go @@ -0,0 +1,88 @@ +package contracts + +import ( + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/require" + "gopkg.in/yaml.v3" +) + +// Process boundary | P0 | Diagnostic artifacts retain state, never raw logs or secrets. +func TestRuntimeDiagnosticsAreBoundedAndRedacted(t *testing.T) { + const secret = "synthetic-private-value" + directory := t.TempDir() + const docker = `#!/usr/bin/env python3 +import base64,json,os,sys,time +args=sys.argv[1:] +if os.getenv('DIAGNOSTIC_STALL') == '1': time.sleep(4) +if args[0] == 'inspect': + print(json.dumps({'State':{'Status':'running','Running':True,'Paused':False,'Restarting':False,'OOMKilled':False,'ExitCode':0,'Pid':123,'StartedAt':'2026-09-13T00:00:00Z','FinishedAt':'0001-01-01T00:00:00Z','Error':'synthetic-private-value'},'RestartCount':2})) +elif 'ps' in args: print('a'*64) +elif 'etcdctl' in args: + print(json.dumps({'kvs':[{'key':base64.b64encode(b'/service/message_service/instance').decode(),'value':base64.b64encode(b'synthetic-private-value').decode(),'lease':123}]})) +elif 'tail' in args: + print(json.dumps({'ts':'2026-09-13T00:00:00Z','trace_id':'a'*32,'user_id':'synthetic-private-value','device_id':'synthetic-private-value','msg':'Gateway RPC failed err=[112] [E112]not connected synthetic-private-value','fields':{'file':'gateway_server.h','line':'134'}})) + print(json.dumps({'msg':'rpc_failed code=4001 msg=mid not found synthetic-private-value'})) + print(json.dumps({'msg':'password=synthetic-private-value'})) +else: + print('synthetic-private-value',file=sys.stderr) + sys.exit(42) +` + require.NoError(t, os.WriteFile(filepath.Join(directory, "docker"), []byte(docker), 0700)) + for _, stalled := range []bool{false, true} { + t.Run(map[bool]string{false: "snapshot", true: "deadline"}[stalled], func(t *testing.T) { + output := filepath.Join(t.TempDir(), "snapshot.json") + args := []string{filepath.Join(repositoryRoot(t), "scripts/collect_test_diagnostics.py"), "--output", output} + if stalled { + args = append(args, "--timeout-sec", "0.2") + } + cmd := exec.Command("python3", args...) + cmd.Env = append(os.Environ(), "PATH="+directory+string(os.PathListSeparator)+os.Getenv("PATH")) + if stalled { + cmd.Env = append(cmd.Env, "DIAGNOSTIC_STALL=1") + } + started := time.Now() + stdout, err := cmd.CombinedOutput() + require.NoError(t, err, string(stdout)) + data, err := os.ReadFile(output) + require.NoError(t, err) + require.NotContains(t, string(data)+string(stdout), secret) + var snapshot map[string]any + require.NoError(t, json.Unmarshal(data, &snapshot)) + if stalled { + require.Less(t, time.Since(started), 2*time.Second) + require.Contains(t, string(data), "deadline") + } else { + require.Contains(t, string(data), `"restart_count": 2`) + require.Contains(t, string(data), `"registered": true`) + require.Contains(t, string(data), `"rpc_code": 112`) + require.Contains(t, string(data), `"error_code": 4001`) + } + }) + } +} + +func TestCorrectnessGatesPreserveFailureDiagnostics(t *testing.T) { + data, err := os.ReadFile(filepath.Join(repositoryRoot(t), ".github/workflows/ci.yml")) + require.NoError(t, err) + var workflow workflowContract + require.NoError(t, yaml.Unmarshal(data, &workflow)) + for _, name := range []string{"bvt", "func", "reliability"} { + job := workflow.Jobs[name] + before, failure, upload := false, false, false + for _, step := range job.Steps { + if strings.Contains(step.Run, "collect_test_diagnostics.py") { + before = before || (step.If == "" && strings.Contains(step.Run, "before.json")) + failure = failure || (step.If == "failure()" && strings.Contains(step.Run, "failure.json")) + } + upload = upload || (step.If == "failure()" && strings.HasPrefix(step.Uses, "actions/upload-artifact@") && step.With["path"] == "test-diagnostics") + } + require.True(t, before && failure && upload, "%s must retain before/failure state and a sanitized artifact", name) + } +} diff --git a/tests/pkg/contracts/readiness_execution_test.go b/tests/pkg/contracts/readiness_execution_test.go new file mode 100644 index 0000000..eb4f272 --- /dev/null +++ b/tests/pkg/contracts/readiness_execution_test.go @@ -0,0 +1,111 @@ +package contracts + +import ( + "context" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/require" +) + +// Repository contract | P0 | Readiness never puts MinIO credentials in process arguments. +func TestReadinessMinIOCredentialsStayOffCommandLine(t *testing.T) { + directory := t.TempDir() + events := filepath.Join(directory, "events") + environment := filepath.Join(directory, "probe-environment.sh") + // Intercept external commands to examine the real readiness script's process + // boundary. No service, credential store, or network connection is accessed. + const commands = ` +curl() { printf '{"status":"green"}\n'; } +nc() { return 0; } +mc() { + for argument in "$@"; do + if [[ "$argument" == "$MINIO_ROOT_USER" || "$argument" == "$MINIO_ROOT_PASSWORD" ]]; then + printf 'credential argument detected\n' >> "$READINESS_TEST_EVENTS" + return 47 + fi + done + if [[ "$1 $2" == 'alias set' ]]; then + local user password + IFS= read -r user + IFS= read -r password + [[ "$user" == "$MINIO_ROOT_USER" && "$password" == "$MINIO_ROOT_PASSWORD" ]] || return 48 + printf 'credentials received on stdin\n' >> "$READINESS_TEST_EVENTS" + elif [[ "$1" == stat ]]; then + printf 'bucket checked\n' >> "$READINESS_TEST_EVENTS" + fi +} +docker() { + case "$*" in + *'redis-cli -p 6379 cluster info'*) + printf 'cluster_state:ok\ncluster_slots_assigned:16384\ncluster_slots_ok:16384\ncluster_known_nodes:6\n' ;; + *information_schema.tables*) printf '17\n' ;; + *'FROM mysql.user'*) printf '5\n' ;; + *'list_users'*) printf 'chatnow_transmite\t[]\nchatnow_message\t[]\nchatnow_push\t[]\n' ;; + *'list_user_permissions'*) printf '/\tpermissions\n' ;; + *'rabbitmq-diagnostics'*) return 0 ;; + *'--entrypoint /bin/sh minio-init -ec'*) bash -ec "${!#}" ;; + *'etcdctl'*) + for service in identity media transmite message relationship conversation presence push; do + printf '/service/%s_service/instance\n' "$service" + done ;; + *) printf 'unexpected Docker invocation\n' >&2; return 49 ;; + esac +} +` + require.NoError(t, os.WriteFile(environment, []byte(commands), 0600)) + writeDockerBoundary(t, directory) + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) + defer cancel() + command := exec.CommandContext(ctx, "bash", filepath.ToSlash(filepath.Join(repositoryRoot(t), "scripts/wait_for_services.sh"))) + command.Env = append(os.Environ(), + "PATH="+directory+string(os.PathListSeparator)+os.Getenv("PATH"), + "BASH_ENV="+filepath.ToSlash(environment), + "READINESS_TEST_EVENTS="+filepath.ToSlash(events), + "MINIO_ROOT_USER=synthetic-readiness-user", + "MINIO_ROOT_PASSWORD=synthetic-readiness-password", + "CHATNOW_READY_TIMEOUT_SEC=10", + "CHATNOW_READY_POLL_INTERVAL_SEC=0.1", + ) + output, runError := command.CombinedOutput() + observations, err := os.ReadFile(events) + require.NoError(t, err) + require.NotContains(t, string(observations), "credential argument detected") + require.NoError(t, runError, "readiness must complete without credentials in argv: %s", output) + require.Equal(t, 1, strings.Count(string(observations), "credentials received on stdin")) + require.Equal(t, 2, strings.Count(string(observations), "bucket checked")) +} + +// Repository contract | P0 | A stalled Docker probe cannot exceed the readiness deadline. +func TestReadinessDeadlineBoundsStalledProbe(t *testing.T) { + directory := t.TempDir() + environment := filepath.Join(directory, "probe-environment.sh") + require.NoError(t, os.WriteFile(environment, []byte("docker() { sleep 4; return 1; }\n"), 0600)) + writeDockerBoundary(t, directory) + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + command := exec.CommandContext(ctx, "bash", filepath.ToSlash(filepath.Join(repositoryRoot(t), "scripts/wait_for_services.sh"))) + command.Env = append(os.Environ(), + "PATH="+directory+string(os.PathListSeparator)+os.Getenv("PATH"), + "BASH_ENV="+filepath.ToSlash(environment), + "CHATNOW_READY_TIMEOUT_SEC=1", + "CHATNOW_READY_POLL_INTERVAL_SEC=0.1", + ) + started := time.Now() + output, err := command.CombinedOutput() + require.Error(t, err, "a stalled dependency must fail readiness") + require.Less(t, time.Since(started), 3*time.Second, "deadline did not interrupt the stalled probe") + require.Contains(t, string(output), "timed out waiting for Redis Cluster") +} + +func writeDockerBoundary(t testing.TB, directory string) { + t.Helper() + // GNU timeout executes an external command. Its Bash child imports only the + // test's controlled BASH_ENV and dispatches to the same Docker boundary. + require.NoError(t, os.WriteFile(filepath.Join(directory, "docker"), + []byte("#!/usr/bin/env bash\ndocker \"$@\"\n"), 0700)) +} diff --git a/tests/pkg/contracts/workflow_runtime_test.go b/tests/pkg/contracts/workflow_runtime_test.go new file mode 100644 index 0000000..eeea976 --- /dev/null +++ b/tests/pkg/contracts/workflow_runtime_test.go @@ -0,0 +1,71 @@ +package contracts + +import ( + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/require" + "gopkg.in/yaml.v3" +) + +func TestAllWorkflowsParse(t *testing.T) { + files, err := filepath.Glob(filepath.Join(repositoryRoot(t), ".github/workflows/*.yml")) + require.NoError(t, err) + for _, file := range files { + t.Run(filepath.Base(file), func(t *testing.T) { + data, err := os.ReadFile(file) + require.NoError(t, err) + var workflow map[string]any + require.NoError(t, yaml.Unmarshal(data, &workflow)) + }) + } +} + +func TestSyntheticEnvironmentDoesNotOverwriteExistingState(t *testing.T) { + root := t.TempDir() + require.NoError(t, os.Mkdir(filepath.Join(root, "scripts"), 0700)) + for _, relative := range []string{"scripts/create_test_env.py", ".env.example"} { + data, err := os.ReadFile(filepath.Join(repositoryRoot(t), relative)) + require.NoError(t, err) + require.NoError(t, os.WriteFile(filepath.Join(root, relative), data, 0600)) + } + run := func() ([]byte, error) { + return exec.Command("python3", filepath.Join(root, "scripts/create_test_env.py")).CombinedOutput() + } + output, err := run() + require.NoError(t, err, string(output)) + before, err := os.ReadFile(filepath.Join(root, ".env")) + require.NoError(t, err) + for _, line := range strings.Split(strings.TrimSpace(string(before)), "\n") { + parts := strings.SplitN(line, "=", 2) + require.Len(t, parts, 2) + require.NotContains(t, string(output), strings.Trim(parts[1], "'")) + } + _, err = run() + require.Error(t, err, "existing credentials must not be rotated") + after, err := os.ReadFile(filepath.Join(root, ".env")) + require.NoError(t, err) + require.True(t, string(before) == string(after), "existing credentials changed") + require.NoError(t, os.Remove(filepath.Join(root, ".env"))) + require.NoError(t, os.MkdirAll(filepath.Join(root, "middle/data"), 0700)) + _, err = run() + require.Error(t, err, "existing persistent data must not receive new credentials") +} + +func TestRuntimeGatesInitializeSyntheticEnvironment(t *testing.T) { + data, err := os.ReadFile(filepath.Join(repositoryRoot(t), ".github/workflows/ci.yml")) + require.NoError(t, err) + var workflow workflowContract + require.NoError(t, yaml.Unmarshal(data, &workflow)) + for name, job := range workflow.Jobs { + for index, step := range job.Steps { + if strings.TrimSpace(step.Run) == "docker compose up -d --build" { + require.Greater(t, index, 0) + require.Equal(t, "python3 scripts/create_test_env.py --github-env", strings.TrimSpace(job.Steps[index-1].Run), name) + } + } + } +} diff --git a/tests/pkg/fixture/auth.go b/tests/pkg/fixture/auth.go index 005779e..034d862 100644 --- a/tests/pkg/fixture/auth.go +++ b/tests/pkg/fixture/auth.go @@ -1,8 +1,11 @@ package fixture import ( + "encoding/base64" + "encoding/json" "fmt" "math/rand" + "strings" "testing" "chatnow-tests/pkg/client" @@ -37,7 +40,22 @@ func RegisterAndLogin(t testing.TB, c *client.HTTPClient) (*client.HTTPClient, s authed.AccessToken = rsp.Tokens.AccessToken authed.RefreshToken = rsp.Tokens.RefreshToken authed.UserID = rsp.UserId - authed.DeviceID = client.NewDeviceID() + // Registration has no device input. ACKs must use the device Identity issued. + parts := strings.Split(authed.AccessToken, ".") + if len(parts) != 3 { + t.Fatal("registration returned malformed access token") + } + payload, err := base64.RawURLEncoding.DecodeString(parts[1]) + if err != nil { + t.Fatal("registration returned malformed token payload") + } + var claims struct { + DeviceID string `json:"did"` + } + if json.Unmarshal(payload, &claims) != nil || claims.DeviceID == "" { + t.Fatal("registration token has no device identity") + } + authed.DeviceID = claims.DeviceID return authed, username, password } @@ -66,6 +84,6 @@ func LoginUser(t testing.TB, c *client.HTTPClient, username, password string) *c authed.AccessToken = rsp.Tokens.AccessToken authed.RefreshToken = rsp.Tokens.RefreshToken authed.UserID = rsp.UserInfo.UserId - authed.DeviceID = client.NewDeviceID() + authed.DeviceID = req.DeviceId return authed } diff --git a/tests/pkg/fixture/jwt.go b/tests/pkg/fixture/jwt.go new file mode 100644 index 0000000..14bbf52 --- /dev/null +++ b/tests/pkg/fixture/jwt.go @@ -0,0 +1,87 @@ +package fixture + +import ( + "crypto/hmac" + "crypto/rand" + "crypto/sha256" + "encoding/base64" + "encoding/json" + "os" + "strings" + "testing" + "time" +) + +// JWTVariant describes a negative token issued only for an isolated synthetic stack. +type JWTVariant struct { + ExpiresAt time.Time + IssuedAt time.Time + NotBefore time.Time + WrongSigningKey bool + UnknownKeyID bool +} + +// SignJWTVariant preserves a fixture account's claims while changing validity. +// CHATNOW_JWT_CONFIG must be the synthetic configuration used by the test stack. +// Errors deliberately omit all token, key, and decoded claim values. +func SignJWTVariant(t testing.TB, token string, variant JWTVariant) string { + t.Helper() + var config struct { + Auth struct { + JWT struct { + Keys map[string]string `json:"keys"` + } `json:"jwt"` + } `json:"auth"` + } + if json.Unmarshal([]byte(os.Getenv("CHATNOW_JWT_CONFIG")), &config) != nil { + t.Fatal("synthetic CHATNOW_JWT_CONFIG is required for signed JWT boundary tests") + } + parts := strings.Split(token, ".") + if len(parts) != 3 { + t.Fatal("fixture token is malformed") + } + decode := func(part string) map[string]any { + data, err := base64.RawURLEncoding.DecodeString(part) + if err != nil { + t.Fatal("fixture token encoding is malformed") + } + var value map[string]any + decoder := json.NewDecoder(strings.NewReader(string(data))) + decoder.UseNumber() + if decoder.Decode(&value) != nil || value == nil { + t.Fatal("fixture token JSON is malformed") + } + return value + } + header, claims := decode(parts[0]), decode(parts[1]) + kid, ok := header["kid"].(string) + key := []byte(config.Auth.JWT.Keys[kid]) + if !ok || header["alg"] != "HS256" || len(key) < 32 { + t.Fatal("fixture token does not match the synthetic HS256 configuration") + } + if variant.UnknownKeyID { + header["kid"] = "unrecognized-test-key" + } + if variant.WrongSigningKey { + key = make([]byte, 32) + if _, err := rand.Read(key); err != nil { + t.Fatal("cannot create synthetic invalid signing key") + } + } + claims["exp"] = variant.ExpiresAt.Unix() + claims["iat"] = variant.IssuedAt.Unix() + if !variant.NotBefore.IsZero() { + claims["nbf"] = variant.NotBefore.Unix() + } + encode := func(value map[string]any) string { + data, err := json.Marshal(value) + if err != nil { + t.Fatal("cannot encode synthetic JWT variant") + } + return base64.RawURLEncoding.EncodeToString(data) + } + unsigned := encode(header) + "." + encode(claims) + mac := hmac.New(sha256.New, key) + _, _ = mac.Write([]byte(unsigned)) + return unsigned + "." + base64.RawURLEncoding.EncodeToString(mac.Sum(nil)) +} diff --git a/tests/pkg/fixture/media.go b/tests/pkg/fixture/media.go index abca52f..111c80c 100644 --- a/tests/pkg/fixture/media.go +++ b/tests/pkg/fixture/media.go @@ -154,3 +154,10 @@ func UploadLargeFile(t testing.TB, c *client.HTTPClient, content []byte, mime st } return initRsp.FileId } + +// MultipartPDFContent creates unique allowed content with a PDF signature. +func MultipartPDFContent(size int) []byte { + content := bytes.Repeat([]byte(" "), size) + copy(content, []byte("%PDF-1.7\n% "+client.NewRequestID()+"\n")) + return content +} diff --git a/tests/pkg/fixture/message.go b/tests/pkg/fixture/message.go index 8f02881..407e171 100644 --- a/tests/pkg/fixture/message.go +++ b/tests/pkg/fixture/message.go @@ -2,14 +2,18 @@ package fixture import ( "testing" + "time" "chatnow-tests/pkg/client" + "chatnow-tests/pkg/verify" msg "chatnow-tests/proto/chatnow/message" transmite "chatnow-tests/proto/chatnow/transmite" ) -// SendTextMessage 发送文本消息，返回 (message_id, seq_id)。 +// SendTextMessage sends once and waits for the durable message before returning. +// Use SendTextMessageWithClientMsgId when testing broker acceptance itself. func SendTextMessage(t testing.TB, c *client.HTTPClient, convID, text string) (int64, uint64) { + t.Helper() req := &transmite.SendMessageReq{ RequestId: client.NewRequestID(), ConversationId: convID, @@ -29,6 +33,9 @@ func SendTextMessage(t testing.TB, c *client.HTTPClient, convID, text string) (i if rsp.Message == nil { t.Fatal("SendTextMessage: response message is nil") } + db := verify.NewDBVerifier(c.Config().Database.MySQLDSN) + defer db.Close() + db.WaitMessageExists(t, rsp.Message.MessageId, 10*time.Second) return rsp.Message.MessageId, rsp.Message.SeqId } diff --git a/tests/pkg/fixture/setup_test.go b/tests/pkg/fixture/setup_test.go index 9f01901..24fab80 100644 --- a/tests/pkg/fixture/setup_test.go +++ b/tests/pkg/fixture/setup_test.go @@ -14,7 +14,7 @@ import ( var HTTP *client.HTTPClient func TestMain(m *testing.M) { - cfg := client.LoadConfig("") + cfg := client.LoadConfig("../../config.yaml") HTTP = client.NewHTTPClient(cfg) if err := cleanup.WaitForStackReady(cfg, 120*1e9); err != nil { panic(err) diff --git a/tests/pkg/fixture/ws.go b/tests/pkg/fixture/ws.go index dd09460..bd0f4ee 100644 --- a/tests/pkg/fixture/ws.go +++ b/tests/pkg/fixture/ws.go @@ -33,6 +33,7 @@ func ConnectWS(t testing.TB, c *client.HTTPClient) *client.WSClient { if err != nil { t.Fatalf("ConnectWS: %v", err) } + t.Cleanup(func() { ws.Close() }) waitForWSOnline(t, c) return ws } @@ -43,6 +44,7 @@ func ConnectWSWithDeviceID(t testing.TB, c *client.HTTPClient, deviceID string) if err != nil { t.Fatalf("ConnectWSWithDeviceID: %v", err) } + t.Cleanup(func() { ws.Close() }) waitForWSOnline(t, c) return ws } diff --git a/tests/pkg/verify/public_file.go b/tests/pkg/verify/public_file.go new file mode 100644 index 0000000..5ec4f47 --- /dev/null +++ b/tests/pkg/verify/public_file.go @@ -0,0 +1,27 @@ +package verify + +import ( + "bytes" + "io" + "net/http" + "testing" + "time" +) + +// FileURLContentEquals verifies an HTTP GET without additional credentials without logging the URL or bytes. +func FileURLContentEquals(t testing.TB, url string, expected []byte) { + t.Helper() + client := &http.Client{Timeout: 10 * time.Second} + response, err := client.Get(url) + if err != nil { + t.Fatal("file URL request failed") + } + defer response.Body.Close() + if response.StatusCode != http.StatusOK { + t.Fatalf("file URL returned HTTP %d", response.StatusCode) + } + actual, err := io.ReadAll(io.LimitReader(response.Body, int64(len(expected))+1)) + if err != nil || !bytes.Equal(actual, expected) { + t.Fatal("file URL content differs from uploaded bytes") + } +} diff --git a/tests/pkg/verify/redis.go b/tests/pkg/verify/redis.go index e880529..d1139b6 100644 --- a/tests/pkg/verify/redis.go +++ b/tests/pkg/verify/redis.go @@ -41,18 +41,29 @@ func RedisTTL(t testing.TB, key string) time.Duration { func BVar(t testing.TB, baseURL, name string) int64 { t.Helper() - rsp, err := http.Get(fmt.Sprintf("%s/vars/%s", baseURL, name)) + client := &http.Client{Timeout: 10 * time.Second} + rsp, err := client.Get(fmt.Sprintf("%s/vars/%s?console=1", baseURL, name)) if err != nil { t.Fatalf("read bvar %s: %v", name, err) } defer rsp.Body.Close() - body, err := io.ReadAll(rsp.Body) + if rsp.StatusCode != http.StatusOK { + t.Fatalf("read bvar %s: HTTP %d", name, rsp.StatusCode) + } + body, err := io.ReadAll(io.LimitReader(rsp.Body, 65536)) if err != nil { t.Fatalf("read bvar body %s: %v", name, err) } - value, err := strconv.ParseInt(strings.TrimSpace(string(body)), 10, 64) + text := strings.TrimSpace(string(body)) + if label, raw, found := strings.Cut(text, ":"); found { + if strings.TrimSpace(label) != name { + t.Fatalf("unexpected bvar label for %s", name) + } + text = strings.TrimSpace(raw) + } + value, err := strconv.ParseInt(text, 10, 64) if err != nil { - t.Fatalf("parse bvar %s=%q: %v", name, body, err) + t.Fatalf("bvar %s did not return an integer", name) } return value } diff --git a/tests/proto/chatnow/media/media_service.pb.go b/tests/proto/chatnow/media/media_service.pb.go index 00a8f90..0d81853 100644 --- a/tests/proto/chatnow/media/media_service.pb.go +++ b/tests/proto/chatnow/media/media_service.pb.go @@ -1,6 +1,6 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.34.2 +// protoc-gen-go v1.36.11 // protoc v3.12.4 // source: media/media_service.proto @@ -12,6 +12,7 @@ import ( protoimpl "google.golang.org/protobuf/runtime/protoimpl" reflect "reflect" sync "sync" + unsafe "unsafe" ) const ( @@ -77,24 +78,23 @@ func (MediaPurpose) EnumDescriptor() ([]byte, []int) { } type FileInfo struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + FileId string `protobuf:"bytes,1,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` + FileName string `protobuf:"bytes,2,opt,name=file_name,json=fileName,proto3" json:"file_name,omitempty"` + FileSize int64 `protobuf:"varint,3,opt,name=file_size,json=fileSize,proto3" json:"file_size,omitempty"` + MimeType string `protobuf:"bytes,4,opt,name=mime_type,json=mimeType,proto3" json:"mime_type,omitempty"` + UploadedAtMs int64 `protobuf:"varint,5,opt,name=uploaded_at_ms,json=uploadedAtMs,proto3" json:"uploaded_at_ms,omitempty"` + // Canonical unsigned URL for committed public objects; empty for private files. + PublicUrl string `protobuf:"bytes,6,opt,name=public_url,json=publicUrl,proto3" json:"public_url,omitempty"` unknownFields protoimpl.UnknownFields - - FileId string `protobuf:"bytes,1,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` - FileName string `protobuf:"bytes,2,opt,name=file_name,json=fileName,proto3" json:"file_name,omitempty"` - FileSize int64 `protobuf:"varint,3,opt,name=file_size,json=fileSize,proto3" json:"file_size,omitempty"` - MimeType string `protobuf:"bytes,4,opt,name=mime_type,json=mimeType,proto3" json:"mime_type,omitempty"` - UploadedAtMs int64 `protobuf:"varint,5,opt,name=uploaded_at_ms,json=uploadedAtMs,proto3" json:"uploaded_at_ms,omitempty"` + sizeCache protoimpl.SizeCache } func (x *FileInfo) Reset() { *x = FileInfo{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[0] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[0] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *FileInfo) String() string { @@ -105,7 +105,7 @@ func (*FileInfo) ProtoMessage() {} func (x *FileInfo) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[0] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -155,26 +155,30 @@ func (x *FileInfo) GetUploadedAtMs() int64 { return 0 } +func (x *FileInfo) GetPublicUrl() string { + if x != nil { + return x.PublicUrl + } + return "" +} + type ApplyUploadReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + FileName string `protobuf:"bytes,2,opt,name=file_name,json=fileName,proto3" json:"file_name,omitempty"` + FileSize int64 `protobuf:"varint,3,opt,name=file_size,json=fileSize,proto3" json:"file_size,omitempty"` + MimeType string `protobuf:"bytes,4,opt,name=mime_type,json=mimeType,proto3" json:"mime_type,omitempty"` + ContentHash string `protobuf:"bytes,5,opt,name=content_hash,json=contentHash,proto3" json:"content_hash,omitempty"` // "sha256:<64hex>" + Purpose MediaPurpose `protobuf:"varint,6,opt,name=purpose,proto3,enum=chatnow.media.MediaPurpose" json:"purpose,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - FileName string `protobuf:"bytes,2,opt,name=file_name,json=fileName,proto3" json:"file_name,omitempty"` - FileSize int64 `protobuf:"varint,3,opt,name=file_size,json=fileSize,proto3" json:"file_size,omitempty"` - MimeType string `protobuf:"bytes,4,opt,name=mime_type,json=mimeType,proto3" json:"mime_type,omitempty"` - ContentHash string `protobuf:"bytes,5,opt,name=content_hash,json=contentHash,proto3" json:"content_hash,omitempty"` // "sha256:<64hex>" - Purpose MediaPurpose `protobuf:"varint,6,opt,name=purpose,proto3,enum=chatnow.media.MediaPurpose" json:"purpose,omitempty"` + sizeCache protoimpl.SizeCache } func (x *ApplyUploadReq) Reset() { *x = ApplyUploadReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[1] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[1] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *ApplyUploadReq) String() string { @@ -185,7 +189,7 @@ func (*ApplyUploadReq) ProtoMessage() {} func (x *ApplyUploadReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[1] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -243,25 +247,22 @@ func (x *ApplyUploadReq) GetPurpose() MediaPurpose { } type ApplyUploadRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache - unknownFields protoimpl.UnknownFields - + state protoimpl.MessageState `protogen:"open.v1"` Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` - AlreadyExists bool `protobuf:"varint,3,opt,name=already_exists,json=alreadyExists,proto3" json:"already_exists,omitempty"` // true=已去重命中，无需上传 - UploadUrl string `protobuf:"bytes,4,opt,name=upload_url,json=uploadUrl,proto3" json:"upload_url,omitempty"` // presigned PUT URL（dedup 时为空） - Headers map[string]string `protobuf:"bytes,5,rep,name=headers,proto3" json:"headers,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"` // 客户端 PUT 必带的 header + AlreadyExists bool `protobuf:"varint,3,opt,name=already_exists,json=alreadyExists,proto3" json:"already_exists,omitempty"` // true=已去重命中，无需上传 + UploadUrl string `protobuf:"bytes,4,opt,name=upload_url,json=uploadUrl,proto3" json:"upload_url,omitempty"` // presigned PUT URL（dedup 时为空） + Headers map[string]string `protobuf:"bytes,5,rep,name=headers,proto3" json:"headers,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` // 客户端 PUT 必带的 header ExpiresInSec int32 `protobuf:"varint,6,opt,name=expires_in_sec,json=expiresInSec,proto3" json:"expires_in_sec,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *ApplyUploadRsp) Reset() { *x = ApplyUploadRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[2] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[2] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *ApplyUploadRsp) String() string { @@ -272,7 +273,7 @@ func (*ApplyUploadRsp) ProtoMessage() {} func (x *ApplyUploadRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[2] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -330,21 +331,18 @@ func (x *ApplyUploadRsp) GetExpiresInSec() int32 { } type CompleteUploadReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` + sizeCache protoimpl.SizeCache } func (x *CompleteUploadReq) Reset() { *x = CompleteUploadReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[3] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[3] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *CompleteUploadReq) String() string { @@ -355,7 +353,7 @@ func (*CompleteUploadReq) ProtoMessage() {} func (x *CompleteUploadReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[3] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -385,21 +383,18 @@ func (x *CompleteUploadReq) GetFileId() string { } type CompleteUploadRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` + FileInfo *FileInfo `protobuf:"bytes,2,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` unknownFields protoimpl.UnknownFields - - Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` - FileInfo *FileInfo `protobuf:"bytes,2,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` + sizeCache protoimpl.SizeCache } func (x *CompleteUploadRsp) Reset() { *x = CompleteUploadRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[4] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[4] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *CompleteUploadRsp) String() string { @@ -410,7 +405,7 @@ func (*CompleteUploadRsp) ProtoMessage() {} func (x *CompleteUploadRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[4] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -440,25 +435,22 @@ func (x *CompleteUploadRsp) GetFileInfo() *FileInfo { } type InitMultipartReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + FileName string `protobuf:"bytes,2,opt,name=file_name,json=fileName,proto3" json:"file_name,omitempty"` + FileSize int64 `protobuf:"varint,3,opt,name=file_size,json=fileSize,proto3" json:"file_size,omitempty"` + MimeType string `protobuf:"bytes,4,opt,name=mime_type,json=mimeType,proto3" json:"mime_type,omitempty"` + ContentHash string `protobuf:"bytes,5,opt,name=content_hash,json=contentHash,proto3" json:"content_hash,omitempty"` + Purpose MediaPurpose `protobuf:"varint,6,opt,name=purpose,proto3,enum=chatnow.media.MediaPurpose" json:"purpose,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - FileName string `protobuf:"bytes,2,opt,name=file_name,json=fileName,proto3" json:"file_name,omitempty"` - FileSize int64 `protobuf:"varint,3,opt,name=file_size,json=fileSize,proto3" json:"file_size,omitempty"` - MimeType string `protobuf:"bytes,4,opt,name=mime_type,json=mimeType,proto3" json:"mime_type,omitempty"` - ContentHash string `protobuf:"bytes,5,opt,name=content_hash,json=contentHash,proto3" json:"content_hash,omitempty"` - Purpose MediaPurpose `protobuf:"varint,6,opt,name=purpose,proto3,enum=chatnow.media.MediaPurpose" json:"purpose,omitempty"` + sizeCache protoimpl.SizeCache } func (x *InitMultipartReq) Reset() { *x = InitMultipartReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[5] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[5] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *InitMultipartReq) String() string { @@ -469,7 +461,7 @@ func (*InitMultipartReq) ProtoMessage() {} func (x *InitMultipartReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[5] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -527,23 +519,20 @@ func (x *InitMultipartReq) GetPurpose() MediaPurpose { } type InitMultipartRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache - unknownFields protoimpl.UnknownFields - + state protoimpl.MessageState `protogen:"open.v1"` Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` UploadId string `protobuf:"bytes,3,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` RecommendedPartSizeBytes int32 `protobuf:"varint,4,opt,name=recommended_part_size_bytes,json=recommendedPartSizeBytes,proto3" json:"recommended_part_size_bytes,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *InitMultipartRsp) Reset() { *x = InitMultipartRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[6] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[6] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *InitMultipartRsp) String() string { @@ -554,7 +543,7 @@ func (*InitMultipartRsp) ProtoMessage() {} func (x *InitMultipartRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[6] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -598,22 +587,19 @@ func (x *InitMultipartRsp) GetRecommendedPartSizeBytes() int32 { } type ApplyPartReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + UploadId string `protobuf:"bytes,2,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` + PartNumber int32 `protobuf:"varint,3,opt,name=part_number,json=partNumber,proto3" json:"part_number,omitempty"` // 1..10000 unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - UploadId string `protobuf:"bytes,2,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` - PartNumber int32 `protobuf:"varint,3,opt,name=part_number,json=partNumber,proto3" json:"part_number,omitempty"` // 1..10000 + sizeCache protoimpl.SizeCache } func (x *ApplyPartReq) Reset() { *x = ApplyPartReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[7] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[7] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *ApplyPartReq) String() string { @@ -624,7 +610,7 @@ func (*ApplyPartReq) ProtoMessage() {} func (x *ApplyPartReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[7] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -661,22 +647,19 @@ func (x *ApplyPartReq) GetPartNumber() int32 { } type ApplyPartRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` + UploadUrl string `protobuf:"bytes,2,opt,name=upload_url,json=uploadUrl,proto3" json:"upload_url,omitempty"` + ExpiresInSec int32 `protobuf:"varint,3,opt,name=expires_in_sec,json=expiresInSec,proto3" json:"expires_in_sec,omitempty"` unknownFields protoimpl.UnknownFields - - Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` - UploadUrl string `protobuf:"bytes,2,opt,name=upload_url,json=uploadUrl,proto3" json:"upload_url,omitempty"` - ExpiresInSec int32 `protobuf:"varint,3,opt,name=expires_in_sec,json=expiresInSec,proto3" json:"expires_in_sec,omitempty"` + sizeCache protoimpl.SizeCache } func (x *ApplyPartRsp) Reset() { *x = ApplyPartRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[8] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[8] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *ApplyPartRsp) String() string { @@ -687,7 +670,7 @@ func (*ApplyPartRsp) ProtoMessage() {} func (x *ApplyPartRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[8] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -724,21 +707,18 @@ func (x *ApplyPartRsp) GetExpiresInSec() int32 { } type PartETag struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + PartNumber int32 `protobuf:"varint,1,opt,name=part_number,json=partNumber,proto3" json:"part_number,omitempty"` + Etag string `protobuf:"bytes,2,opt,name=etag,proto3" json:"etag,omitempty"` unknownFields protoimpl.UnknownFields - - PartNumber int32 `protobuf:"varint,1,opt,name=part_number,json=partNumber,proto3" json:"part_number,omitempty"` - Etag string `protobuf:"bytes,2,opt,name=etag,proto3" json:"etag,omitempty"` + sizeCache protoimpl.SizeCache } func (x *PartETag) Reset() { *x = PartETag{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[9] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[9] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *PartETag) String() string { @@ -749,7 +729,7 @@ func (*PartETag) ProtoMessage() {} func (x *PartETag) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[9] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -779,22 +759,19 @@ func (x *PartETag) GetEtag() string { } type CompleteMultipartReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + UploadId string `protobuf:"bytes,2,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` + Parts []*PartETag `protobuf:"bytes,3,rep,name=parts,proto3" json:"parts,omitempty"` // 升序 part_number unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - UploadId string `protobuf:"bytes,2,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` - Parts []*PartETag `protobuf:"bytes,3,rep,name=parts,proto3" json:"parts,omitempty"` // 升序 part_number + sizeCache protoimpl.SizeCache } func (x *CompleteMultipartReq) Reset() { *x = CompleteMultipartReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[10] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[10] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *CompleteMultipartReq) String() string { @@ -805,7 +782,7 @@ func (*CompleteMultipartReq) ProtoMessage() {} func (x *CompleteMultipartReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[10] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -842,21 +819,18 @@ func (x *CompleteMultipartReq) GetParts() []*PartETag { } type CompleteMultipartRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` + FileInfo *FileInfo `protobuf:"bytes,2,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` unknownFields protoimpl.UnknownFields - - Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` - FileInfo *FileInfo `protobuf:"bytes,2,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` + sizeCache protoimpl.SizeCache } func (x *CompleteMultipartRsp) Reset() { *x = CompleteMultipartRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[11] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[11] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *CompleteMultipartRsp) String() string { @@ -867,7 +841,7 @@ func (*CompleteMultipartRsp) ProtoMessage() {} func (x *CompleteMultipartRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[11] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -897,21 +871,18 @@ func (x *CompleteMultipartRsp) GetFileInfo() *FileInfo { } type AbortMultipartReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + UploadId string `protobuf:"bytes,2,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - UploadId string `protobuf:"bytes,2,opt,name=upload_id,json=uploadId,proto3" json:"upload_id,omitempty"` + sizeCache protoimpl.SizeCache } func (x *AbortMultipartReq) Reset() { *x = AbortMultipartReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[12] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[12] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *AbortMultipartReq) String() string { @@ -922,7 +893,7 @@ func (*AbortMultipartReq) ProtoMessage() {} func (x *AbortMultipartReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[12] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -952,20 +923,17 @@ func (x *AbortMultipartReq) GetUploadId() string { } type AbortMultipartRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` unknownFields protoimpl.UnknownFields - - Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` + sizeCache protoimpl.SizeCache } func (x *AbortMultipartRsp) Reset() { *x = AbortMultipartRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[13] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[13] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *AbortMultipartRsp) String() string { @@ -976,7 +944,7 @@ func (*AbortMultipartRsp) ProtoMessage() {} func (x *AbortMultipartRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[13] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -999,21 +967,18 @@ func (x *AbortMultipartRsp) GetHeader() *common.ResponseHeader { } type ApplyDownloadReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` + sizeCache protoimpl.SizeCache } func (x *ApplyDownloadReq) Reset() { *x = ApplyDownloadReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[14] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[14] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *ApplyDownloadReq) String() string { @@ -1024,7 +989,7 @@ func (*ApplyDownloadReq) ProtoMessage() {} func (x *ApplyDownloadReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[14] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -1054,23 +1019,20 @@ func (x *ApplyDownloadReq) GetFileId() string { } type ApplyDownloadRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` + DownloadUrl string `protobuf:"bytes,2,opt,name=download_url,json=downloadUrl,proto3" json:"download_url,omitempty"` // presigned GET URL + ExpiresInSec int32 `protobuf:"varint,3,opt,name=expires_in_sec,json=expiresInSec,proto3" json:"expires_in_sec,omitempty"` + FileInfo *FileInfo `protobuf:"bytes,4,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` unknownFields protoimpl.UnknownFields - - Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` - DownloadUrl string `protobuf:"bytes,2,opt,name=download_url,json=downloadUrl,proto3" json:"download_url,omitempty"` // presigned GET URL - ExpiresInSec int32 `protobuf:"varint,3,opt,name=expires_in_sec,json=expiresInSec,proto3" json:"expires_in_sec,omitempty"` - FileInfo *FileInfo `protobuf:"bytes,4,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` + sizeCache protoimpl.SizeCache } func (x *ApplyDownloadRsp) Reset() { *x = ApplyDownloadRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[15] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[15] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *ApplyDownloadRsp) String() string { @@ -1081,7 +1043,7 @@ func (*ApplyDownloadRsp) ProtoMessage() {} func (x *ApplyDownloadRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[15] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -1125,21 +1087,18 @@ func (x *ApplyDownloadRsp) GetFileInfo() *FileInfo { } type GetFileInfoReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - FileId string `protobuf:"bytes,2,opt,name=file_id,json=fileId,proto3" json:"file_id,omitempty"` + sizeCache protoimpl.SizeCache } func (x *GetFileInfoReq) Reset() { *x = GetFileInfoReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[16] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[16] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *GetFileInfoReq) String() string { @@ -1150,7 +1109,7 @@ func (*GetFileInfoReq) ProtoMessage() {} func (x *GetFileInfoReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[16] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -1180,21 +1139,18 @@ func (x *GetFileInfoReq) GetFileId() string { } type GetFileInfoRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` + FileInfo *FileInfo `protobuf:"bytes,2,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` unknownFields protoimpl.UnknownFields - - Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` - FileInfo *FileInfo `protobuf:"bytes,2,opt,name=file_info,json=fileInfo,proto3" json:"file_info,omitempty"` + sizeCache protoimpl.SizeCache } func (x *GetFileInfoRsp) Reset() { *x = GetFileInfoRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[17] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[17] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *GetFileInfoRsp) String() string { @@ -1205,7 +1161,7 @@ func (*GetFileInfoRsp) ProtoMessage() {} func (x *GetFileInfoRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[17] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -1235,21 +1191,18 @@ func (x *GetFileInfoRsp) GetFileInfo() *FileInfo { } type SpeechRecognitionReq struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache + state protoimpl.MessageState `protogen:"open.v1"` + RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` + SpeechContent []byte `protobuf:"bytes,2,opt,name=speech_content,json=speechContent,proto3" json:"speech_content,omitempty"` unknownFields protoimpl.UnknownFields - - RequestId string `protobuf:"bytes,1,opt,name=request_id,json=requestId,proto3" json:"request_id,omitempty"` - SpeechContent []byte `protobuf:"bytes,2,opt,name=speech_content,json=speechContent,proto3" json:"speech_content,omitempty"` + sizeCache protoimpl.SizeCache } func (x *SpeechRecognitionReq) Reset() { *x = SpeechRecognitionReq{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[18] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[18] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *SpeechRecognitionReq) String() string { @@ -1260,7 +1213,7 @@ func (*SpeechRecognitionReq) ProtoMessage() {} func (x *SpeechRecognitionReq) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[18] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -1290,21 +1243,18 @@ func (x *SpeechRecognitionReq) GetSpeechContent() []byte { } type SpeechRecognitionRsp struct { - state protoimpl.MessageState - sizeCache protoimpl.SizeCache - unknownFields protoimpl.UnknownFields - + state protoimpl.MessageState `protogen:"open.v1"` Header *common.ResponseHeader `protobuf:"bytes,1,opt,name=header,proto3" json:"header,omitempty"` RecognitionResult string `protobuf:"bytes,2,opt,name=recognition_result,json=recognitionResult,proto3" json:"recognition_result,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *SpeechRecognitionRsp) Reset() { *x = SpeechRecognitionRsp{} - if protoimpl.UnsafeEnabled { - mi := &file_media_media_service_proto_msgTypes[19] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) - } + mi := &file_media_media_service_proto_msgTypes[19] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) } func (x *SpeechRecognitionRsp) String() string { @@ -1315,7 +1265,7 @@ func (*SpeechRecognitionRsp) ProtoMessage() {} func (x *SpeechRecognitionRsp) ProtoReflect() protoreflect.Message { mi := &file_media_media_service_proto_msgTypes[19] - if protoimpl.UnsafeEnabled && x != nil { + if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) @@ -1346,253 +1296,134 @@ func (x *SpeechRecognitionRsp) GetRecognitionResult() string { var File_media_media_service_proto protoreflect.FileDescriptor -var file_media_media_service_proto_rawDesc = []byte{ - 0x0a, 0x19, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2f, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x5f, 0x73, 0x65, - 0x72, 0x76, 0x69, 0x63, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x0d, 0x63, 0x68, 0x61, - 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x1a, 0x15, 0x63, 0x6f, 0x6d, 0x6d, - 0x6f, 0x6e, 0x2f, 0x65, 0x6e, 0x76, 0x65, 0x6c, 0x6f, 0x70, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, - 0x6f, 0x22, 0xa0, 0x01, 0x0a, 0x08, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x17, - 0x0a, 0x07, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x06, 0x66, 0x69, 0x6c, 0x65, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, - 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, - 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x73, 0x69, 0x7a, - 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x03, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x53, 0x69, 0x7a, - 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x6d, 0x69, 0x6d, 0x65, 0x5f, 0x74, 0x79, 0x70, 0x65, 0x18, 0x04, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x6d, 0x69, 0x6d, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x24, - 0x0a, 0x0e, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x65, 0x64, 0x5f, 0x61, 0x74, 0x5f, 0x6d, 0x73, - 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0c, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x65, 0x64, - 0x41, 0x74, 0x4d, 0x73, 0x22, 0xe0, 0x01, 0x0a, 0x0e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x55, 0x70, - 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x65, 0x71, 0x12, 0x1d, 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, - 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, - 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x6e, - 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x4e, - 0x61, 0x6d, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x73, 0x69, 0x7a, 0x65, - 0x18, 0x03, 0x20, 0x01, 0x28, 0x03, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x53, 0x69, 0x7a, 0x65, - 0x12, 0x1b, 0x0a, 0x09, 0x6d, 0x69, 0x6d, 0x65, 0x5f, 0x74, 0x79, 0x70, 0x65, 0x18, 0x04, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x08, 0x6d, 0x69, 0x6d, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x21, 0x0a, - 0x0c, 0x63, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x5f, 0x68, 0x61, 0x73, 0x68, 0x18, 0x05, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x0b, 0x63, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x48, 0x61, 0x73, 0x68, - 0x12, 0x35, 0x0a, 0x07, 0x70, 0x75, 0x72, 0x70, 0x6f, 0x73, 0x65, 0x18, 0x06, 0x20, 0x01, 0x28, - 0x0e, 0x32, 0x1b, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, - 0x61, 0x2e, 0x4d, 0x65, 0x64, 0x69, 0x61, 0x50, 0x75, 0x72, 0x70, 0x6f, 0x73, 0x65, 0x52, 0x07, - 0x70, 0x75, 0x72, 0x70, 0x6f, 0x73, 0x65, 0x22, 0xcf, 0x02, 0x0a, 0x0e, 0x41, 0x70, 0x70, 0x6c, - 0x79, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, 0x70, 0x12, 0x36, 0x0a, 0x06, 0x68, 0x65, - 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, 0x2e, 0x63, 0x68, 0x61, - 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, 0x68, 0x65, 0x61, 0x64, - 0x65, 0x72, 0x12, 0x17, 0x0a, 0x07, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x06, 0x66, 0x69, 0x6c, 0x65, 0x49, 0x64, 0x12, 0x25, 0x0a, 0x0e, 0x61, - 0x6c, 0x72, 0x65, 0x61, 0x64, 0x79, 0x5f, 0x65, 0x78, 0x69, 0x73, 0x74, 0x73, 0x18, 0x03, 0x20, - 0x01, 0x28, 0x08, 0x52, 0x0d, 0x61, 0x6c, 0x72, 0x65, 0x61, 0x64, 0x79, 0x45, 0x78, 0x69, 0x73, - 0x74, 0x73, 0x12, 0x1d, 0x0a, 0x0a, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x5f, 0x75, 0x72, 0x6c, - 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x55, 0x72, - 0x6c, 0x12, 0x44, 0x0a, 0x07, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x73, 0x18, 0x05, 0x20, 0x03, - 0x28, 0x0b, 0x32, 0x2a, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, - 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, - 0x70, 0x2e, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x07, - 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x73, 0x12, 0x24, 0x0a, 0x0e, 0x65, 0x78, 0x70, 0x69, 0x72, - 0x65, 0x73, 0x5f, 0x69, 0x6e, 0x5f, 0x73, 0x65, 0x63, 0x18, 0x06, 0x20, 0x01, 0x28, 0x05, 0x52, - 0x0c, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x49, 0x6e, 0x53, 0x65, 0x63, 0x1a, 0x3a, 0x0a, - 0x0c, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, - 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, - 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, - 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, 0x4b, 0x0a, 0x11, 0x43, 0x6f, 0x6d, - 0x70, 0x6c, 0x65, 0x74, 0x65, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x65, 0x71, 0x12, 0x1d, - 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, - 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, 0x17, 0x0a, - 0x07, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, - 0x66, 0x69, 0x6c, 0x65, 0x49, 0x64, 0x22, 0x81, 0x01, 0x0a, 0x11, 0x43, 0x6f, 0x6d, 0x70, 0x6c, - 0x65, 0x74, 0x65, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, 0x70, 0x12, 0x36, 0x0a, 0x06, - 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, 0x2e, 0x63, - 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x52, 0x65, - 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, 0x68, 0x65, - 0x61, 0x64, 0x65, 0x72, 0x12, 0x34, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x6e, 0x66, - 0x6f, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, - 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, - 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x22, 0xe2, 0x01, 0x0a, 0x10, 0x49, - 0x6e, 0x69, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x52, 0x65, 0x71, 0x12, - 0x1d, 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, 0x1b, - 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, - 0x09, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x66, - 0x69, 0x6c, 0x65, 0x5f, 0x73, 0x69, 0x7a, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x03, 0x52, 0x08, - 0x66, 0x69, 0x6c, 0x65, 0x53, 0x69, 0x7a, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x6d, 0x69, 0x6d, 0x65, - 0x5f, 0x74, 0x79, 0x70, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x6d, 0x69, 0x6d, - 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x63, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, - 0x5f, 0x68, 0x61, 0x73, 0x68, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x63, 0x6f, 0x6e, - 0x74, 0x65, 0x6e, 0x74, 0x48, 0x61, 0x73, 0x68, 0x12, 0x35, 0x0a, 0x07, 0x70, 0x75, 0x72, 0x70, - 0x6f, 0x73, 0x65, 0x18, 0x06, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x1b, 0x2e, 0x63, 0x68, 0x61, 0x74, - 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x4d, 0x65, 0x64, 0x69, 0x61, 0x50, - 0x75, 0x72, 0x70, 0x6f, 0x73, 0x65, 0x52, 0x07, 0x70, 0x75, 0x72, 0x70, 0x6f, 0x73, 0x65, 0x22, - 0xbf, 0x01, 0x0a, 0x10, 0x49, 0x6e, 0x69, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, - 0x74, 0x52, 0x73, 0x70, 0x12, 0x36, 0x0a, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, - 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, - 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, - 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x12, 0x17, 0x0a, 0x07, - 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x66, - 0x69, 0x6c, 0x65, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x5f, - 0x69, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, - 0x49, 0x64, 0x12, 0x3d, 0x0a, 0x1b, 0x72, 0x65, 0x63, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x64, 0x65, - 0x64, 0x5f, 0x70, 0x61, 0x72, 0x74, 0x5f, 0x73, 0x69, 0x7a, 0x65, 0x5f, 0x62, 0x79, 0x74, 0x65, - 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x05, 0x52, 0x18, 0x72, 0x65, 0x63, 0x6f, 0x6d, 0x6d, 0x65, - 0x6e, 0x64, 0x65, 0x64, 0x50, 0x61, 0x72, 0x74, 0x53, 0x69, 0x7a, 0x65, 0x42, 0x79, 0x74, 0x65, - 0x73, 0x22, 0x6b, 0x0a, 0x0c, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x50, 0x61, 0x72, 0x74, 0x52, 0x65, - 0x71, 0x12, 0x1d, 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, - 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, - 0x12, 0x1b, 0x0a, 0x09, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x08, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x49, 0x64, 0x12, 0x1f, 0x0a, - 0x0b, 0x70, 0x61, 0x72, 0x74, 0x5f, 0x6e, 0x75, 0x6d, 0x62, 0x65, 0x72, 0x18, 0x03, 0x20, 0x01, - 0x28, 0x05, 0x52, 0x0a, 0x70, 0x61, 0x72, 0x74, 0x4e, 0x75, 0x6d, 0x62, 0x65, 0x72, 0x22, 0x8b, - 0x01, 0x0a, 0x0c, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x50, 0x61, 0x72, 0x74, 0x52, 0x73, 0x70, 0x12, - 0x36, 0x0a, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, - 0x1e, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, - 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, - 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x12, 0x1d, 0x0a, 0x0a, 0x75, 0x70, 0x6c, 0x6f, 0x61, - 0x64, 0x5f, 0x75, 0x72, 0x6c, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x75, 0x70, 0x6c, - 0x6f, 0x61, 0x64, 0x55, 0x72, 0x6c, 0x12, 0x24, 0x0a, 0x0e, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, - 0x73, 0x5f, 0x69, 0x6e, 0x5f, 0x73, 0x65, 0x63, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0c, - 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x49, 0x6e, 0x53, 0x65, 0x63, 0x22, 0x3f, 0x0a, 0x08, - 0x50, 0x61, 0x72, 0x74, 0x45, 0x54, 0x61, 0x67, 0x12, 0x1f, 0x0a, 0x0b, 0x70, 0x61, 0x72, 0x74, - 0x5f, 0x6e, 0x75, 0x6d, 0x62, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0a, 0x70, - 0x61, 0x72, 0x74, 0x4e, 0x75, 0x6d, 0x62, 0x65, 0x72, 0x12, 0x12, 0x0a, 0x04, 0x65, 0x74, 0x61, - 0x67, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x65, 0x74, 0x61, 0x67, 0x22, 0x81, 0x01, - 0x0a, 0x14, 0x43, 0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, - 0x61, 0x72, 0x74, 0x52, 0x65, 0x71, 0x12, 0x1d, 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, - 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x5f, - 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, - 0x49, 0x64, 0x12, 0x2d, 0x0a, 0x05, 0x70, 0x61, 0x72, 0x74, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, - 0x0b, 0x32, 0x17, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, - 0x61, 0x2e, 0x50, 0x61, 0x72, 0x74, 0x45, 0x54, 0x61, 0x67, 0x52, 0x05, 0x70, 0x61, 0x72, 0x74, - 0x73, 0x22, 0x84, 0x01, 0x0a, 0x14, 0x43, 0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x4d, 0x75, - 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x52, 0x73, 0x70, 0x12, 0x36, 0x0a, 0x06, 0x68, 0x65, - 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, 0x2e, 0x63, 0x68, 0x61, - 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, 0x68, 0x65, 0x61, 0x64, - 0x65, 0x72, 0x12, 0x34, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x6e, 0x66, 0x6f, 0x18, - 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, - 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x08, - 0x66, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x22, 0x4f, 0x0a, 0x11, 0x41, 0x62, 0x6f, 0x72, - 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x52, 0x65, 0x71, 0x12, 0x1d, 0x0a, - 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, - 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, - 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x08, 0x75, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x49, 0x64, 0x22, 0x4b, 0x0a, 0x11, 0x41, 0x62, 0x6f, - 0x72, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x52, 0x73, 0x70, 0x12, 0x36, - 0x0a, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, - 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, - 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, - 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x22, 0x4a, 0x0a, 0x10, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x44, - 0x6f, 0x77, 0x6e, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x65, 0x71, 0x12, 0x1d, 0x0a, 0x0a, 0x72, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, - 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, 0x17, 0x0a, 0x07, 0x66, 0x69, 0x6c, - 0x65, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x66, 0x69, 0x6c, 0x65, - 0x49, 0x64, 0x22, 0xc9, 0x01, 0x0a, 0x10, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x44, 0x6f, 0x77, 0x6e, - 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, 0x70, 0x12, 0x36, 0x0a, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, - 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, - 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, - 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x12, - 0x21, 0x0a, 0x0c, 0x64, 0x6f, 0x77, 0x6e, 0x6c, 0x6f, 0x61, 0x64, 0x5f, 0x75, 0x72, 0x6c, 0x18, - 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x64, 0x6f, 0x77, 0x6e, 0x6c, 0x6f, 0x61, 0x64, 0x55, - 0x72, 0x6c, 0x12, 0x24, 0x0a, 0x0e, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x5f, 0x69, 0x6e, - 0x5f, 0x73, 0x65, 0x63, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0c, 0x65, 0x78, 0x70, 0x69, - 0x72, 0x65, 0x73, 0x49, 0x6e, 0x53, 0x65, 0x63, 0x12, 0x34, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, - 0x5f, 0x69, 0x6e, 0x66, 0x6f, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x63, 0x68, - 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x46, 0x69, 0x6c, 0x65, - 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x22, 0x48, - 0x0a, 0x0e, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x65, 0x71, - 0x12, 0x1d, 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, - 0x17, 0x0a, 0x07, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x06, 0x66, 0x69, 0x6c, 0x65, 0x49, 0x64, 0x22, 0x7e, 0x0a, 0x0e, 0x47, 0x65, 0x74, 0x46, - 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x73, 0x70, 0x12, 0x36, 0x0a, 0x06, 0x68, 0x65, - 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, 0x2e, 0x63, 0x68, 0x61, - 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, 0x68, 0x65, 0x61, 0x64, - 0x65, 0x72, 0x12, 0x34, 0x0a, 0x09, 0x66, 0x69, 0x6c, 0x65, 0x5f, 0x69, 0x6e, 0x66, 0x6f, 0x18, - 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, - 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x08, - 0x66, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x22, 0x5c, 0x0a, 0x14, 0x53, 0x70, 0x65, 0x65, - 0x63, 0x68, 0x52, 0x65, 0x63, 0x6f, 0x67, 0x6e, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, - 0x12, 0x1d, 0x0a, 0x0a, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x12, - 0x25, 0x0a, 0x0e, 0x73, 0x70, 0x65, 0x65, 0x63, 0x68, 0x5f, 0x63, 0x6f, 0x6e, 0x74, 0x65, 0x6e, - 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0d, 0x73, 0x70, 0x65, 0x65, 0x63, 0x68, 0x43, - 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x22, 0x7d, 0x0a, 0x14, 0x53, 0x70, 0x65, 0x65, 0x63, 0x68, - 0x52, 0x65, 0x63, 0x6f, 0x67, 0x6e, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x73, 0x70, 0x12, 0x36, - 0x0a, 0x06, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1e, - 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, - 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x65, 0x61, 0x64, 0x65, 0x72, 0x52, 0x06, - 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x12, 0x2d, 0x0a, 0x12, 0x72, 0x65, 0x63, 0x6f, 0x67, 0x6e, - 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x72, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x18, 0x02, 0x20, 0x01, - 0x28, 0x09, 0x52, 0x11, 0x72, 0x65, 0x63, 0x6f, 0x67, 0x6e, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x52, - 0x65, 0x73, 0x75, 0x6c, 0x74, 0x2a, 0x62, 0x0a, 0x0c, 0x4d, 0x65, 0x64, 0x69, 0x61, 0x50, 0x75, - 0x72, 0x70, 0x6f, 0x73, 0x65, 0x12, 0x1d, 0x0a, 0x19, 0x4d, 0x45, 0x44, 0x49, 0x41, 0x5f, 0x50, - 0x55, 0x52, 0x50, 0x4f, 0x53, 0x45, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, - 0x45, 0x44, 0x10, 0x00, 0x12, 0x0a, 0x0a, 0x06, 0x41, 0x56, 0x41, 0x54, 0x41, 0x52, 0x10, 0x01, - 0x12, 0x10, 0x0a, 0x0c, 0x47, 0x52, 0x4f, 0x55, 0x50, 0x5f, 0x41, 0x56, 0x41, 0x54, 0x41, 0x52, - 0x10, 0x02, 0x12, 0x08, 0x0a, 0x04, 0x43, 0x48, 0x41, 0x54, 0x10, 0x03, 0x12, 0x0b, 0x0a, 0x07, - 0x53, 0x54, 0x49, 0x43, 0x4b, 0x45, 0x52, 0x10, 0x04, 0x32, 0x97, 0x06, 0x0a, 0x0c, 0x4d, 0x65, - 0x64, 0x69, 0x61, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x4b, 0x0a, 0x0b, 0x41, 0x70, - 0x70, 0x6c, 0x79, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x12, 0x1d, 0x2e, 0x63, 0x68, 0x61, 0x74, - 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x55, - 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x65, 0x71, 0x1a, 0x1d, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, - 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x55, 0x70, - 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, 0x70, 0x12, 0x54, 0x0a, 0x0e, 0x43, 0x6f, 0x6d, 0x70, 0x6c, - 0x65, 0x74, 0x65, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x12, 0x20, 0x2e, 0x63, 0x68, 0x61, 0x74, - 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x43, 0x6f, 0x6d, 0x70, 0x6c, 0x65, - 0x74, 0x65, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x65, 0x71, 0x1a, 0x20, 0x2e, 0x63, 0x68, - 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x43, 0x6f, 0x6d, 0x70, - 0x6c, 0x65, 0x74, 0x65, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, 0x70, 0x12, 0x57, 0x0a, - 0x13, 0x49, 0x6e, 0x69, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x55, 0x70, - 0x6c, 0x6f, 0x61, 0x64, 0x12, 0x1f, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, - 0x65, 0x64, 0x69, 0x61, 0x2e, 0x49, 0x6e, 0x69, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, - 0x72, 0x74, 0x52, 0x65, 0x71, 0x1a, 0x1f, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, - 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x49, 0x6e, 0x69, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, - 0x61, 0x72, 0x74, 0x52, 0x73, 0x70, 0x12, 0x4b, 0x0a, 0x0f, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x50, - 0x61, 0x72, 0x74, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x12, 0x1b, 0x2e, 0x63, 0x68, 0x61, 0x74, - 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x50, - 0x61, 0x72, 0x74, 0x52, 0x65, 0x71, 0x1a, 0x1b, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, - 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x50, 0x61, 0x72, 0x74, - 0x52, 0x73, 0x70, 0x12, 0x63, 0x0a, 0x17, 0x43, 0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x4d, - 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, 0x12, 0x23, - 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x43, - 0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, - 0x52, 0x65, 0x71, 0x1a, 0x23, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, - 0x64, 0x69, 0x61, 0x2e, 0x43, 0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x4d, 0x75, 0x6c, 0x74, - 0x69, 0x70, 0x61, 0x72, 0x74, 0x52, 0x73, 0x70, 0x12, 0x5a, 0x0a, 0x14, 0x41, 0x62, 0x6f, 0x72, - 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x55, 0x70, 0x6c, 0x6f, 0x61, 0x64, - 0x12, 0x20, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, - 0x2e, 0x41, 0x62, 0x6f, 0x72, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, 0x74, 0x52, - 0x65, 0x71, 0x1a, 0x20, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, - 0x69, 0x61, 0x2e, 0x41, 0x62, 0x6f, 0x72, 0x74, 0x4d, 0x75, 0x6c, 0x74, 0x69, 0x70, 0x61, 0x72, - 0x74, 0x52, 0x73, 0x70, 0x12, 0x51, 0x0a, 0x0d, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x44, 0x6f, 0x77, - 0x6e, 0x6c, 0x6f, 0x61, 0x64, 0x12, 0x1f, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, - 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x44, 0x6f, 0x77, 0x6e, 0x6c, - 0x6f, 0x61, 0x64, 0x52, 0x65, 0x71, 0x1a, 0x1f, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, - 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x44, 0x6f, 0x77, 0x6e, - 0x6c, 0x6f, 0x61, 0x64, 0x52, 0x73, 0x70, 0x12, 0x4b, 0x0a, 0x0b, 0x47, 0x65, 0x74, 0x46, 0x69, - 0x6c, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x1d, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, - 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, - 0x66, 0x6f, 0x52, 0x65, 0x71, 0x1a, 0x1d, 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, - 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x49, 0x6e, 0x66, - 0x6f, 0x52, 0x73, 0x70, 0x12, 0x5d, 0x0a, 0x11, 0x53, 0x70, 0x65, 0x65, 0x63, 0x68, 0x52, 0x65, - 0x63, 0x6f, 0x67, 0x6e, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x23, 0x2e, 0x63, 0x68, 0x61, 0x74, - 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x53, 0x70, 0x65, 0x65, 0x63, 0x68, - 0x52, 0x65, 0x63, 0x6f, 0x67, 0x6e, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x1a, 0x23, - 0x2e, 0x63, 0x68, 0x61, 0x74, 0x6e, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x2e, 0x53, - 0x70, 0x65, 0x65, 0x63, 0x68, 0x52, 0x65, 0x63, 0x6f, 0x67, 0x6e, 0x69, 0x74, 0x69, 0x6f, 0x6e, - 0x52, 0x73, 0x70, 0x42, 0x03, 0x80, 0x01, 0x01, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, -} +const file_media_media_service_proto_rawDesc = "" + + "\n" + + "\x19media/media_service.proto\x12\rchatnow.media\x1a\x15common/envelope.proto\"\xbf\x01\n" + + "\bFileInfo\x12\x17\n" + + "\afile_id\x18\x01 \x01(\tR\x06fileId\x12\x1b\n" + + "\tfile_name\x18\x02 \x01(\tR\bfileName\x12\x1b\n" + + "\tfile_size\x18\x03 \x01(\x03R\bfileSize\x12\x1b\n" + + "\tmime_type\x18\x04 \x01(\tR\bmimeType\x12$\n" + + "\x0euploaded_at_ms\x18\x05 \x01(\x03R\fuploadedAtMs\x12\x1d\n" + + "\n" + + "public_url\x18\x06 \x01(\tR\tpublicUrl\"\xe0\x01\n" + + "\x0eApplyUploadReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x1b\n" + + "\tfile_name\x18\x02 \x01(\tR\bfileName\x12\x1b\n" + + "\tfile_size\x18\x03 \x01(\x03R\bfileSize\x12\x1b\n" + + "\tmime_type\x18\x04 \x01(\tR\bmimeType\x12!\n" + + "\fcontent_hash\x18\x05 \x01(\tR\vcontentHash\x125\n" + + "\apurpose\x18\x06 \x01(\x0e2\x1b.chatnow.media.MediaPurposeR\apurpose\"\xcf\x02\n" + + "\x0eApplyUploadRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x12\x17\n" + + "\afile_id\x18\x02 \x01(\tR\x06fileId\x12%\n" + + "\x0ealready_exists\x18\x03 \x01(\bR\ralreadyExists\x12\x1d\n" + + "\n" + + "upload_url\x18\x04 \x01(\tR\tuploadUrl\x12D\n" + + "\aheaders\x18\x05 \x03(\v2*.chatnow.media.ApplyUploadRsp.HeadersEntryR\aheaders\x12$\n" + + "\x0eexpires_in_sec\x18\x06 \x01(\x05R\fexpiresInSec\x1a:\n" + + "\fHeadersEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" + + "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"K\n" + + "\x11CompleteUploadReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x17\n" + + "\afile_id\x18\x02 \x01(\tR\x06fileId\"\x81\x01\n" + + "\x11CompleteUploadRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x124\n" + + "\tfile_info\x18\x02 \x01(\v2\x17.chatnow.media.FileInfoR\bfileInfo\"\xe2\x01\n" + + "\x10InitMultipartReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x1b\n" + + "\tfile_name\x18\x02 \x01(\tR\bfileName\x12\x1b\n" + + "\tfile_size\x18\x03 \x01(\x03R\bfileSize\x12\x1b\n" + + "\tmime_type\x18\x04 \x01(\tR\bmimeType\x12!\n" + + "\fcontent_hash\x18\x05 \x01(\tR\vcontentHash\x125\n" + + "\apurpose\x18\x06 \x01(\x0e2\x1b.chatnow.media.MediaPurposeR\apurpose\"\xbf\x01\n" + + "\x10InitMultipartRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x12\x17\n" + + "\afile_id\x18\x02 \x01(\tR\x06fileId\x12\x1b\n" + + "\tupload_id\x18\x03 \x01(\tR\buploadId\x12=\n" + + "\x1brecommended_part_size_bytes\x18\x04 \x01(\x05R\x18recommendedPartSizeBytes\"k\n" + + "\fApplyPartReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x1b\n" + + "\tupload_id\x18\x02 \x01(\tR\buploadId\x12\x1f\n" + + "\vpart_number\x18\x03 \x01(\x05R\n" + + "partNumber\"\x8b\x01\n" + + "\fApplyPartRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x12\x1d\n" + + "\n" + + "upload_url\x18\x02 \x01(\tR\tuploadUrl\x12$\n" + + "\x0eexpires_in_sec\x18\x03 \x01(\x05R\fexpiresInSec\"?\n" + + "\bPartETag\x12\x1f\n" + + "\vpart_number\x18\x01 \x01(\x05R\n" + + "partNumber\x12\x12\n" + + "\x04etag\x18\x02 \x01(\tR\x04etag\"\x81\x01\n" + + "\x14CompleteMultipartReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x1b\n" + + "\tupload_id\x18\x02 \x01(\tR\buploadId\x12-\n" + + "\x05parts\x18\x03 \x03(\v2\x17.chatnow.media.PartETagR\x05parts\"\x84\x01\n" + + "\x14CompleteMultipartRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x124\n" + + "\tfile_info\x18\x02 \x01(\v2\x17.chatnow.media.FileInfoR\bfileInfo\"O\n" + + "\x11AbortMultipartReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x1b\n" + + "\tupload_id\x18\x02 \x01(\tR\buploadId\"K\n" + + "\x11AbortMultipartRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\"J\n" + + "\x10ApplyDownloadReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x17\n" + + "\afile_id\x18\x02 \x01(\tR\x06fileId\"\xc9\x01\n" + + "\x10ApplyDownloadRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x12!\n" + + "\fdownload_url\x18\x02 \x01(\tR\vdownloadUrl\x12$\n" + + "\x0eexpires_in_sec\x18\x03 \x01(\x05R\fexpiresInSec\x124\n" + + "\tfile_info\x18\x04 \x01(\v2\x17.chatnow.media.FileInfoR\bfileInfo\"H\n" + + "\x0eGetFileInfoReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12\x17\n" + + "\afile_id\x18\x02 \x01(\tR\x06fileId\"~\n" + + "\x0eGetFileInfoRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x124\n" + + "\tfile_info\x18\x02 \x01(\v2\x17.chatnow.media.FileInfoR\bfileInfo\"\\\n" + + "\x14SpeechRecognitionReq\x12\x1d\n" + + "\n" + + "request_id\x18\x01 \x01(\tR\trequestId\x12%\n" + + "\x0espeech_content\x18\x02 \x01(\fR\rspeechContent\"}\n" + + "\x14SpeechRecognitionRsp\x126\n" + + "\x06header\x18\x01 \x01(\v2\x1e.chatnow.common.ResponseHeaderR\x06header\x12-\n" + + "\x12recognition_result\x18\x02 \x01(\tR\x11recognitionResult*b\n" + + "\fMediaPurpose\x12\x1d\n" + + "\x19MEDIA_PURPOSE_UNSPECIFIED\x10\x00\x12\n" + + "\n" + + "\x06AVATAR\x10\x01\x12\x10\n" + + "\fGROUP_AVATAR\x10\x02\x12\b\n" + + "\x04CHAT\x10\x03\x12\v\n" + + "\aSTICKER\x10\x042\x97\x06\n" + + "\fMediaService\x12K\n" + + "\vApplyUpload\x12\x1d.chatnow.media.ApplyUploadReq\x1a\x1d.chatnow.media.ApplyUploadRsp\x12T\n" + + "\x0eCompleteUpload\x12 .chatnow.media.CompleteUploadReq\x1a .chatnow.media.CompleteUploadRsp\x12W\n" + + "\x13InitMultipartUpload\x12\x1f.chatnow.media.InitMultipartReq\x1a\x1f.chatnow.media.InitMultipartRsp\x12K\n" + + "\x0fApplyPartUpload\x12\x1b.chatnow.media.ApplyPartReq\x1a\x1b.chatnow.media.ApplyPartRsp\x12c\n" + + "\x17CompleteMultipartUpload\x12#.chatnow.media.CompleteMultipartReq\x1a#.chatnow.media.CompleteMultipartRsp\x12Z\n" + + "\x14AbortMultipartUpload\x12 .chatnow.media.AbortMultipartReq\x1a .chatnow.media.AbortMultipartRsp\x12Q\n" + + "\rApplyDownload\x12\x1f.chatnow.media.ApplyDownloadReq\x1a\x1f.chatnow.media.ApplyDownloadRsp\x12K\n" + + "\vGetFileInfo\x12\x1d.chatnow.media.GetFileInfoReq\x1a\x1d.chatnow.media.GetFileInfoRsp\x12]\n" + + "\x11SpeechRecognition\x12#.chatnow.media.SpeechRecognitionReq\x1a#.chatnow.media.SpeechRecognitionRspB\x03\x80\x01\x01b\x06proto3" var ( file_media_media_service_proto_rawDescOnce sync.Once - file_media_media_service_proto_rawDescData = file_media_media_service_proto_rawDesc + file_media_media_service_proto_rawDescData []byte ) func file_media_media_service_proto_rawDescGZIP() []byte { file_media_media_service_proto_rawDescOnce.Do(func() { - file_media_media_service_proto_rawDescData = protoimpl.X.CompressGZIP(file_media_media_service_proto_rawDescData) + file_media_media_service_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_media_media_service_proto_rawDesc), len(file_media_media_service_proto_rawDesc))) }) return file_media_media_service_proto_rawDescData } @@ -1672,253 +1503,11 @@ func file_media_media_service_proto_init() { if File_media_media_service_proto != nil { return } - if !protoimpl.UnsafeEnabled { - file_media_media_service_proto_msgTypes[0].Exporter = func(v any, i int) any { - switch v := v.(*FileInfo); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[1].Exporter = func(v any, i int) any { - switch v := v.(*ApplyUploadReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[2].Exporter = func(v any, i int) any { - switch v := v.(*ApplyUploadRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[3].Exporter = func(v any, i int) any { - switch v := v.(*CompleteUploadReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[4].Exporter = func(v any, i int) any { - switch v := v.(*CompleteUploadRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[5].Exporter = func(v any, i int) any { - switch v := v.(*InitMultipartReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[6].Exporter = func(v any, i int) any { - switch v := v.(*InitMultipartRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[7].Exporter = func(v any, i int) any { - switch v := v.(*ApplyPartReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[8].Exporter = func(v any, i int) any { - switch v := v.(*ApplyPartRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[9].Exporter = func(v any, i int) any { - switch v := v.(*PartETag); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[10].Exporter = func(v any, i int) any { - switch v := v.(*CompleteMultipartReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[11].Exporter = func(v any, i int) any { - switch v := v.(*CompleteMultipartRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[12].Exporter = func(v any, i int) any { - switch v := v.(*AbortMultipartReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[13].Exporter = func(v any, i int) any { - switch v := v.(*AbortMultipartRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[14].Exporter = func(v any, i int) any { - switch v := v.(*ApplyDownloadReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[15].Exporter = func(v any, i int) any { - switch v := v.(*ApplyDownloadRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[16].Exporter = func(v any, i int) any { - switch v := v.(*GetFileInfoReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[17].Exporter = func(v any, i int) any { - switch v := v.(*GetFileInfoRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[18].Exporter = func(v any, i int) any { - switch v := v.(*SpeechRecognitionReq); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - file_media_media_service_proto_msgTypes[19].Exporter = func(v any, i int) any { - switch v := v.(*SpeechRecognitionRsp); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields - default: - return nil - } - } - } type x struct{} out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), - RawDescriptor: file_media_media_service_proto_rawDesc, + RawDescriptor: unsafe.Slice(unsafe.StringData(file_media_media_service_proto_rawDesc), len(file_media_media_service_proto_rawDesc)), NumEnums: 1, NumMessages: 21, NumExtensions: 0, @@ -1930,7 +1519,6 @@ func file_media_media_service_proto_init() { MessageInfos: file_media_media_service_proto_msgTypes, }.Build() File_media_media_service_proto = out.File - file_media_media_service_proto_rawDesc = nil file_media_media_service_proto_goTypes = nil file_media_media_service_proto_depIdxs = nil } diff --git a/tests/reliability/discovery_test.go b/tests/reliability/discovery_test.go new file mode 100644 index 0000000..6bfb2f6 --- /dev/null +++ b/tests/reliability/discovery_test.go @@ -0,0 +1,56 @@ +//go:build reliability + +package reliability_test + +import ( + "testing" + "time" + + "github.com/stretchr/testify/require" + + "chatnow-tests/pkg/chaos" + "chatnow-tests/pkg/client" + "chatnow-tests/pkg/fixture" + identity "chatnow-tests/proto/chatnow/identity" + msg "chatnow-tests/proto/chatnow/message" + transmite "chatnow-tests/proto/chatnow/transmite" +) + +// RL-DISCOVERY-01 | P1 | RPC callers recover after a service hostname changes IP. +func TestRL_DiscoveryRecoversAfterIdentityAddressChange(t *testing.T) { + user, _, _ := fixture.RegisterAndLogin(t, HTTP) + peer, _, _ := fixture.RegisterAndLogin(t, HTTP) + convID := fixture.CreateGroupWithMembers(t, user, []*client.HTTPClient{peer}, "rl-discovery") + profileWorks := func() bool { + rsp := &identity.GetProfileRsp{} + err := user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{RequestId: client.NewRequestID()}, rsp) + return err == nil && rsp.GetHeader().GetSuccess() + } + require.True(t, profileWorks(), "Identity must work before the address fault") + // Registered before the fault controller so this runs after network cleanup. + // A restored interface alone does not prove the next suite can call Identity. + t.Cleanup(func() { + require.Eventually(t, profileWorks, 45*time.Second, time.Second, + "account RPC must recover after restoring the original Identity endpoint") + }) + checkProcesses := chaos.ChangeIdentityAddress(t, HTTP.Config()) + started := time.Now() + // This user has not sent before the fault, so Transmite must fetch Identity + // profile data instead of succeeding from a prewarmed user-info cache. + require.Eventually(t, func() bool { + if !profileWorks() { + return false + } + rsp := &transmite.SendMessageRsp{} + err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ + RequestId: client.NewRequestID(), ConversationId: convID, + Content: &msg.MessageContent{Type: msg.MessageType_TEXT, + Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "after-address-change"}}}, + ClientMsgId: client.NewRequestID(), + }, rsp) + return err == nil && rsp.GetHeader().GetSuccess() + }, 45*time.Second, time.Second, + "Gateway and Transmite must resolve the new Identity address without a caller restart") + checkProcesses() + t.Logf("account and message RPCs recovered in %s", time.Since(started)) +} diff --git a/tests/reliability/idempotent_response_test.go b/tests/reliability/idempotent_response_test.go new file mode 100644 index 0000000..29540d4 --- /dev/null +++ b/tests/reliability/idempotent_response_test.go @@ -0,0 +1,126 @@ +//go:build reliability + +package reliability_test + +import ( + "fmt" + "testing" + "time" + + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/proto" + + "chatnow-tests/pkg/chaos" + "chatnow-tests/pkg/cleanup" + "chatnow-tests/pkg/client" + "chatnow-tests/pkg/fixture" + "chatnow-tests/pkg/verify" + authmeta "chatnow-tests/proto/chatnow/common/auth" + msg "chatnow-tests/proto/chatnow/message" + transmite "chatnow-tests/proto/chatnow/transmite" +) + +// RL-MESSAGE-01 | P0 | Idempotent success stays complete across lookup outages. +func TestRL_IdempotentResponseDuringMessageOutage(t *testing.T) { + for _, persisted := range []bool{false, true} { + t.Run(fmt.Sprintf("persisted_%t", persisted), func(t *testing.T) { + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 15*time.Second)) + // This layer owns a disposable synthetic stack; restore the service + // before suite cleanup touches any asynchronous message state. + t.Cleanup(func() { cleanup.CleanupAll(t, HTTP.Config()) }) + user, peer, convID := fixture.MakeFriends(t, HTTP) + db := verify.NewDBVerifier(HTTP.Config().Database.MySQLDSN) + t.Cleanup(db.Close) + clientID := client.NewRequestID() + endpoints := reliabilityTransmiteEndpoints(t) + require.Len(t, endpoints, 1) + metadata, err := proto.Marshal(&authmeta.RpcMetadata{ + TraceId: client.NewRequestID(), UserId: user.UserID, DeviceId: user.DeviceID, + }) + require.NoError(t, err) + send := func() *transmite.SendMessageRsp { + rsp := &transmite.SendMessageRsp{} + require.NoError(t, user.DoInternalRPC(endpoints[0], "chatnow.transmite.MsgTransmitService", "SendMessage", &transmite.SendMessageReq{ + RequestId: client.NewRequestID(), ConversationId: convID, ClientMsgId: clientID, + Content: &msg.MessageContent{Type: msg.MessageType_TEXT, + Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "idempotent-outage"}}}, + }, rsp, metadata)) + return rsp + } + var restore func() + if !persisted { + restore = chaos.StopMessage(t, HTTP.Config()) + } + first := send() + require.True(t, first.GetHeader().GetSuccess()) + require.NotZero(t, first.GetMessage().GetMessageId()) + require.NotZero(t, first.GetMessage().GetSeqId()) + if persisted { + db.WaitMessageExists(t, first.GetMessage().GetMessageId(), 10*time.Second) + require.True(t, send().GetHeader().GetSuccess(), "direct duplicate control must succeed before the fault") + restore = chaos.StopMessage(t, HTTP.Config()) + } else { + db.MessageCount(t, convID, 0) + } + t.Cleanup(func() { + restore() + db.WaitMessageExists(t, first.GetMessage().GetMessageId(), 10*time.Second) + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 15*time.Second), "fault cleanup must restore service readiness") + }) + key := "im:msg:idem:" + user.UserID + ":" + clientID + for _, state := range []string{"accepted", "persisted"} { + // Exercise both existing cache formats, without changing the + // identity of the broker-confirmed message or replaying publication. + value := fmt.Sprintf("%s:%d", state, first.GetMessage().GetMessageId()) + verify.RedisCLI(t, "SET", key, value, "EX", "86400") + started := time.Now() + repeated := send() + if repeated.GetHeader().GetSuccess() { + require.Equal(t, first.GetMessage().GetMessageId(), repeated.GetMessage().GetMessageId()) + require.Equal(t, first.GetMessage().GetSeqId(), repeated.GetMessage().GetSeqId(), + "successful duplicate must preserve the original nonzero sequence") + require.Equal(t, convID, repeated.GetMessage().GetConversationId()) + } else { + require.EqualValues(t, 9002, repeated.GetHeader().GetErrorCode()) + require.Nil(t, repeated.GetMessage(), "unavailable lookup must not expose a partial result") + } + require.Less(t, time.Since(started), 5*time.Second, "lookup must be bounded") + require.Equal(t, value, verify.RedisCLI(t, "GET", key), "retain the deduplication guard") + } + restore() + db.WaitMessageExists(t, first.GetMessage().GetMessageId(), 10*time.Second) + for _, state := range []string{"accepted", "persisted", "pending"} { + value := state + if state != "pending" { + value = fmt.Sprintf("%s:%d", state, first.GetMessage().GetMessageId()) + } + verify.RedisCLI(t, "SET", key, value, "EX", "86400") + deadline := time.Now().Add(10 * time.Second) + recovered := send() + for !recovered.GetHeader().GetSuccess() && time.Now().Before(deadline) { + require.EqualValues(t, 9002, recovered.GetHeader().GetErrorCode()) + require.Nil(t, recovered.GetMessage()) + time.Sleep(50 * time.Millisecond) + recovered = send() + } + if !recovered.GetHeader().GetSuccess() { + lookup := &msg.SelectByClientMsgIdRsp{} + err := user.DoAuth("/service/message/select_by_client_msg_id", &msg.SelectByClientMsgIdReq{ + RequestId: client.NewRequestID(), ClientMsgId: clientID, + }, lookup) + t.Logf("recovery diagnostic: duplicate_code=%d lookup_transport_ok=%t lookup_success=%t lookup_code=%d has_message=%t nonzero_seq=%t", + recovered.GetHeader().GetErrorCode(), err == nil, lookup.GetHeader().GetSuccess(), lookup.GetHeader().GetErrorCode(), + lookup.GetMessage() != nil, lookup.GetMessage().GetSeqId() != 0) + } + require.True(t, recovered.GetHeader().GetSuccess(), "recovery must return the original message") + require.Equal(t, first.GetMessage().GetMessageId(), recovered.GetMessage().GetMessageId()) + require.Equal(t, first.GetMessage().GetSeqId(), recovered.GetMessage().GetSeqId()) + require.Equal(t, convID, recovered.GetMessage().GetConversationId()) + require.Equal(t, "idempotent-outage", recovered.GetMessage().GetContent().GetText().GetText()) + } + db.MessageCount(t, convID, 1) + db.UserTimelineExists(t, user.UserID, convID, 1) + db.UserTimelineExists(t, peer.UserID, convID, 1) + }) + } +} diff --git a/tests/reliability/lease_test.go b/tests/reliability/lease_test.go new file mode 100644 index 0000000..37944c8 --- /dev/null +++ b/tests/reliability/lease_test.go @@ -0,0 +1,39 @@ +//go:build reliability + +package reliability_test + +import ( + "testing" + "time" + + "github.com/stretchr/testify/require" + + "chatnow-tests/pkg/chaos" + "chatnow-tests/pkg/cleanup" + "chatnow-tests/pkg/client" + "chatnow-tests/pkg/fixture" + identity "chatnow-tests/proto/chatnow/identity" +) + +// RL-DISCOVERY-02 | P0 | Expired registration recovers without process restarts. +func TestRL_RegistryRecoversAfterLeaseExpiry(t *testing.T) { + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 15*time.Second)) + t.Cleanup(func() { cleanup.CleanupAll(t, HTTP.Config()) }) + user, _, _ := fixture.RegisterAndLogin(t, HTTP) + profileWorks := func() bool { + rsp := &identity.GetProfileRsp{} + return user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{RequestId: client.NewRequestID()}, rsp) == nil && rsp.GetHeader().GetSuccess() + } + require.True(t, profileWorks(), "profile control must succeed before the fault") + t.Cleanup(func() { + require.Eventually(t, profileWorks, 30*time.Second, 200*time.Millisecond, "fault cleanup must restore Identity RPC") + }) + checkProcesses := chaos.ExpireIdentityLease(t, HTTP.Config()) + started := time.Now() + require.Eventually(t, func() bool { + registered, err := chaos.IdentityRegistered(HTTP.Config()) + return err == nil && registered && profileWorks() + }, 60*time.Second, 500*time.Millisecond, "expired lease must recover registration and account RPC without restarting") + checkProcesses() + t.Logf("registration and account RPC recovered in %s", time.Since(started)) +} diff --git a/tests/reliability/message_fixture_test.go b/tests/reliability/message_fixture_test.go new file mode 100644 index 0000000..055ea84 --- /dev/null +++ b/tests/reliability/message_fixture_test.go @@ -0,0 +1,93 @@ +//go:build reliability + +package reliability_test + +import ( + "context" + "fmt" + "os/exec" + "strconv" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/require" + + "chatnow-tests/pkg/chaos" + "chatnow-tests/pkg/cleanup" + "chatnow-tests/pkg/fixture" + "chatnow-tests/pkg/verify" +) + +// RL-MESSAGE-02 | P0 | Text fixtures wait for durable messages, not broker acceptance. +func TestRL_TextFixtureWaitsForPersistence(t *testing.T) { + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 15*time.Second)) + t.Cleanup(func() { cleanup.CleanupAll(t, HTTP.Config()) }) + sender, _, conversation := fixture.MakeFriends(t, HTTP) + db := verify.NewDBVerifier(HTTP.Config().Database.MySQLDSN) + t.Cleanup(db.Close) + restore := chaos.StopMessage(t, HTTP.Config()) + finished := make(chan struct{}) + t.Cleanup(func() { + restore() + select { + case <-finished: + case <-time.After(15 * time.Second): + t.Error("message fixture did not finish after consumer restoration") + } + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 15*time.Second)) + }) + go func() { + defer close(finished) + id, seq := fixture.SendTextMessage(t, sender, conversation, "fixture-persistence") + if id == 0 || seq == 0 { + t.Error("message fixture returned incomplete identity") + return + } + db.MessageExists(t, id) + }() + + // The exact synthetic sender's accepted guard proves publication happened + // while its consumer was stopped. No token or message body is inspected. + var acceptedID int64 + deadline := time.Now().Add(5 * time.Second) + for acceptedID == 0 && time.Now().Before(deadline) { + for node := 1; node <= 6; node++ { + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + output, err := exec.CommandContext(ctx, "docker", "exec", fmt.Sprintf("redis-node%d", node), + "redis-cli", "-p", strconv.Itoa(6378+node), "--scan", "--pattern", "im:msg:idem:"+sender.UserID+":*").Output() + cancel() + require.NoError(t, err, "inspect synthetic sender's publication guard") + for _, key := range strings.Fields(string(output)) { + value := verify.RedisCLI(t, "GET", key) + if raw, found := strings.CutPrefix(value, "accepted:"); found { + acceptedID, err = strconv.ParseInt(raw, 10, 64) + require.NoError(t, err) + } + } + } + if acceptedID == 0 { + time.Sleep(25 * time.Millisecond) + } + } + require.NotZero(t, acceptedID, "broker acceptance was not observed during the fault") + db.MessageCount(t, conversation, 0) + t.Cleanup(func() { + restore() + db.WaitMessageExists(t, acceptedID, 10*time.Second) + }) + // This is a bounded negative observation under an established fault, not + // a readiness delay. The positive assertion below checks actual MySQL state. + select { + case <-finished: + t.Error("message fixture returned before the stopped consumer could persist") + case <-time.After(250 * time.Millisecond): + } + restore() + select { + case <-finished: + case <-time.After(15 * time.Second): + t.Fatal("message fixture did not converge after restoring the consumer") + } + db.MessageExists(t, acceptedID) +} diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go index b33859c..81b5890 100644 --- a/tests/reliability/redis_failover_test.go +++ b/tests/reliability/redis_failover_test.go @@ -4,19 +4,23 @@ package reliability_test import ( "bytes" + "context" "fmt" "net" + "os" "os/exec" "strings" "testing" "time" "github.com/stretchr/testify/require" + "google.golang.org/protobuf/proto" "chatnow-tests/pkg/chaos" "chatnow-tests/pkg/client" "chatnow-tests/pkg/fixture" "chatnow-tests/pkg/verify" + authmeta "chatnow-tests/proto/chatnow/common/auth" identity "chatnow-tests/proto/chatnow/identity" msg "chatnow-tests/proto/chatnow/message" transmite "chatnow-tests/proto/chatnow/transmite" @@ -39,19 +43,64 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) { }, prewarmRsp)) require.True(t, prewarmRsp.GetHeader().GetSuccess()) endpoints := reliabilityTransmiteEndpoints(t) + require.Len(t, endpoints, 1, "RL-05 requires one isolated Transmite for per-request circuit evidence") + metadata, err := proto.Marshal(&authmeta.RpcMetadata{ + TraceId: client.NewRequestID(), UserId: user.UserID, DeviceId: user.DeviceID, + }) + require.NoError(t, err) + internalSend := func(marker string) (*transmite.SendMessageRsp, error) { + rsp := &transmite.SendMessageRsp{} + err := user.DoInternalRPC(endpoints[0], "chatnow.transmite.MsgTransmitService", "SendMessage", + &transmite.SendMessageReq{ + RequestId: client.NewRequestID(), ConversationId: convID, ClientMsgId: client.NewRequestID(), + Content: &msg.MessageContent{Type: msg.MessageType_TEXT, + Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: marker}}}, + }, rsp, metadata) + return rsp, err + } + internalWarm, err := internalSend("internal-prewarm") + require.NoError(t, err) + require.True(t, internalWarm.GetHeader().GetSuccess(), "direct RPC control must reach the real service") openedBefore := reliabilitySumBVar(t, endpoints, "redis_circuit_open_total") rejectedBefore := reliabilitySumBVar(t, endpoints, "redis_circuit_rejected_total") recoveredBefore := reliabilitySumBVar(t, endpoints, "redis_circuit_recovered_total") - t.Cleanup(func() { - chaos.StartRedisCluster(t, HTTP.Config()) - chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second) - }) - chaos.StopRedisCluster(t, HTTP.Config()) + redisPaused := true + // The same idempotency key bounds recovery writes even after an ambiguous + // transport response. Redis cluster health alone does not close RPC circuits. + recoveryID := client.NewRequestID() + restore := func() { + if redisPaused { + chaos.UnpauseRedisCluster(t, HTTP.Config()) + chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second) + redisPaused = false + } + uid := user.UserID + require.Eventually(t, func() bool { + profile := &identity.GetProfileRsp{} + if user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{ + RequestId: client.NewRequestID(), UserId: &uid, + }, profile) != nil || !profile.GetHeader().GetSuccess() { + return false + } + sent := &transmite.SendMessageRsp{} + return user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ + RequestId: client.NewRequestID(), ConversationId: convID, ClientMsgId: recoveryID, + Content: &msg.MessageContent{Type: msg.MessageType_TEXT, + Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "recovered"}}}, + }, sent) == nil && sent.GetHeader().GetSuccess() + }, 30*time.Second, 100*time.Millisecond, "Redis cleanup must restore account and message RPCs") + } + t.Cleanup(restore) + chaos.PauseRedisCluster(t, HTTP.Config()) rateLimited := 0 seqUnavailable := 0 - for i := 0; i < 650; i++ { + responseCodes := make(map[int32]int) + outageStarted := time.Now() + // The dedicated RL-05 stack uses 8 user tokens per minute. A bounded + // burst exhausts fallback capacity even with refill during failed Redis calls. + for i := 0; i < 32; i++ { sendRsp := &transmite.SendMessageRsp{} err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ RequestId: client.NewRequestID(), @@ -63,6 +112,7 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) { ClientMsgId: client.NewRequestID(), }, sendRsp) require.NoError(t, err) + responseCodes[sendRsp.GetHeader().GetErrorCode()]++ if sendRsp.GetHeader().GetErrorMessage() == "rate_limited" { rateLimited++ } @@ -70,11 +120,51 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) { seqUnavailable++ } } + t.Logf("outage responses: codes=%v rate_limited=%d seq_unavailable=%d elapsed=%s", + responseCodes, rateLimited, seqUnavailable, time.Since(outageStarted)) require.Greater(t, rateLimited, 0, "Redis outage must retain bounded rate limiting") require.Greater(t, seqUnavailable, 0, "SeqGen truth source must fail unavailable") require.Greater(t, reliabilitySumBVar(t, endpoints, "redis_circuit_open_total"), openedBefore) require.Greater(t, reliabilitySumBVar(t, endpoints, "redis_circuit_rejected_total"), rejectedBefore) + // A Gateway call includes a separate Redis circuit. Measure the Transmite + // boundary directly and classify every sample using its actual counters. + // No slow Open rejection can be discarded by retrying for a faster sample. + // This delay deliberately admits a recovery probe; it is not readiness + // evidence. Counter deltas below must prove that the probe actually ran. + time.Sleep(1100 * time.Millisecond) + openSamples, probeSamples := 0, 0 + for i := 0; i < 8 && openSamples < 3; i++ { + failures := reliabilitySumBVar(t, endpoints, "redis_call_failure_total") + rejections := reliabilitySumBVar(t, endpoints, "redis_circuit_rejected_total") + opens := reliabilitySumBVar(t, endpoints, "redis_circuit_open_total") + started := time.Now() + failed, err := internalSend("fast-fail") + elapsed := time.Since(started) + require.NoError(t, err) + require.False(t, failed.GetHeader().GetSuccess()) + failureDelta := reliabilitySumBVar(t, endpoints, "redis_call_failure_total") - failures + rejectedDelta := reliabilitySumBVar(t, endpoints, "redis_circuit_rejected_total") - rejections + openDelta := reliabilitySumBVar(t, endpoints, "redis_circuit_open_total") - opens + t.Logf("Transmite circuit sample %d: elapsed=%s failures=%d rejected=%d opened=%d", + i, elapsed, failureDelta, rejectedDelta, openDelta) + if failureDelta > 0 { + require.Positive(t, openDelta, "an admitted recovery attempt must reopen the paused Redis circuit") + probeSamples++ + continue + } + require.Zero(t, failureDelta, "counters must be monotonic") + require.Zero(t, openDelta, "Open rejection must not change circuit generation") + require.Positive(t, rejectedDelta, "sample must prove an actual circuit rejection") + require.Less(t, elapsed, 50*time.Millisecond, "every proven Open rejection must meet the latency bound") + openSamples++ + } + require.Equal(t, 3, openSamples, "insufficient phase-proven Open rejections in bounded sample set") + require.Positive(t, probeSamples, "the expired Open interval must exercise an actual recovery probe") + if os.Getenv("CHATNOW_REDIS_CLEANUP_CHILD") == "1" { + t.Fatal("injected RL-05 assertion failure") + } + uid := user.UserID for i := 0; i < 3; i++ { rsp := &identity.GetProfileRsp{} @@ -83,42 +173,44 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) { }, rsp) } - started := time.Now() - fastFail := &transmite.SendMessageRsp{} - err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ - RequestId: client.NewRequestID(), ConversationId: convID, - Content: &msg.MessageContent{Type: msg.MessageType_TEXT, - Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "fast-fail"}}}, - ClientMsgId: client.NewRequestID(), - }, fastFail) - require.NoError(t, err) - require.False(t, fastFail.GetHeader().GetSuccess()) - require.Less(t, time.Since(started), 50*time.Millisecond) - - chaos.StartRedisCluster(t, HTTP.Config()) - chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second) - time.Sleep(1100 * time.Millisecond) - rsp := &identity.GetProfileRsp{} - require.NoError(t, user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{ - RequestId: client.NewRequestID(), UserId: &uid, - }, rsp)) - require.True(t, rsp.GetHeader().GetSuccess()) - - recoveredSend := &transmite.SendMessageRsp{} - require.NoError(t, user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{ - RequestId: client.NewRequestID(), ConversationId: convID, - Content: &msg.MessageContent{Type: msg.MessageType_TEXT, - Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "recovered"}}}, - ClientMsgId: client.NewRequestID(), - }, recoveredSend)) - require.True(t, recoveredSend.GetHeader().GetSuccess(), recoveredSend.GetHeader().GetErrorMessage()) + restore() require.Greater(t, reliabilitySumBVar(t, endpoints, "redis_circuit_recovered_total"), recoveredBefore, "an Open->HalfOpen probe must recover the Transmite Redis circuit") } +// RL-REDIS-01 | P0 | Assertion failure must restore actual message availability. +func TestRL_RedisCircuitCleanupAfterAssertionFailure(t *testing.T) { + sender, _, _ := fixture.RegisterAndLogin(t, HTTP) + peer, _, _ := fixture.RegisterAndLogin(t, HTTP) + convID := fixture.CreateGroupWithMembers(t, sender, []*client.HTTPClient{peer}, "rl-cleanup") + sendReliabilityMessage(t, sender, convID, "cleanup-prewarm") + // The parent owns emergency unpause if the bounded child is interrupted. + childCompleted := false + t.Cleanup(func() { + if !childCompleted { + chaos.UnpauseRedisCluster(t, HTTP.Config()) + chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second) + } + }) + executable, err := os.Executable() + require.NoError(t, err) + ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, executable, "-test.run=^TestRL_RedisCircuitFastFailAndRecovery$", "-test.v", "-test.count=1") + cmd.Env = append(os.Environ(), "CHATNOW_REDIS_CLEANUP_CHILD=1") + out, err := cmd.CombinedOutput() + if exit, ok := err.(*exec.ExitError); ok && exit.ExitCode() == 1 { + childCompleted = true // Go completed all test cleanups before exit(1). + } + require.Error(t, err, "child must execute the injected failure") + require.Contains(t, string(out), "injected RL-05 assertion failure", "child failed before the cleanup boundary") + // Do not poll here: the child's cleanup owns the convergence deadline. + sendReliabilityMessage(t, sender, convID, "cleanup-confirmed") +} + // RL-05 Push truth source: Rabbit accepts while Push is paused; after Redis is -// stopped, resuming Push must requeue before websocket delivery. Recovery then -// permits the half-open probe, durable Unacked write, and at-least-once delivery. +// paused without withdrawing DNS, resuming Push must requeue before delivery. +// Recovery permits the half-open probe, durable Unacked write, and delivery. func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) { // This black-box choreography is intentionally tied to the compose topology: // one Push container owns the websocket and exposes the sole configured bvar @@ -134,7 +226,11 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) { ws, err := client.OpenWebSocket(HTTP.Config()) require.NoError(t, err) t.Cleanup(func() { _ = ws.Close() }) - require.NoError(t, ws.WriteBinary(client.PushAuthNotify(recipient.AccessToken, "default_device"))) + require.NoError(t, ws.WriteBinary(client.PushAuthNotify(recipient.AccessToken, recipient.DeviceID))) + senderWS, err := client.OpenWebSocket(HTTP.Config()) + require.NoError(t, err) + t.Cleanup(func() { _ = senderWS.Close() }) + require.NoError(t, senderWS.WriteBinary(client.PushAuthNotify(sender.AccessToken, sender.DeviceID))) deviceKey := fmt.Sprintf("im:dev:{%s}", recipient.UserID) deadline := time.Now().Add(5 * time.Second) @@ -151,23 +247,25 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) { require.NoError(t, readErr) } - // A successful end-to-end delivery warms that instance's route L1 before it - // is paused. The test-only TTL keeps the route through the outage choreography. + // Warm both member routes: an offline sender has no positive L1 route and + // would fail route discovery before the Unacked persistence under test. + // The test-only TTL keeps both routes through the outage choreography. warmMarker := "rl-unacked-warm-" + client.NewRequestID() sendReliabilityMessage(t, sender, convID, warmMarker) require.True(t, readWebSocketMarker(ws, warmMarker, 5*time.Second), "warm Push delivery missing") + require.True(t, readWebSocketMarker(senderWS, warmMarker, 5*time.Second), "warm sender Push delivery missing") persistBefore := verify.BVar(t, pushEndpoint, "push_unacked_persist_failure_total") requeueBefore := verify.BVar(t, pushEndpoint, "push_message_requeue_total") requireDocker(t, "pause", pushContainer) paused := true - redisStopped := false + redisPaused := false t.Cleanup(func() { if paused { _, _ = exec.Command("docker", "unpause", pushContainer).CombinedOutput() } - if redisStopped { - chaos.StartRedisCluster(t, HTTP.Config()) + if redisPaused { + chaos.UnpauseRedisCluster(t, HTTP.Config()) chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second) } }) @@ -175,25 +273,24 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) { marker := "rl-unacked-" + client.NewRequestID() sendReliabilityMessage(t, sender, convID, marker) - chaos.StopRedisCluster(t, HTTP.Config()) - redisStopped = true + redisPaused = true + chaos.PauseRedisCluster(t, HTTP.Config()) requireDocker(t, "unpause", pushContainer) paused = false requireBVarIncrease(t, pushEndpoint, "push_unacked_persist_failure_total", persistBefore, 5*time.Second) requireBVarIncrease(t, pushEndpoint, "push_message_requeue_total", requeueBefore, 5*time.Second) - if payload, readErr := ws.ReadFrame(500 * time.Millisecond); readErr == nil { - t.Fatalf("Push delivered before durable Unacked persistence: %x", payload) - } else if timeout, ok := readErr.(net.Error); !ok || !timeout.Timeout() { - t.Fatalf("websocket failed while awaiting Redis outage: %v", readErr) - } + // Other already durable messages may be redelivered; only the newly queued + // marker is forbidden before its own Unacked write succeeds. + require.False(t, readWebSocketMarker(ws, marker, 500*time.Millisecond), + "Push delivered the outage message before durable Unacked persistence") - chaos.StartRedisCluster(t, HTTP.Config()) + chaos.UnpauseRedisCluster(t, HTTP.Config()) chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second) - redisStopped = false + redisPaused = false require.True(t, readWebSocketMarker(ws, marker, 20*time.Second), "requeued Push was not delivered after Redis recovery") - unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID) + unackedKey := fmt.Sprintf("im:unack:{%s:%s}", recipient.UserID, recipient.DeviceID) require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", unackedKey), "delivery must follow durable Unacked persistence") } diff --git a/tests/reliability/worker_test.go b/tests/reliability/worker_test.go new file mode 100644 index 0000000..e88d70b --- /dev/null +++ b/tests/reliability/worker_test.go @@ -0,0 +1,27 @@ +//go:build reliability + +package reliability_test + +import ( + "testing" + "time" + + "github.com/stretchr/testify/require" + + "chatnow-tests/pkg/chaos" + "chatnow-tests/pkg/cleanup" +) + +// RL-WORKER-01 | P0 | Worker lease loss exits explicitly, including as container PID 1. +func TestRL_WorkerLeaseLossFencesProcess(t *testing.T) { + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 20*time.Second)) + t.Cleanup(func() { require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 45*time.Second)) }) + inspect, restore := chaos.RevokeWorkerLease(t, HTTP.Config()) + exit := chaos.WaitWorkerExit(t, inspect) + require.False(t, exit.OOMKilled, "worker protection must not be an OOM kill") + require.Equal(t, 1, exit.ExitCode, "worker lease loss must exit explicitly, not through PID 1 signal fallback") + require.True(t, exit.ProtectionLogged, "exit must identify worker ownership loss without sensitive data") + restore() + require.NoError(t, cleanup.WaitForStackReady(HTTP.Config(), 45*time.Second)) + chaos.ObserveWorkerRecovery(t, HTTP.Config()) +} diff --git a/transmite/CMakeLists.txt b/transmite/CMakeLists.txt index f203d8c..869ec4b 100644 --- a/transmite/CMakeLists.txt +++ b/transmite/CMakeLists.txt @@ -34,7 +34,8 @@ aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/source src_files) # 5. 声明目标及依赖 add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs}) -target_link_libraries(${target} -lgflags -lspdlog +find_package(jsoncpp CONFIG REQUIRED) +target_link_libraries(${target} jsoncpp_lib -lgflags -lspdlog -lglog -lfmt -lbrpc -lssl -lcrypto -lprotobuf -lleveldb -letcd-cpp-api -lcpprest -lcurl -lamqpcpp -lev diff --git a/transmite/Dockerfile b/transmite/Dockerfile index fca8bd0..373a6ea 100644 --- a/transmite/Dockerfile +++ b/transmite/Dockerfile @@ -1,7 +1,6 @@ # 声明基础镜像来源 ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${UBUNTU_IMAGE} -ENV LD_LIBRARY_PATH=/im/depends # 声明工作路径 WORKDIR /im RUN mkdir -p /im/logs &&\ @@ -13,5 +12,6 @@ COPY ./build/transmite_server /im/bin # 将可执行程序依赖，拷贝进入镜像 COPY ./depends/ /im/depends/ RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/* +ENV LD_LIBRARY_PATH=/im/depends # 设置容器的启动默认操作 --- 运行程序 CMD /im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf diff --git a/transmite/source/transmite_server.cc b/transmite/source/transmite_server.cc index 2cca554..f5fbaa2 100644 --- a/transmite/source/transmite_server.cc +++ b/transmite/source/transmite_server.cc @@ -30,7 +30,7 @@ DEFINE_bool(redis_keep_alive, true, "Redis 长连接"); DEFINE_int32(redis_pool_size, 8, "Redis 连接池大小"); DEFINE_string(mq_user, "root", "消息队列服务器访问用户名"); -DEFINE_string(mq_host, "127.0.0.1:5672", "消息队列服务器访问地址"); +DEFINE_string(mq_host, "127.0.0.1", "RabbitMQ hostname (port is fixed at 5672)"); // publisher-only：exchange 必须与 message 服务 mq_msg_exchange 一致 DEFINE_string(mq_msg_exchange, "chat_msg_exchange", "持久化消息的发布交换机名称（FANOUT，必须与 message.mq_msg_exchange 完全一致）"); DEFINE_string(mq_msg_queue, "", "publisher-only：留空，避免声明孤儿队列"); diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h index 86130e7..148b7b6 100644 --- a/transmite/source/transmite_server.h +++ b/transmite/source/transmite_server.h @@ -32,6 +32,8 @@ #include "transmite/transmite_service.pb.h" #include #include +#include +#include DECLARE_int32(rate_limit_user_max); DECLARE_int32(rate_limit_session_max); @@ -177,15 +179,19 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService state.status == IdempotencyStatus::Persisted) { auto persisted_msg = select_existing_message_by_client_msg_( uid, client_msg_id, rid, static_cast(controller)); + if (!persisted_msg.has_value() || persisted_msg->message_id() == 0 || + persisted_msg->seq_id() == 0) { + // This request does not own the existing acceptance guard. + // Keep it so a retry can recover the original durable result + // without allocating another sequence or publishing again. + return err_response(rid, chatnow::error::kSystemUnavailable, + "duplicate request in flight"); + } LOG_INFO("请求ID: {} - 命中幂等 client_msg_id={} 直接返回旧消息", rid, client_msg_id); response->mutable_header()->set_request_id(rid); response->mutable_header()->set_success(true); - if (persisted_msg.has_value()) { - response->mutable_message()->CopyFrom(*persisted_msg); - } else { - response->mutable_message()->set_message_id(state.message_id); - } + response->mutable_message()->CopyFrom(*persisted_msg); return; } if (state.status == IdempotencyStatus::Corrupt) { @@ -338,7 +344,8 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService std::make_shared>(false); try { std::map _mq_headers; - ::chatnow::mq::mq_inject_trace_headers(_mq_headers); + // Capture the authenticated trace explicitly at the asynchronous MQ boundary. + if (!auth.trace_id.empty()) _mq_headers[::chatnow::mq::kTraceHeader] = auth.trace_id; _publisher->publish_confirm(internal_msg.SerializeAsString(), _mq_headers, [async_done, response, rid, done_called, redis, idem_key, msg_id](PublishStatus status, const std::string &mq_msg) { @@ -413,6 +420,15 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService randomized_ttl(std::chrono::seconds(60))); }; + auto fetch_uncached_members = [&]() -> MembersResult { + // An unavailable cache cannot authorize from stale L1 state. Consult the + // authoritative service for this request, without publishing an unfenced fill. + auto members = fetch_members_from_conversation_service_( + chat_session_id, rid, caller_cntl); + if (!members) return {}; + return {std::move(*members), false, kUnknownCacheVersion}; + }; + // ① L1 hit → fast path if (auto local = try_local()) return *local; @@ -437,6 +453,7 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService if (!snap.stable) { metrics::g_members_cache_snapshot_race_total << 1; release_guard(); + if (!cache_version_is_known(snap.version)) return fetch_uncached_members(); return {}; } auto members = std::move(snap.members); @@ -500,6 +517,7 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService auto snap = _members_cache->list_snapshot(chat_session_id); if (!snap.stable) { metrics::g_members_cache_snapshot_race_total << 1; + if (!cache_version_is_known(snap.version)) return fetch_uncached_members(); return {}; } auto members = std::move(snap.members); @@ -714,6 +732,10 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService chatnow::message::SelectByClientMsgIdReq req; chatnow::message::SelectByClientMsgIdRsp rsp; brpc::Controller cntl; + // Leave time for Gateway to return the existing retryable envelope. + // A lookup is read-only; recovery retries remain the client's decision. + cntl.set_timeout_ms(1000); + cntl.set_max_retry(0); if (caller_cntl) chatnow::auth::forward_auth_metadata(caller_cntl, &cntl); req.set_request_id(rid); req.set_client_msg_id(client_msg_id); @@ -766,11 +788,8 @@ class TransmiteServer if(_watchdog_thread.joinable()) _watchdog_thread.join(); } - /* M4: 搭建RPC服务器，并启动服务器 - * - 启动 lease_lost watchdog：每 1s 轮询 worker_id 租约状态， - * 一旦发现租约丢失（其他实例占走了同一 worker_id），主动停服 + abort， - * 避免 SnowflakeId 用已被别人占据的 worker_id 继续发号产生重号。 - */ + // Poll worker ownership every second and terminate the entire process on + // loss. An in-flight handler must not keep allocating IDs under that worker. void start() { if(_worker_allocator || _etcd_worker_allocator) { _watchdog_running.store(true); @@ -780,13 +799,12 @@ class TransmiteServer if (_worker_allocator) lost = _worker_allocator->lease_lost(); if (!lost && _etcd_worker_allocator) lost = _etcd_worker_allocator->lease_lost(); if (lost) { - // brpc Stop(0) 不会打断 in-flight handler； - // 在 worker_id 已被别人占走的状态下，每多发一个雪花 ID 都 - // 必然撞向对端实例，污染 message 主键唯一约束。 - // → 唯一正确语义：立刻 abort，让所有 bthread 一起死。 - LOG_ERROR("worker_id 租约丢失，立即 abort 防雪花 ID 重号"); - try { if(_reg_client) _reg_client->unregister(); } catch(...) {} - std::abort(); + // abort() can fall through to SIGSEGV as container PID 1. + // Do not wait for network unregister or destructors while + // RPC handlers can still mint IDs. Registry TTL/restart + // handles the stale registration after this fail-stop. + std::fputs("worker_lease_lost action=exit code=1\n", stderr); + std::_Exit(EXIT_FAILURE); } std::this_thread::sleep_for(std::chrono::seconds(1)); } @@ -891,7 +909,7 @@ class TransmiteServerBuilder } _exchange_name = exchange_name; _routing_key.clear(); // publisher-only：FANOUT 路由忽略 routing_key，固定空串 - std::string amqp_url = "amqp://" + user + ":" + password + "@" + host + ":5672/"; + std::string amqp_url = make_amqp_url(user, password, host); _mq_client = std::make_shared(amqp_url); declare_settings settings { .exchange = exchange_name,