Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b02522a
fix(ci): integrate compose runtime and restore linux gates
ULookup Sep 12, 2026
1b41db5
fix(ci): enable proto3 optional fields on ubuntu runners
ULookup Sep 12, 2026
81bfede
fix(ci): resolve jsoncpp target scope and minio image source
ULookup Sep 12, 2026
1b74a89
fix(docker): isolate package installation from service libraries
ULookup Sep 12, 2026
1dc17ca
fix(compose): probe etcd without a shell in the image
ULookup Sep 12, 2026
a7d37df
fix(ci): install the compiler required by ubuntu odb
ULookup Sep 12, 2026
09b6cbe
fix(tests): resolve runtime configuration from suite directories
ULookup Sep 12, 2026
64c908c
fix(tests): handle read-only redis replicas during cleanup
ULookup Sep 12, 2026
6133368
fix(tests): align smoke fixtures with runtime identity and presence
ULookup Sep 12, 2026
23a3f78
fix(build): propagate jsoncpp headers to all service targets
ULookup Sep 12, 2026
e901242
fix(build): link message logging symbols explicitly
ULookup Sep 12, 2026
80d95d8
fix(ci): enable amqp tcp and load functional suite configuration
ULookup Sep 12, 2026
e6b0c3c
fix(runtime): restore minio upload and clustered presence
ULookup Sep 12, 2026
77afd72
fix(build): link the logging dependency for common service headers
ULookup Sep 12, 2026
a75af29
fix(tests): align functional fixtures with the running stack
ULookup Sep 12, 2026
2603ed7
fix(tests): preserve issued device identity and bounded state checks
ULookup Sep 12, 2026
494fc13
test(chat): verify device coexistence and personal deletion
ULookup Sep 12, 2026
2e3e96f
fix(runtime): repair CI functional and reliability regressions
ULookup Sep 12, 2026
be8854f
fix(ci): secure and bound readiness probes
ULookup Sep 13, 2026
3596568
fix(auth): classify expired JWTs after signature verification
ULookup Sep 13, 2026
db48940
fix(discovery): refresh RPC endpoints when service addresses change
ULookup Sep 13, 2026
4d2db8b
test(reliability): preflight address faults on an explicit test subnet
ULookup Sep 13, 2026
2264f43
fix(ci): keep contract and fault-controller gates explicit
ULookup Sep 13, 2026
86145ab
test(reliability): skip reserved Docker address candidates
ULookup Sep 13, 2026
f44dce8
test(reliability): prove Redis circuit phases and restore RPC readiness
ULookup Sep 13, 2026
6ee40be
fix(transmite): reject incomplete idempotent message responses
ULookup Sep 13, 2026
9139063
fix(discovery): recover expired service registration leases
ULookup Sep 13, 2026
fb2bb88
fix(discovery): bound registration renewal and shutdown calls
ULookup Sep 13, 2026
d698079
fix(build): align default targets with Go test architecture
ULookup Sep 13, 2026
8f1b345
fix(ci): preserve the existing contract gate command
ULookup Sep 13, 2026
ddd3fa9
test(ci): enforce the native default build graph gate
ULookup Sep 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 41 additions & 3 deletions .agents/skills/chatnow-orienting/references/core-flows.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,20 @@

Target version: `3.0-dev`
Status: Current
Verified: 2026-07-22
Verified: 2026-09-13

These are current-state flows for the `3.0-dev` line. Re-verify affected symbols at the target commit and keep proposals in a separate section.

The release contract confirmed on 2026-09-13 preserves concurrent authenticated devices for one account. Logging in on another device must not invalidate the earlier device merely because it is newer. `DeleteMessages` removes only the authenticated caller's timeline references; it does not recall the shared message or remove another user's history. SC-07 and FN-MS-10 verify these boundaries with real sessions and persisted timelines.

## HTTP request flow

**Flow:** Client -> Gateway HTTP -> discovered brpc service -> Protobuf response envelope.

- Entry/contracts: `gateway/source/gateway_server.h`; affected `proto/*/*_service.proto`; `proto/common/envelope.proto`.
- Boundary: Gateway parses Protobuf, authenticates JWT except whitelisted Identity routes, derives `user_id`/`device_id`/JTI, adds trace context, and serializes `RpcMetadata` into the brpc attachment.
- Discovery: `ServiceManager` resolves service instances through etcd; calls are synchronous with route-specific timeouts.
- Discovery: `ServiceManager` resolves service instances through etcd; numeric endpoints retain direct brpc channels, while hostname endpoints use brpc's periodic DNS naming service. The same registration value can therefore recover after an IP change without restarting callers. Calls retain `baidu_std` and existing route-specific timeouts; DNS discovery does not add application retries or delivery guarantees.
- Registration: `Registry` grants a 30-second lease and replaces it every ten seconds using bounded grant/put/revoke calls. It publishes the remembered key/value under the fresh lease before revoking the previous one; repeated PUT values retain existing channels. Each etcd operation has a two-second timeout; unsuccessful replacement retries after one second. This avoids the pinned keepalive stream's unbounded cancellation path. Shutdown serializes with publication, stops and joins the worker, then revokes only its owned lease, so an old instance cannot delete a newer instance's registration. Initial registration failure retains its startup failure behavior. RL-DISCOVERY-02 expires the lease and requires registration and account RPC to recover in the same Identity and caller processes.
- Failure: malformed input, unavailable backends, and RPC timeouts are translated into a Protobuf `ResponseHeader`; retries remain a client decision unless a flow states otherwise.
- Tests: `tests/bvt`, affected `tests/func`, `tests/func/auth_middleware_test.go`, `tests/func/security_test.go`.
- Invariants: client identity fields never override server-derived auth context; services validate required metadata; trace context propagates where supported.
Expand All @@ -21,14 +24,28 @@ These are current-state flows for the `3.0-dev` line. Re-verify affected symbols

**Flow:** Client -> Gateway -> Transmite -> RabbitMQ message exchange -> Message/MySQL -> RabbitMQ push queue -> Push -> WebSocket.

- Entry/contracts: Gateway `/service/message/send`; `proto/transmite/transmite_service.proto`; `proto/message/message_internal.proto`; `proto/push/notify.proto`.
- Entry/contracts: Gateway `/service/transmite/send`; `proto/transmite/transmite_service.proto`; `proto/message/message_internal.proto`; `proto/push/notify.proto`.
- Transmite: validates auth/membership and content, allocates conversation and per-user sequences in Redis, creates a Snowflake message ID, protects `client_msg_id` with a Redis idempotency key, and completes the brpc request after publisher confirm.
- Message: consumes `InternalMessage`, persists the message then per-user timelines in MySQL, treats duplicate message inserts idempotently, publishes push only after persistence, and asynchronously indexes text in Elasticsearch.
- Push: consumes the push event, resolves Redis/local routes, records per-device unacked payloads, sends locally or uses asynchronous cross-instance `PushBatch`, then WebSocket delivery reaches the client.
- Async/recovery: RabbitMQ and WebSocket delivery are at-least-once. Message uses Redis Push/ES outboxes and reapers for publish failures; Push uses a cross-instance outbox. Consumer redelivery and duplicate delivery require idempotent effects.
- Tests: `tests/bvt/message_test.go`, `tests/func/transmite_test.go`, `tests/func/message_test.go`, `tests/func/ws_notify_test.go`, `tests/func/scenarios_test.go`, `tests/perf/send_msg_test.go`, `tests/perf/sync_test.go`.
- Invariants: Message/MySQL is the stored-message source of truth; persistence precedes normal push publication; `request_id`, `client_msg_id`, message ID, conversation sequence, and user sequence have distinct roles; do not claim exactly-once delivery.

An `accepted:<message_id>` or `persisted:<message_id>` Redis hit is a deduplication guard, not a complete message response. Transmite retrieves the original through the authenticated `SelectByClientMsgId` RPC with a one-second timeout and no transport retries. A missing/unreadable message or zero identity/sequence returns the existing `9002` / `duplicate request in flight` envelope without a message. The caller does not own that prior guard and must not delete it, allocate sequences, or republish. Once Message persists the original, a client retry returns its complete durable result. The pending-record lookup uses the same RPC budget. Cache formats and the 24-hour TTL remain unchanged; mixed-version older Transmite instances retain their old incomplete-success behavior until upgraded. FN-TM-02, RL-MESSAGE-01, and SC-06 cover the unreadable, unavailable, and recovered paths.

Message commits the conversation `max_seq` watermark in the same MySQL transaction as the message and timeline inserts. The watermark never decreases when deliveries arrive out of order; Conversation metadata writers preserve the latest committed value under a row lock. The Message database principal therefore needs `SELECT, UPDATE` on `conversation`.

Log-context storage must initialize in Release builds as well as Debug; initialization cannot depend on assertions. Transmite copies the authenticated RPC trace into MQ headers at publish time. Push invalidates its local route cache while binding a newly authenticated device under the same per-user lock used by route fills; it does not cache empty routes. Dismissed conversations are rejected by `GetMemberIds` before consulting cached membership.

Redis availability classification includes the pinned client's exhausted shard-refresh wrapper, while Redis command errors remain separate. An unavailable member snapshot has an unknown version: Transmite queries Conversation for that request and does not publish the result into L1/L2 without a version fence. Known-version races retain retry behavior. Sequence allocation and Unacked persistence remain Redis truth-source operations and fail unavailable during an outage.

Both MQ consumer overloads translate `NackRequeue` and callback exceptions to `reject(deliveryTag, AMQP::requeue)`. The library parameter is a bitmask; passing a boolean does not request requeue. `NackDiscard` uses zero flags. Push must requeue on Unacked persistence failure and deliver only after durable persistence succeeds.

## Business notifications

Relationship emits friend-request and accepted-request notifications, and Conversation emits creation notifications through a bounded Push `PushBatch` RPC after the domain write commits. Auth metadata and trace are forwarded. These online notifications are best effort, with no new durable retry guarantee. A creation broadcast omits the creator's `self` member state; each recipient obtains its own state through Conversation APIs. Tests: FN-WS-02/03/04.

## Delivery ACK convergence

**Flow:** Client WebSocket `MSG_PUSH_ACK` -> Push validation -> Redis unacked removal -> asynchronous Message `UpdateReadAck` -> MySQL convergence.
Expand All @@ -52,6 +69,8 @@ These are current-state flows for the `3.0-dev` line. Re-verify affected symbols
- Tests: `tests/bvt/auth_test.go`, `tests/func/identity_test.go`, `tests/func/auth_middleware_test.go`, `tests/func/security_test.go`, `tests/func/scenarios_test.go`, and `tests/func/ws_notify_test.go` (`FN-WS-09`).
- Invariants: only Identity issues/refreshes tokens; access and refresh token purposes remain distinct; downstream identity comes from verified claims and forwarded metadata, not request bodies; Push admission must resolve revocation before publishing any authenticated-session side effect.

The shared `JwtCodec` reports `1002` for a correctly signed expired JWT regardless of its age. In the expired branch, signature verification and strict `iat`/`nbf` checks run before the unconditional expiration rejection; that secondary verifier does not check `exp` again. This does not extend token validity or return authenticated claims. Invalid signatures, unknown keys, and invalid future time claims remain rejected with `1003`. FN-AM-07 exercises Identity refresh responses and Gateway rejection using synthetic signed tokens.

## Runtime secrets

### Current
Expand All @@ -71,18 +90,29 @@ Redis authentication, dynamic reload, automatic rotation, and additional credent

**Flow:** Apply/init -> presigned MinIO upload -> complete -> MySQL metadata/quota -> authenticated download request -> presigned MinIO GET.

Ordinary PUT URLs use the standard S3 presigner with the headers returned to the client, without implicit SSE-C headers. `use_path_style` disables AWS virtual addressing for internal MinIO hostnames; internal object verification and public presigning can use different endpoints.

- Entry/contracts: Gateway Media routes; `proto/media/media_service.proto`; `media/source/media_server.h`; `media/source/upload_handler.hpp`; `media/source/multipart_handler.hpp`; `media/source/download_handler.hpp`.
- Stores: MinIO holds bytes; MySQL holds `media_file`, blob-ref/dedup, multipart, and per-user quota state; Redis coordinates cleanup/locks where implemented.
- Boundaries: clients upload/download directly with short-lived presigned URLs. Apply validates size/MIME/hash/quota and records pending metadata; complete verifies object existence/size, converges dedup/refcount/quota, and is idempotent for committed files.
- Authorization: Gateway requires authentication. The current download handler permits an authenticated caller with a committed `file_id`; it does not enforce owner or conversation membership, as confirmed by `tests/func/media_test.go`'s other-user case. Do not overstate this boundary as resource-level authorization.
- Async/retry: cleanup handles stale pending, quarantine, and unreferenced object paths; client retries must preserve file/upload identifiers and completion idempotency.
- Tests: `tests/bvt/media_test.go`, `tests/func/media_test.go`, `tests/func/concurrency_test.go`, `tests/func/scenarios_test.go`, `tests/perf/upload_test.go`, `tests/pkg/verify/minio.go`.
- Invariants: service processes metadata rather than normal file bytes; only committed objects are downloadable; MySQL metadata/quota and MinIO object state must converge; preserve dedup and completion idempotency.
- Deduplication shares the stored object, not the file identifier: repeated uploads receive distinct metadata references to the same bucket/object key. Functional checks must validate both the distinct references and shared bytes.

Multipart routes require the same JWT boundary as single uploads. `partNumber` and `uploadId` are included before SigV4 signing. Test content follows the MIME allowlist and uses non-final parts of at least 5 MiB; FN-MD-07 verifies the downloaded bytes, and FN-MD-21 verifies signature tampering is rejected.

Media `FileInfo.public_url` is additive field 6: it contains the canonical configured public prefix plus the committed object's key, and stays empty for private objects. Identity discovers Media and resolves stored avatar file IDs through authenticated `GetFileInfo`; it returns an empty avatar URL when resolution is unavailable. Media configuration is authoritative for the public prefix. FN-ID-08 checks an actual HTTP GET and persisted profile reads.

Speech recognition rejects empty/unaligned PCM16 input. Until an ASR backend is integrated, valid input returns an explicit unavailable error rather than empty success (FN-MD-17/18/23).

## Presence and typing

**Flow:** Push WebSocket lifecycle -> Redis presence/routes -> Presence aggregation/subscriptions -> Presence or Push notification -> WebSocket.

Push routes each per-device write pipeline by its Redis key and stores enum names (`ONLINE`/`OFFLINE`). Presence accepts those names and legacy numeric enum values; malformed states are ignored. Registration alone does not create an online connection. BVT-018 opens an authenticated socket before asserting ONLINE.

- Entry/contracts: `push/source/push_server.h`; `proto/presence/presence_service.proto`; `presence/source/presence_server.h`; `proto/push/notify.proto`.
- Ownership: Push owns connections and route binding, writes per-device online/offline/heartbeat TTL state, and emits lifecycle notifications. Presence aggregates Redis device state, manages subscription sets and typing TTLs, and calls Push for delivery.
- Async/retry: lifecycle and typing notifications are fail-soft asynchronous Push RPCs; Redis TTL supplies eventual offline behavior; multi-instance fanout uses Push routing and cross-instance RPC.
Expand All @@ -92,3 +122,11 @@ Redis authentication, dynamic reload, automatic rotation, and additional credent
## Architecture-change synchronization

Any change to these paths, ownership boundaries, stores, protocols, topology, ordering, retry, idempotency, trust, or failure semantics must update this reference and any affected `technology-stack.md` or `repository-map.md` content in the same PR.

## Disposable CI startup

CI builds the nine native services once in the pinned Ubuntu builder, restores that artifact into each fresh test checkout, generates synthetic credentials, and runs Compose initialization before semantic readiness. MySQL, Redis Cluster, RabbitMQ, and MinIO initialization must converge before application services and runtime tests proceed. Existing environment files or persisted data cause test bootstrap to fail closed. Runtime results must be reported separately from static contracts.

Reliability RL-05 runs in its own disposable stack with Transmite user/session limits of 8/40 per minute. Its bounded outage burst exercises local fallback without depending on exhausting production defaults (600/3000). Push outage tests use the authenticated device ID and prohibit delivery of the particular unpersisted marker, while allowing redelivery of older durable messages. Multi-device login and caller-only deletion semantics remain unchanged.

The Redis fault pauses processes without withdrawing container DNS. Its recovery evidence does not cover stop/recreate, resolver failure or topology changes; those remain separate from the tested circuit behavior.
Loading
Loading