[ImgBot] Optimize images#79
Closed
imgbot[bot] wants to merge 314 commits intomainfrom
Closed
Conversation
Release workflow (triggered by v* tags): - Builds binaries for linux/amd64, linux/arm64, darwin/amd64, darwin/arm64 - Smoke tests: verifies binaries execute, registry starts/stops, pilotctl responds - Integration harness: spins up registry + beacon + 2 daemons, verifies info/health - Creates GitHub release with tar.gz archives and SHA-256 checksums - Pre-release detection for -rc/-beta tags Skip 5 tests in CI that fail due to GitHub Actions runner constraints: - TestIPv6EndToEnd: IPv6 UDP routing unavailable - TestGracefulShutdown: deregister timing race - TestWebhook_NodeDeregistered: webhook delivery timing - TestTunnelEncryptionBackwardCompat: encryption fallback timing - TestRegistryReplication: standby persistence timing All 5 pass locally — skipped only when CI env var is set.
PILOT_RC=1 curl -fsSL https://pilotprotocol.network/install.sh | sh When set, the installer fetches the newest release (including pre-releases) instead of only the latest stable release. Default behavior unchanged.
- Replace static sitemap.xml with build-time generated sitemap.xml.ts that auto-includes all blog posts and doc pages - Add publish-worker (Cloudflare Worker) with HMAC-SHA256 auth, payload validation, GitHub Git Data API atomic commits, and auto-generated OG banner images via Gemini + Browser Rendering - Add deploy-website.yml GitHub Action to build and deploy the website to Cloudflare Pages on pushes to web/** - 63 tests across auth, template, banner, and handler modules
- /for/mcp: Landing page targeting MCP users — "Give your MCP servers a network" - /for/p2p: Landing page targeting P2P builders — "Direct peer-to-peer for AI agents" - Blog: Connecting MCP Servers to Agents Across Any Network - Blog: Peer-to-Peer Agent Communication: No Server Required - Both landing pages emphasize one-command install and zero config
… enrichment, security hardening - Registry: add admin_token fallback for set_hostname, set_visibility, set_tags, set_task_exec, set_key_expiry (console can manage nodes without node signatures) - Registry: add in-memory audit ring buffer (1000 entries) with get_audit_log handler - Registry: enrich list_nodes with public/tags fields for enterprise members - Registry: add requireAdminTokenLocked to avoid deadlock when handlers hold s.mu.Lock - Registry client: add SetHostnameAdmin, SetVisibilityAdmin, SetTagsAdmin, SetTaskExecAdmin, SetKeyExpiryAdmin, GetAuditLog methods - Security: phase 1 audit fixes (H3 registry auth, H4 replication auth, H12 per-port accept, M12 P2P handshake signing) - Tests: update enterprise test suite, add enterprise_gate_test.go
Replace Research and Playground nav items with a Solutions dropdown containing links to /for/mcp and /for/p2p landing pages. Mobile menu updated with grouped sections.
…layground Apply consistent navbar to index.html, plans.html, BlogFooter.astro: replace Research and Playground with Solutions dropdown (MCP, P2P). Update footer links site-wide.
- Emit polo_score.updated audit event on score delta updates - Emit polo_score.set audit event on absolute score changes - Emit network.enterprise_changed audit event on enterprise flag toggle
- Emit node.re_registered audit event for all 4 re-registration modes (existing_identity, reclaimed_identity, owner_key_update, owner_reclaim) - Emit node.reaped audit event when stale nodes are removed by background reaper
…xtraction, add audit API tests - Add polo_score, tags, public fields to backbone (network 0) node listing - Add polo_score to network member listing - Fix appendAudit to extract network_id from attrs (was only in details string) - Exclude network_id from details string to avoid duplication - Add TestAuditLogAPI: tests get_audit_log endpoint, filtering, auth, ordering - Add TestAuditLogRingBuffer: verifies 1000-entry cap and default limit
- TestEnterpriseToggle: verifies enable/disable enterprise flag gates RBAC operations and emits audit events for each toggle - TestPoloScoreConcurrent: 50 workers each increment by +1, verifies final score is exactly 50 (mutex serialization)
TestAdminNodeManagement exercises SetHostnameAdmin, SetVisibilityAdmin, SetTagsAdmin, SetTaskExecAdmin, and SetKeyExpiryAdmin via admin_token. Verifies both successful operations and wrong-token rejection.
- Persist audit log entries in the registry JSON snapshot - Restore audit log on load (separate auditMu lock, no deadlock) - Fix appendAudit to properly extract network_id from attrs - Add TestAuditLogPersistence: verifies entries survive restart
- handleDeregister now accepts admin_token fallback (same H3 pattern) - Add DeregisterAdmin client method - Add TestAdminDeregister: verifies admin force-deregister and token rejection
applySnapshot() was missing restoration of critical enterprise fields: - Node: Tags, PoloScore, TaskExec, LANAddrs, KeyMeta (lifecycle) - Network: AdminToken, Enterprise flag, MemberRoles (RBAC), Policy - Collections: InviteInbox, AuditLog Also adds audit log and invite inbox to snapshotJSON() for replication, adds read-only standby whitelist for enterprise queries (get_polo_score, get_key_info, get_network_policy, get_audit_log), enriches list_nodes with member roles, and adds TestReplicationEnterpriseData covering full enterprise data round-trip through replication + failover.
… audit tests Stale node reap audit now includes last_seen_ago duration and network count. New TestAuditMemberOperations validates promote/demote/kick emit audit events via the ring buffer API. New TestAuditEnterpriseToggle validates enterprise flag toggle and policy change audit events.
Heartbeat now rejects nodes with expired keys (key.expired_heartbeat_blocked audit event). SetKeyExpiry uses s.now() for consistent clock override in tests. Delete network audit now includes member count and enterprise flag. New tests: TestKeyExpiryEnforcementOnHeartbeat (clock advance past expiry), TestKeyExpiryWarningOnHeartbeat (24h warning), TestAuditDeleteNetworkEnriched.
Network deletion now removes pending invites for the deleted network. Key expiry can be cleared with expires_at="" or "never", emitting key.expiry_cleared audit event. Polo score is bounded to +/-1000000 with clamping on delta overflow. New tests: TestClearKeyExpiry, TestDeleteNetworkCleansInvites, TestPoloScoreBounds. Updated TestPoloScoreEdgeCases for clamping behavior.
…test TestReplicationTokenValidation verifies H4 fix: standby with wrong token is rejected and receives no data, while correct token gets snapshots. TestListNodesEnrichedFields verifies that list_nodes response includes role, polo_score, tags, public, and last_seen for enterprise members.
Add RBAC test cases for SetNetworkPolicy, PromoteMember, DemoteMember, and DeleteNetwork using per-network admin tokens. Verifies cross-network token rejection for each operation.
- Add 30-day invite TTL: expired invites filtered on poll (with audit), rejected on respond with clear error message - Initialize RBAC roles when toggling enterprise=true on existing network: first member gets owner, others get member - Pass "has expired" errors through to client (was generic "request failed") - Enrich tags.changed audit with tags_count - Add TestInviteTTLExpiry, TestEnterpriseToggleRBACInit, TestInviteRespondAfterNetworkDelete
- Leave network now cleans up pending invites for the leaving node+network - Kick member now cleans up pending invites for the kicked node+network - Hostname audit includes old_hostname and new_hostname - Visibility audit includes old_public and new_public - Promote/demote audit includes old_role and new_role - Add TestLeaveNetworkCleansInvites, TestKickMemberCleansInvites, TestAuditEnrichedHostnameVisibility, TestAuditEnrichedPromoteDemote
- Deregister now cleans up invite inbox and RBAC roles for all networks - Emit network.owner_lost audit when owner deregisters with remaining members - Enrich node.deregistered audit with network count - Unify expired error passthrough (covers key expiry and invite expiry) - Add TestDeregisterCleansEnterprise, TestDeregisterOwnerAudit, TestKeyRotationPreservesExpiry (documents intentional behavior)
- New transfer_ownership handler: current owner can transfer to any member - Old owner demoted to admin, new owner promoted to owner - Enterprise gate required (non-enterprise networks rejected) - Validates: must be owner, target must be member, cannot transfer to self - Client method: TransferOwnership(networkID, ownerID, newOwnerID, adminToken) - Audit event: network.ownership_transferred with old/new owner + old role - Tests: TestTransferOwnership (full flow + edge cases), TestTransferOwnershipNonEnterprise
… tests - Add audit event for beacon registration (beacon.registered) - Add get_member_role and check_trust to standby read whitelist - Add TestTransferOwnershipThenReTransfer: verifies chain of A→B→C transfers with audit trail - Add TestMaxMembersOnInviteAccept: verifies max_members policy enforcement when accepting invite at network capacity
srv.ListenAndServe spins up the bind in a goroutine; macOS-latest CI is slow enough that Dial fired before the listener accepted and saw `dial: daemon: connection refused`. Retry Dial briefly so the server goroutine has a chance to bind.
- tests/integration/local/logs/{.queue,.results,.timing.tsv}: per-run
output of run-all.sh.
- tests/integration/local/results/{gw_last_task.txt,p2p_summary.txt}:
per-run test output artifacts.
- web/src/pages/blog/.astro/: astro content-type generation cache.
Files stay on disk (regenerated by the tooling); only removed from
tracking. Added matching .gitignore rules to prevent re-commit.
Each ships as an open-join, default-deny network with the same service-tag-gated traffic shape as data-exchange-policy: peers tagged "service" can connect/dial freely on data ports; everyone else gets echo (port 7) and text (port 1000), and can receive files (port 1001) from service nodes. Categories: science, geo, reference, health, government, news, finance, dev, transit, sports, academic, language, security, food, entertainment, knowledge, flights, weather, traffic, vehicles, gov-finance, economics, climate, packages, culture, data, books, music, space, nature. All 30 pass TestShippedNetworkBlueprintsLoadAndValidate.
- 30 open-data networks (academic, books, climate, culture, data, dev, economics, entertainment, finance, flights, food, geo, gov-finance, government, health, knowledge, language, music, nature, news, packages, reference, science, security, space, sports, traffic, transit, vehicles, weather) switch from service-tag-gated to default-allow with explicit open-connect / open-dial / open-datagram rules. Any peer can talk to any other peer on any port. - SHIPPED.md catalogs all 64 networks (33 reputation/membership policies + 30 open-data + data-exchange-policy reference). - Untrack .claude/PROBLEM-REGISTRY.md (already in .gitignore via .claude/). The apply-networks workflow will re-provision each of the 30 changed configs against the production registry; findOrCreateNetwork looks up by name so existing IDs (44–73) are reused, only the expr_policy is replaced.
When a peer restarts (identity preserved, in-memory crypto lost) we keep sending data with the now-stale shared key. The peer drops those ciphertexts because they have no matching key and replies with a fresh key_exchange. Our handleAuthKeyExchange installs the new key, but the data we just sent under the old key was already lost — fire-and-forget RPCs (task submit, task send-results) had no recovery path. Fix: per-peerCrypto plaintext ring buffer. - recordSalvage(pc, plaintext) is called in SendTo before encryptFrame. Bounded at salvageMaxEntries=32 entries × salvageMaxAge=5s. Caller's buffer is copied, not referenced. Worst-case ~48 KiB per peer, ~48 MiB across maxCryptoPeers=1024. - replaySalvage(oldPC, newPC, peerNodeID, addr) is called in handleAuthKeyExchange and handleKeyExchange when keyChanged=true. Re-encrypts each non-aged plaintext from the old buffer with the new key and ships via writeFrame. The old buffer is drained on entry so a reciprocal rekey doesn't double-replay. - Webhook event tunnel.desync_salvage fires with the replayed count for observability. Tests: - 7 unit tests in pkg/daemon/tunnel_desync_salvage_test.go cover the copy-not-reference invariant, size + age bounds, nil-arg branches. - tests/integration/local/test_tunnel_desync_recovery.sh: docker compose restart agent-b mid-tunnel, then submit a task a->b. Without salvage the submit either EOFs at the door or stays NEW forever; with salvage the task lands and completes within 1s. Verified passing 7/7. Sister test test_sender_clean_restart_midflight hit the happy path (not the previously-added P1-010 tolerance fallback). Closes the data-loss half of P1-010. The tunnel-state half (5-min "peer marked for relay" loop under sustained packet loss) is a separate symptom; this fix addresses the silent fire-and-forget loss that broke task pipeline tests.
Companion to the salvage fix (commit 7f76e75 / a4af0f9). The salvage half recovered data lost across a peer-initiated rekey; this half addresses the case where the rekey itself fails to converge under sustained loss. Symptom (per problem registry): every ~5 min, agent-a logs "encrypted packet from node but no key peer_node_id=N rekey_sent=true" followed 2s later by "peer marked for relay node_id=N", with no successful "encrypted tunnel established" until compose down -v. The 5-min cadence matches RelayProbeInterval — A only attempts rekey when its 5-min relay probe encrypts a packet B can't decrypt. If A's key_exchange reply (or B's request) is dropped, the tunnel waits another 5 min. Fix: per-peer pending-rekey state + a sweeper goroutine that retransmits on rekeyRetransmitInterval (4 s) up to maxRekeyAttempts (5 retries = 6 total). Cleared on first successful inbound decrypt from peer. Mechanism: - sendKeyExchangeToNode now calls markPendingRekey on success. - handleEncrypted's success path calls clearPendingRekey and recordInboundDecrypt. - rekeyRetransmitLoop runs every 4 s; rekeyRetransmitTick is the per-tick body, split out for direct unit testing. - Webhook event tunnel.rekey_gave_up fires when a peer hits the cap. Inspection helpers (lastInboundDecrypt + inboundDecryptStale) are included for future use; the symmetric-reply gate loosening I tried first caused an infinite key_exchange ping-pong on initial handshake (reverted) — the retransmit loop alone is sufficient. Tests: - 9 unit tests in tunnel_rekey_retransmit_test.go cover initial marking, attempt-bumping, clear, the two stale states, and the retransmit-tick give-up + skip-fresh paths. - Existing pkg/daemon and tests/ Go suites green. - Integration: test_tunnel_desync_recovery 7/0, test_sender_clean_restart_midflight 6/0, test_chaos_packet_loss 9/0, test_force_relay_task 4/0. Constants added: rekeyRetransmitInterval = 4 s maxRekeyAttempts = 5 keyExchangeReplyStaleThreshold = 6 s (helper, not currently gated)
Lightweight Reynolds flocking sim (50 triangles, separation/alignment/cohesion plus a soft edge cushion) rendered in an SVG layered behind the hero copy. Honors prefers-reduced-motion; opacity tuned low so the flock reads as ambient movement, not a graph.
The 32-entry cap caused the rekey-replay path to burst-flood the receiver's nonce window: under typical conditions the dataexchange retransmit layer fills salvage during the rekey-pending window, and on rekey all 32 frames go out in a tight loop. The receiver's pc[a] was just freshly installed (maxRecvNonce=0), so the 32 nonces (1..32) arrived out of order on the wire and the replay-window check started flagging the late-arriving lower nonces as replays. Net effect: salvage's whole purpose (recover dropped data on rekey) defeated by its own batch size. 4 covers the realistic shape — one task submit + send-results + a couple of dataexchange retries — without exceeding what a fresh peerCrypto's window can absorb in arbitrary order. Memory drops to ~6 KiB / peer (was 48 KiB), worst case ~6 MiB across maxCryptoPeers. Empirically with this cap: - test_tunnel_desync_recovery: 7/0 (was hanging on direct-path-silent) - test_midrekey_send_file: 5/0 (was 4/1) - test_midrekey_send_message: 6/0 (was 5/1) - test_midrekey_task_results: 6/0 (was 4/2) - test_midrekey_task_submit: 5/0 (was 3/2) - test_peer_restarted_send_file: 4/0 (was 3/1) - test_peer_restarted_send_message: 5/0 (was 3/2) The midrekey_* tests are documented P1-009 regression repros — this fix closes them too.
Bug pinned by TestHandleKeyExchangeDuplicatePreservesCryptoState. When a peer sent a SECOND key_exchange with the SAME X25519 ephemeral pubkey (e.g. their reply crossed our retransmit on the wire), handleAuthKeyExchange and handleKeyExchange unconditionally replaced tm.crypto[N] with a freshly-derived peerCrypto. The freshly-derived pc has the same shared secret (pubkeys unchanged → same ECDH output) but RESET nonce counter, RESET maxRecvNonce, and EMPTY replay bitmap. Net effect: subsequent encrypted sends from us used counter 1,2,3... on the new pc, while peer's pc[us] still had a high maxRecvNonce from before. Peer dropped our packets as "outside replay window" or, if a counter happened to land on a clear bit before the reset became visible, "replay detected." Data plane silently broke under the harmless event of receiving a duplicate handshake. The fix: replace tm.crypto[N] only when there is no existing entry OR the peer's pubkey actually changed. The shared secret is deterministic from the pubkey pair; preserving the existing pc keeps the nonce monotonic and the replay window valid. This was the same fix I tried earlier this session (and reverted) because it broke other tests. Those failures were caused by the salvage cap of 32 — large replay batches were running over the receiver's nonce window. After 5de736f reduced the cap to 4, the salvage replay batch is small enough that the duplicate-key_exchange preservation fix becomes safe. Tests: - 2 new unit tests in tunnel_dup_keyexchange_test.go cover the preserve-on-duplicate and replace-on-real-rekey paths. - All 10 packages green on go test ./pkg/... ./tests/. - 10 targeted integration tests pass: tunnel_desync_recovery, midrekey_{send_file,send_message,task_results,task_submit}, peer_restarted_{send_file,send_message}, sender_clean_restart_midflight, chaos_packet_loss, force_relay_task.
… CLI Adds a delegated event surface in src/scripts/analytics.ts driven by data-* attributes on call sites. Sixteen events total: - click: cta_click, nav_click, outbound_click (subdomain-aware), modal_open, modal_close (with via=escape|backdrop|button), external_doc_click - engagement: section_view, section_dwell, scroll_depth, page_active_time, live_chip_hover, network_viz_engage, blog_post_read, install_command_copy - friction: web_vital (LCP / CLS / TTFB), dead_click Tagged the homepage, /for/* heads, sitewide nav, blog article body, and the OSI modal close paths. Outbound treats *.pilotprotocol.network as internal. PostHog (heatmaps + session recordings) lazy-loads via dynamic import only after the user clicks Allow on ConsentBanner.astro. Inert without PUBLIC_POSTHOG_KEY env var. Anonymous mode, all inputs masked. scripts/ga-daily.mjs is a zero-dep terminal digest against the GA4 Data API (auths via the existing service-account key). Surfaces totals, top pages, top events, CTA breakdowns, outbound destinations, scroll-depth histogram, section views, top referrers. Plus --source <name> for cohort filtering and --funnel install for end-to-end conversion math. Taxonomy + author guide in web/ANALYTICS.md.
PostHog runs in anonymous mode (person_profiles: 'never', all inputs masked), so no consent gate is needed. Banner is removed; analytics.ts loads PostHog whenever PUBLIC_POSTHOG_KEY is set. Unset the key to disable.
*Total -- 8,186.77kb -> 6,633.46kb (18.97%) /web/public/img/ietf-logo.png -- 356.02kb -> 152.44kb (57.18%) /web/public/blog/scriptorium/chart_context.png -- 60.22kb -> 47.42kb (21.26%) /web/public/blog/scriptorium/chart_tokens.png -- 70.97kb -> 56.88kb (19.85%) /web/public/blog/scriptorium/chart_scale.png -- 78.15kb -> 64.41kb (17.58%) /web/public/blog/banners/scriptorium-replace-agentic-active-research-ready-intelligence.png -- 7,505.02kb -> 6,208.69kb (17.27%) /web/public/blog/scriptorium/chart_latency.png -- 36.40kb -> 30.31kb (16.72%) /web/public/blog/banners/ietf-internet-drafts-pilot-protocol-revision-01.jpg -- 79.98kb -> 73.30kb (8.35%) Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Beep boop. Your images are optimized!
Your image file size has been reduced by 19% 🎉
Details
📝 docs |
repo | 🙋🏾 issues | 🏪 marketplace
~Imgbot - Part of Optimole family