@@ -6,98 +6,90 @@ This guide is for developers working on SandD itself.
66
77- ** Rust** : Install from https://rustup.rs/
88- ** Python 3.8+** : With pip
9- - ** Maturin** : ` pip install maturin `
9+ - ** Docker** : Only for the e2e tests and images
10+
11+ The Makefile creates ` .venv/ ` and installs maturin, pytest and ruff into it as needed.
1012
1113## Project Structure
1214
1315```
1416SandD/
15- ├── server/ # Rust server with PyO3 bindings
17+ ├── protocol/ # Shared message types (sandd-protocol crate)
18+ │ └── src/lib.rs
19+ │
20+ ├── server/ # Controller: Rust WebSocket server + PyO3 bindings
1621│ ├── src/
17- │ │ ├── lib.rs # Python API bindings
18- │ │ ├── server.rs # WebSocket server (axum)
22+ │ │ ├── lib.rs # Crate root
23+ │ │ ├── server.rs # WebSocket server (axum): /ws, /stats, /health
1924│ │ ├── registry.rs # Daemon connection registry
20- │ │ └── protocol .rs # Message protocol
25+ │ │ └── python .rs # Python API bindings (behind the `python` feature)
2126│ └── Cargo.toml
2227│
23- ├── sandd/ # Rust daemon binary
28+ ├── sandd/ # Daemon binary (runs on workers)
2429│ ├── src/
25- │ │ ├── main.rs # Daemon entry point
30+ │ │ ├── main.rs # Entry point, connection loop, tunnel setup
2631│ │ ├── executor.rs # Command execution
2732│ │ ├── session.rs # Interactive sessions (PTY)
28- │ │ └── protocol.rs # Message protocol
33+ │ │ └── snapshot/ # Workspace snapshots
34+ │ ├── benches/ # Criterion benchmarks
2935│ └── Cargo.toml
3036│
31- ├── python/sandd/ # Python package wrapper
32- │ └── __init__.py
37+ ├── python/
38+ │ ├── sandd/ # Python package (Server, models, tunnel keys; AsyncServer is a stub)
39+ │ └── tests/ # Unit, integration and e2e tests
3340│
41+ ├── hack/ # Dockerfiles, compose stacks, install script
3442└── examples/ # Usage examples
3543```
3644
3745## Building
3846
39- ### Quick Build (Development)
40-
4147``` bash
42- # Build everything
43- ./test_build.sh
48+ make dev # Build and install the Python package (debug, editable)
49+ make daemon-build # Build the daemon (debug): ./target/debug/sandd
4450
45- # Or step by step:
46- make dev # Build Python package (debug)
47- make daemon-build # Build daemon (debug)
48- ```
49-
50- ### Release Build
51-
52- ``` bash
5351make release # Python package (optimized)
54- make daemon-release # Daemon binary (optimized)
52+ make daemon-release # Daemon binary (optimized): ./target/release/sandd
5553```
5654
57- ### Manual Build
55+ Run ` make help ` for the full list of targets.
5856
59- ``` bash
60- # Python package
61- maturin develop --release -m server/Cargo.toml
62-
63- # Daemon
64- cargo build --package sandd --release
65- ```
57+ The ` python ` feature of ` server/ ` is off by default, so plain ` cargo build ` and
58+ ` cargo test ` work without a Python interpreter. Maturin turns it on when building the
59+ wheel (see ` [tool.maturin] ` in ` pyproject.toml ` ).
6660
6761## Development Workflow
6862
6963### 1. Make Changes
7064
71- Edit files in ` server/src/ ` , ` sandd/src/ ` , or ` python/sandd/ `
65+ Edit files in ` protocol/src/ ` , ` server/src/ ` , ` sandd/src/ ` , or ` python/sandd/ ` .
7266
7367### 2. Rebuild
7468
7569``` bash
76- # If you changed server/
70+ # If you changed protocol/, server/ or python/sandd /
7771make dev
7872
79- # If you changed daemon /
73+ # If you changed protocol/ or sandd /
8074make daemon-build
81-
82- # If you changed Python wrapper only
83- # (no rebuild needed, it's just a wrapper)
8475```
8576
86- ### 3. Test
77+ ### 3. Try It
8778
8879``` bash
89- # Terminal 1: Start test agent
90- python3 examples/simple_test.py
91-
92- # Terminal 2: Start daemon
93- ./target/debug/sandd --server-url ws://127.0.0.1:8765/ws
80+ # Terminal 1: start a controller
81+ python3 -c "
82+ from sandd import Server
83+ server = Server()
84+ server.wait_for_daemon('dev-1', timeout=60)
85+ print(server.exec('dev-1', 'uname -a').stdout)
86+ "
87+
88+ # Terminal 2: start a daemon
89+ ./target/debug/sandd --server-url ws://127.0.0.1:8765/ws --daemon-id dev-1
9490```
9591
96- ### 4. Run Examples
97-
98- ``` bash
99- python3 examples/agent_example.py
100- ```
92+ See [ examples/] ( ../examples ) for sessions, snapshots and tunnel mode.
10193
10294## Architecture
10395
@@ -130,39 +122,35 @@ file_transfers: Vec<Vec<u8>> // Chunked buffering
130122
131123## Testing
132124
133- ### Unit Tests
134-
135125``` bash
136- cargo test --workspace
126+ make test # Lint + Rust tests + Python tests (no Docker)
127+ make test-e2e # Direct-mode e2e against daemons in Docker (slow)
128+ make test-e2e-tunnel # Tunnel-mode e2e with a Headscale mesh in Docker (slow)
129+ make lint # ruff check
130+ make format # ruff fix + format
131+ make benchmark # Daemon benchmarks (criterion)
137132```
138133
139- ### Integration Tests
140-
141- ``` bash
142- pytest tests/ # (when tests are added)
143- ```
144-
145- ### Manual Testing
146-
147- Use ` examples/agent_example.py ` to test all features.
134+ Python tests are selected by marker: ` e2e ` for Docker-based tests, ` tunnel ` for the
135+ tunnel stack. ` make test ` runs ` -m "not e2e" ` .
148136
149137## Common Tasks
150138
151- ### Adding a New Command Type
139+ ### Adding a New Message Type
152140
153- 1 . Add to ` protocol.rs ` (both server and daemon):
141+ 1 . Add the variant to ` Message ` in ` protocol/src/lib .rs ` (shared by server and daemon):
154142 ``` rust
155- Message :: MyNewCommand { field : String }
143+ MyNewCommand { field : String }
156144 ```
157145
158- 2 . Handle in ` server/src/server.rs ` :
146+ 2 . Handle it in ` server/src/server.rs ` :
159147 ``` rust
160148 Message :: MyNewCommand { field } => {
161149 // Forward to daemon or handle
162150 }
163151 ```
164152
165- 3 . Handle in ` sandd/src/main.rs ` :
153+ 3 . Handle it in ` sandd/src/main.rs ` :
166154 ``` rust
167155 Message :: MyNewCommand { field } => {
168156 // Execute and respond
@@ -171,7 +159,7 @@ Use `examples/agent_example.py` to test all features.
171159
172160### Adding Python API
173161
174- 1 . Add method to ` Server ` in ` server/src/lib .rs ` :
162+ 1 . Add a method to ` Server ` in ` server/src/python .rs ` :
175163 ``` rust
176164 #[pymethods]
177165 impl Server {
@@ -181,7 +169,7 @@ Use `examples/agent_example.py` to test all features.
181169 }
182170 ```
183171
184- 2 . Add wrapper in ` python/sandd/__init__ .py ` :
172+ 2 . Add a wrapper in ` python/sandd/server .py ` :
185173 ``` python
186174 def my_method (self , arg : str ) -> None :
187175 """ User-friendly docstring"""
@@ -195,30 +183,21 @@ Use `examples/agent_example.py` to test all features.
195183RUST_LOG=debug ./target/debug/sandd --server-url ws://127.0.0.1:8765/ws
196184```
197185
198- ** Python side:**
199- ``` python
200- import logging
201- logging.basicConfig(level = logging.DEBUG )
202- ```
203-
204- ** Check WebSocket traffic:**
205- ``` bash
206- # In server
207- RUST_LOG=server=debug python3 examples/simple_test.py
208- ```
209-
210- ## Known Issues & Limitations (MVP)
186+ ** Controller logs:** ` Server(verbose=True) ` (the default) logs at INFO; set
187+ ` RUST_LOG=sandbox_server=debug ` for more.
211188
212- ### Not Implemented
189+ ## Known Issues & Limitations
213190
214- 1 . ** File Transfer** : Protocol defined, daemon just logs
215- - Reason: Deferred for MVP
216- - Fix: Implement actual file I/O in daemon
191+ 1 . ** File transfer is not supported yet.** ` Server.upload_file ` and
192+ ` Server.download_file ` exist, and the protocol messages are defined, but the
193+ feature is unfinished: don't rely on either call.
194+ 2 . ** No daemon authentication.** Any client that speaks ` sandd.v1 ` can register as any
195+ daemon id. Run the controller on a trusted network or in tunnel mode. See
196+ [ ARCHITECTURE.md] ( ARCHITECTURE.md#future-authentication ) .
217197
218198### Warnings
219199
220200- PyO3 ` non_local_definitions ` warning: Safe to ignore (PyO3 macro limitation)
221- - Unused imports/variables: Run ` cargo fix ` to clean up
222201
223202## Performance
224203
@@ -232,21 +211,22 @@ At 200 concurrent daemons:
232211
233212### Before Submitting PR
234213
235- 1 . Run tests: ` cargo test --workspace `
236- 2 . Check formatting: ` cargo fmt --all `
237- 3 . Check lints: ` cargo clippy --all `
238- 4 . Test manually with examples
239- 5 . Update docs if adding features
214+ 1 . Run ` make test `
215+ 2 . Check formatting: ` cargo fmt --all ` and ` make format `
216+ 3 . Check lints: ` cargo clippy --workspace --all-targets `
217+ 4 . Update docs if adding features
240218
241219### Commit Style
242220
243221```
244- Add feature : brief description
222+ feat : brief description
245223
246224More detailed explanation if needed.
247225Include motivation and context.
248226```
249227
228+ Use a ` feat: ` , ` fix: ` or ` chore: ` prefix, as in the existing history.
229+
250230### Adding Dependencies
251231
252232- Keep dependencies minimal
@@ -255,23 +235,23 @@ Include motivation and context.
255235
256236## Release Process
257237
258- 1 . Update version in ` pyproject.toml ` and ` Cargo.toml ` files
259- 2 . Update ` CHANGELOG.md ` (when added)
260- 3 . Build release: ` make release && make daemon-release `
261- 4 . Test release build
262- 5 . Tag: ` git tag v0.x.0 `
263- 6 . Build wheel : ` maturin build --release -m server/Cargo.toml `
264- 7 . Publish: ` maturin publish ` (when ready )
238+ 1 . Update the version in ` pyproject.toml ` , ` protocol/ Cargo.toml` , ` server/Cargo.toml `
239+ and ` sandd/Cargo.toml ` (and the ` sandd-protocol ` dependency version in
240+ ` sandd/Cargo.toml ` )
241+ 2 . Tag and push: ` git tag v0.x.y && git push upstream v0.x.y ` . The ` Release ` workflow
242+ builds the daemon binaries and creates the GitHub release.
243+ 3 . Publish packages : ` make publish ` (wheels to PyPI, needs ` INFTYAI_PYPI_TOKEN ` ; crates
244+ to crates.io )
265245
266246## Troubleshooting
267247
268248### User Issues
269249
270250** Daemon won't connect:**
271- - Check agent URL is reachable : ` curl -v ws ://agent -host:8765/ws `
251+ - Check the controller is up : ` curl http ://controller -host:8765/health `
272252- Verify firewall allows outbound WebSocket connections
273- - Check agent server is running: ` ps aux | grep python `
274253- Check daemon logs: ` RUST_LOG=info ./target/release/sandd ... `
254+ - Check what the controller sees: ` curl http://controller-host:8765/stats `
275255
276256** Commands timing out:**
277257- Increase ` timeout ` parameter in ` exec() ` (in seconds)
@@ -323,5 +303,4 @@ For complete protocol specification, see [proposals/PROTOCOL.md](proposals/PROTO
323303
324304- Check [ ARCHITECTURE.md] ( ARCHITECTURE.md ) for design details
325305- Check [ proposals/PROTOCOL.md] ( proposals/PROTOCOL.md ) for protocol specification
326- - Check [ STATUS.md] ( STATUS.md ) for implementation status
327306- Check [ QUICKSTART.md] ( QUICKSTART.md ) for usage examples
0 commit comments