Aether

English | 中文

A distributed key-value store built in Rust, using Raft consensus for replication, RocksDB for storage, and gRPC for client communication.

Why Aether?

Aether is a distributed key-value store for coordination, configuration, and service discovery -- a Rust alternative to etcd.

• Single binary, zero dependencies -- drop it on any machine, no Go runtime or shared libraries needed
• No GC, predictable latency -- Rust's ownership model eliminates stop-the-world pauses that plague Go and JVM-based stores under high write throughput
• Stateless gateway -- horizontally scale read capacity by adding gateway nodes that proxy to the Raft cluster without participating in consensus
• Coordination built-in -- distributed locks, leader election, barriers, and FIFO queues are first-class primitives, not recipes you assemble yourself

Features

Beyond the core KV store, Aether provides:

• Transactions -- Compare-and-swap (CAS) operations via Txn RPC
• Watch -- Bidirectional streaming for real-time key-range change notifications
• Leases -- TTL-based keys with automatic expiry and keep-alive streaming
• Distributed Lock -- Mutex and read-write lock primitives
• Leader Election -- Campaign-based election with observer streaming
• Sessions -- Client session lifecycle with timeout negotiation
• Barriers & Queues -- Distributed coordination primitives
• Data Sharding -- Region-based key-range sharding with split/merge scheduling
• Cluster Discovery -- Static, DNS, and token-based peer discovery for dynamic cluster bootstrap
• Auth -- JWT authentication with RBAC (users, roles, key-range permissions)
• Observability -- Prometheus metrics, structured JSON logging, health check endpoints

Quick Start

Single Node

# Download the binary from releases, then:
aether --node-id 1 --addr 127.0.0.1:2379

3-Node Cluster

Create a config file for each node:

node1.toml

node_id = 1
addr = "127.0.0.1:2379"
data_dir = "/tmp/aether-node1"

[[cluster.peers]]
node_id = 2
addr = "127.0.0.1:2380"

[[cluster.peers]]
node_id = 3
addr = "127.0.0.1:2381"

node2.toml

node_id = 2
addr = "127.0.0.1:2380"
data_dir = "/tmp/aether-node2"

[[cluster.peers]]
node_id = 1
addr = "127.0.0.1:2379"

[[cluster.peers]]
node_id = 3
addr = "127.0.0.1:2381"

node3.toml

node_id = 3
addr = "127.0.0.1:2381"
data_dir = "/tmp/aether-node3"

[[cluster.peers]]
node_id = 1
addr = "127.0.0.1:2379"

[[cluster.peers]]
node_id = 2
addr = "127.0.0.1:2380"

aether --config node1.toml   # terminal 1
aether --config node2.toml   # terminal 2
aether --config node3.toml   # terminal 3

Gateway Mode

Stateless proxy that routes requests to a backend cluster:

aether --gateway --addr 127.0.0.1:2379 \
  --backends 127.0.0.1:2380,127.0.0.1:2381,127.0.0.1:2382

Configuration

Aether reads aether.toml from the current directory by default. CLI flags override config file values.

Full config reference

node_id = 1
addr = "127.0.0.1:2379"
data_dir = "/var/lib/aether"

[cluster]
heartbeat_interval_ms = 1000
election_timeout_ms = 10000
snapshot_trigger_log_entries = 10000

[[cluster.peers]]
node_id = 2
addr = "10.0.0.2:2379"

[cluster.discovery]
method = "static"              # static, dns, token

[auth]
enabled = false
token_expiry_hours = 24
signing_key = ""

[lease]
max_leases = 10000
max_ttl = 86400

[log]
level = "info"                 # trace, debug, info, warn, error
json = false
log_dir = ""                   # empty = stdout only
log_file_prefix = "aether"

[metrics]
listen_addr = "127.0.0.1:9090"

[shard]
max_regions = 1024
max_region_size_bytes = 67108864
auto_split = false

API

All services are exposed on the same gRPC address (default 127.0.0.1:2379).

service AetherKV {
    rpc Put(PutRequest) returns (PutResponse);
    rpc Get(GetRequest) returns (GetResponse);
    rpc Delete(DeleteRequest) returns (DeleteResponse);
    rpc Range(RangeRequest) returns (RangeResponse);
    rpc Txn(TxnRequest) returns (TxnResponse);
}

Service	Operations
AetherWatch	Watch (bidirectional streaming)
AetherLease	LeaseGrant, LeaseRevoke, LeaseKeepAlive, LeaseTimeToLive, LeaseLeases
AetherAuth	Authenticate, UserAdd/Delete/Get/List, RoleAdd/Delete/Get/List, UserGrantRole, UserRevokeRole
AetherCluster	MemberAdd, MemberRemove, MemberList, MemberPromote
AetherMaintenance	Status, AlarmList, AlarmAdd, AlarmDelete, Defragment
AetherLock	Lock, Unlock
AetherElection	Campaign, Observe, Leader, Resign
AetherBarrier	Create, Await
AetherQueue	Enqueue, Dequeue
AetherSession	Create, Renew, Close
AetherShard	CreateRegion, SplitRegion, MergeRegions, GetRegion

Proto definitions are in proto/.

Observability

# Prometheus metrics (on the same address as gRPC, port from metrics.listen_addr)
curl http://127.0.0.1:9090/metrics

# Health checks (same address as metrics)
curl http://127.0.0.1:9090/health/live    # liveness
curl http://127.0.0.1:9090/health/ready   # readiness (Raft + storage)

Benchmarks

Run cargo bench to reproduce. Results on Apple M-series (single-threaded):

Storage (RocksDB)

Operation	Latency
put (256 B)	1.6 us
put (1 KB)	2.2 us
put (64 KB)	26 us
get (hit)	442 ns
get (miss)	121 ns
delete	1.4 us
batch_write (10 ops)	5.8 us
scan (100 keys)	12.9 us
scan (1000 keys)	149 us

Raft Log Store

Operation	Latency
append_entries (1 entry)	23 us
append_entries (10 entries)	48 us
append_entries (100 entries)	90 us
entries_read (100)	13.3 us
entries_read (1000)	140 us
save_hard_state	18.6 us
compact (100 entries)	37 us

Docker

docker build -t aether .

docker run -d --name aether-1 \
  -p 2379:2379 \
  -v aether-data:/data \
  aether --node-id 1 --addr 0.0.0.0:2379 --data-dir /data

Development

Requires Rust 1.85+ (Edition 2024).

cargo build                    # debug build
cargo build --release          # release build
cargo test                     # all tests
cargo clippy -- -D warnings    # lint
cargo fmt -- --check           # format check

License

Licensed under the Apache License, Version 2.0.