Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agent/CODING_TASTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ quantified and bounded.
parallel paths (a separate `GetAppKeyAmd` was rejected on these grounds, #630). When an
existing API is the wrong shape, add a purpose-built one rather than overloading a return
value (`is_app_allowed` returning policy → add `auth_api.get_app_policy` instead, #538).
- **Names must say what the thing does.** `GetQuote` for an app key → `GetAttestationForAppKey`
- **Names must say what the thing does.** `GetQuote` for an app key → `AttestAppKey`
(#360). An RPC named `ComposeHash` that returns an `app_id` is wrong (#181).
- **Avoid enums in protobuf APIs** that surface as JSON — proto has no way to express
snake_case serde renaming, so use strings (#241).
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- vmm: optionally randomize the KMS and gateway URL orders written to each CVM's system configuration so new CVMs distribute their initial requests across service nodes; both are enabled by default in `vmm.toml`
- http-client: HTTP clients are built once and shared instead of per request, which is what every `http_request*` call did until now -- each one paid for a connection pool, a DNS resolver and a TLS configuration it then threw away. Callers choose whether requests may reuse a connection (`RequestOptions::connection_reuse`, `PrpcClient::with_connection_reuse`); the default is to reuse. The gateway's health poller opts out: opening the connection is half of what a probe asks, since an agent that has run out of file descriptors keeps serving connections it already has while refusing every new one -- and every connection the gateway proxies to an app is a new one
- os/yocto: nerdctl 2.2.1 → 2.3.5, so `nerdctl compose` honours the Compose `healthcheck:` field (only translated into `--health-*` flags from 2.3.1 on). Requires openembedded-core to move to `wrynose` head for go 1.26.5, which also brings gcc 15.2 → 15.3 — every guest image measurement changes, so the new image hashes need whitelisting in KMS
- guest-agent: `GetQuote` is restricted to Intel TDX. It used to answer on every platform, returning an empty `quote` plus a `GetQuoteResponse.attestation` field carrying the versioned attestation — a shape only that one RPC produced, and one `Attest` already covers. Platforms without TDX now get an error telling them to call `Attest`, and the `attestation` field is gone from the RPC and from the Rust, Python, Go and JS SDKs. GCP Confidential VMs still get an answer — the gate is whether the platform has a TDX quote — but only the TDX half of one: `GetQuoteResponse` has no field for the vTPM quote GCP's verification also binds, so relying parties there want `Attest`, and the docs say so. `Tappd.TdxQuote`/`RawQuote` share the same backend path, so they fail closed there too instead of returning an empty quote
- guest-agent: `Worker.GetAttestationForAppKey` is replaced by `Worker.AttestAppKey`. The old method returned a `GetQuoteResponse`, so restricting `GetQuote` to Intel TDX left it unable to answer anywhere else — and the external listener with no way to attest an app key at all, since `Attest` is on the internal socket and an external caller could not use it anyway, not knowing the app key's public key until the agent derives it. `AttestAppKey` takes the same request and returns an `AttestResponse`, on every platform. This is a breaking change to an RPC present since v0.5.7; it ships no SDK method and has no known callers


## [0.5.5] - 2025-10-20
Expand Down
6 changes: 3 additions & 3 deletions docs/security/cvm-boundaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ The dstack-guest-agent runs an HTTP server on port 8090 inside the CVM. This por
|--------|-------------|------------|
| Info | Get application information | AppInfo |
| Version | Get guest agent version | WorkerVersion |
| GetAttestationForAppKey | Attest a key the app derived | GetQuoteResponse |
| AttestAppKey | Attest a key the app derived | AttestResponse |
| Health | Report whether the application is serving | HealthResponse |

Everything on this listener is unauthenticated, so each method is bounded in
Expand All @@ -204,8 +204,8 @@ what it costs and in what it says:
shape its first two lines have; the path itself is already public, since it is
measured into the compose hash. The file's *contents* are never quoted back.
Container names and statuses were already public through the dashboard below.
- `GetAttestationForAppKey` generates a TDX quote per call and is by far the
most expensive method here.
- `AttestAppKey` generates a fresh platform attestation per call and is by far
the most expensive method here.

The service also provides a web dashboard at the root URL (`/`) showing basic CVM information. View the dashboard template [here](../../dstack/guest-agent/templates/dashboard.html).

Expand Down
12 changes: 12 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,18 @@ services:
curl --unix-socket /var/run/dstack.sock http://localhost/GetQuote?report_data=0x1234deadbeef | jq .
```

`GetQuote` needs Intel TDX. On platforms without it (AMD SEV-SNP, AWS Nitro) it
returns an error. On GCP Confidential VMs it answers, but with the TDX quote
alone -- GCP's verification also binds a vTPM quote, which this response has no
field for, so a verifier that takes it at face value checks less than it should.

`Attest` is the replacement in both cases. It returns a platform-adaptive
attestation carrying whatever evidence the platform actually produces:

```bash
curl --unix-socket /var/run/dstack.sock http://localhost/Attest?report_data=0x1234deadbeef | jq .
```

For advanced compatibility with unmodified binaries that expect native Linux TEE interfaces such as `/dev/tdx_guest`, `/dev/sev-guest`, or configfs-tsm, see [Advanced Native TEE Interfaces in Containers](./native-tee-interfaces.md).

## Container Logs
Expand Down
3 changes: 3 additions & 0 deletions dstack/dstack-attest/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ rsa = { workspace = true, optional = true }
tpm2 = { workspace = true, optional = true }

[features]
# Synthetic-attestation helpers used to build fixtures. Simulator only -- see
# the gated `impl Attestation` in src/v1.rs for why production must not have it.
simulator = []
quote = [
"aws-nitro-enclaves-nsm-api",
"ciborium",
Expand Down
135 changes: 122 additions & 13 deletions dstack/dstack-attest/src/v1.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ impl PlatformEvidence {
pub fn tdx_quote(&self) -> Option<&[u8]> {
match self {
Self::Tdx { quote, .. } | Self::GcpTdx { quote, .. } => Some(quote.as_slice()),
_ => None,
Self::NitroEnclave { .. } | Self::AwsNitroTpm { .. } | Self::SevSnp { .. } => None,
}
}

Expand All @@ -101,42 +101,57 @@ impl PlatformEvidence {
Self::Tdx { event_log, .. } | Self::GcpTdx { event_log, .. } => {
Some(event_log.as_slice())
}
_ => None,
Self::NitroEnclave { .. } | Self::AwsNitroTpm { .. } | Self::SevSnp { .. } => None,
}
}

pub fn tpm_quote(&self) -> Option<&TpmQuote> {
match self {
Self::GcpTdx { tpm_quote, .. } => Some(tpm_quote),
_ => None,
Self::Tdx { .. }
| Self::NitroEnclave { .. }
| Self::AwsNitroTpm { .. }
| Self::SevSnp { .. } => None,
}
}

pub fn nsm_quote(&self) -> Option<&[u8]> {
match self {
Self::NitroEnclave { nsm_quote } => Some(nsm_quote.as_slice()),
_ => None,
Self::Tdx { .. }
| Self::GcpTdx { .. }
| Self::AwsNitroTpm { .. }
| Self::SevSnp { .. } => None,
}
}

pub fn sev_snp_report(&self) -> Option<&[u8]> {
match self {
Self::SevSnp { report, .. } => Some(report.as_slice()),
_ => None,
Self::Tdx { .. }
| Self::GcpTdx { .. }
| Self::NitroEnclave { .. }
| Self::AwsNitroTpm { .. } => None,
}
}

pub fn sev_snp_cert_chain(&self) -> Option<&[Vec<u8>]> {
match self {
Self::SevSnp { cert_chain, .. } => Some(cert_chain.as_slice()),
_ => None,
Self::Tdx { .. }
| Self::GcpTdx { .. }
| Self::NitroEnclave { .. }
| Self::AwsNitroTpm { .. } => None,
}
}

pub fn sev_snp_mr_config_document(&self) -> Option<&str> {
match self {
Self::SevSnp { mr_config, .. } => Some(mr_config.as_str()),
_ => None,
Self::Tdx { .. }
| Self::GcpTdx { .. }
| Self::NitroEnclave { .. }
| Self::AwsNitroTpm { .. } => None,
}
}

Expand All @@ -147,8 +162,8 @@ impl PlatformEvidence {

pub fn tdx_event_log_mut(&mut self) -> Option<&mut Vec<TdxEvent>> {
match self {
Self::Tdx { event_log, .. } => Some(event_log),
_ => None,
Self::Tdx { event_log, .. } | Self::GcpTdx { event_log, .. } => Some(event_log),
Self::NitroEnclave { .. } | Self::AwsNitroTpm { .. } | Self::SevSnp { .. } => None,
}
}

Expand All @@ -171,7 +186,11 @@ impl PlatformEvidence {
event_log: strip_tdx_runtime_event_log(event_log),
tpm_quote,
},
other => other,
// No TDX event log to strip. Listed rather than caught by a
// wildcard so that a new platform has to state its answer here.
evidence @ (Self::NitroEnclave { .. }
| Self::AwsNitroTpm { .. }
| Self::SevSnp { .. }) => evidence,
}
}
}
Expand Down Expand Up @@ -323,8 +342,23 @@ impl Attestation {
stack: self.stack.into_dstack_pod(report_data_payload),
}
}
}

/// Return a new attestation with the report_data patched in both platform quote and stack.
/// Helpers for assembling synthetic attestations from a captured fixture.
///
/// Simulator only. Patching the report data into a quote leaves the quote's
/// signature covering the bytes that were there before, so what comes out
/// cannot pass verification -- it is evidence-shaped, not evidence. That is
/// fine for a simulator whose callers skip verification, and wrong everywhere
/// else, so these are behind a feature that only the simulator turns on: a
/// per-package build of kms, gateway or verifier cannot reach them at all.
///
/// (`cargo build --workspace` unifies features, so a workspace-wide build does
/// compile them into every crate. The gate is a statement of intent and a
/// backstop for the documented per-package release builds, not a hard wall.)
#[cfg(any(test, feature = "simulator"))]
impl Attestation {
/// Patch `report_data` into both the platform quote and the stack evidence.
pub fn with_report_data(self, report_data: [u8; 64]) -> Self {
use crate::attestation::{SNP_REPORT_DATA_RANGE, TDX_QUOTE_REPORT_DATA_RANGE};

Expand All @@ -338,6 +372,26 @@ impl Attestation {
}
PlatformEvidence::Tdx { quote, event_log }
}
// Same TDX quote layout, so the same patch applies. The vTPM quote
// beside it cannot follow: its `qualified_data` is
// `sha256(tdx_quote)`, and re-deriving that would need the AK to
// re-sign. So this severs the TPM binding on top of the quote
// signature the patch already invalidates -- acceptable only
// because nothing verifies a simulator attestation.
PlatformEvidence::GcpTdx {
mut quote,
event_log,
tpm_quote,
} => {
if quote.len() >= TDX_QUOTE_REPORT_DATA_RANGE.end {
quote[TDX_QUOTE_REPORT_DATA_RANGE].copy_from_slice(&report_data);
}
PlatformEvidence::GcpTdx {
quote,
event_log,
tpm_quote,
}
}
PlatformEvidence::SevSnp {
mut report,
cert_chain,
Expand All @@ -352,8 +406,28 @@ impl Attestation {
mr_config,
}
}
other => other,
// Nothing to patch: both carry their report data inside a signed
// document, and rewriting it would only invalidate the signature.
// Listed rather than caught by a wildcard so that a new platform
// has to state its answer here instead of silently getting this
// one -- GcpTdx was skipped for exactly that reason.
evidence @ (PlatformEvidence::NitroEnclave { .. }
| PlatformEvidence::AwsNitroTpm { .. }) => evidence,
};
Self {
version: self.version,
platform,
stack: self.stack,
}
.with_stack_report_data(report_data)
}

/// Patch `report_data` into the stack evidence only, leaving the platform
/// quote untouched.
///
/// For callers that go on to generate a real quote over the same report
/// data: patching the fixture's quote first would only be overwritten.
pub fn with_stack_report_data(self, report_data: [u8; 64]) -> Self {
let stack = match self.stack {
StackEvidence::Dstack {
runtime_events,
Expand All @@ -378,7 +452,7 @@ impl Attestation {
};
Self {
version: self.version,
platform,
platform: self.platform,
stack,
}
}
Expand Down Expand Up @@ -547,6 +621,41 @@ mod tests {
assert_eq!(stripped[3].event_payload, vec![0x42]);
}

#[test]
fn tdx_event_log_accessors_agree_on_gcp_tdx() {
// The mut accessor used to match bare TDX only, so `fill_v2_preimages`
// silently skipped GCP attestations and returned V2 events without the
// preimage the guest-agent proto promises clients can verify. Whatever
// the read accessor reaches, the mut one has to reach too.
let mut evidence = PlatformEvidence::GcpTdx {
quote: vec![0u8; 64],
event_log: vec![TdxEvent {
imr: 3,
event_type: cc_eventlog::DSTACK_RUNTIME_EVENT_TYPE,
digest: vec![0u8; 48],
event: "test-event".into(),
event_payload: b"payload".to_vec(),
version: EventLogVersion::V2,
preimage: None,
}],
tpm_quote: TpmQuote {
message: Vec::new(),
signature: Vec::new(),
pcr_values: Vec::new(),
ak_cert: Vec::new(),
platform: dstack_types::Platform::Gcp,
event_log: Vec::new(),
},
};

assert!(evidence.tdx_event_log().is_some());
let log = evidence
.tdx_event_log_mut()
.expect("mut accessor must reach GCP TDX too");
cc_eventlog::tdx::fill_v2_preimages(log);
assert!(evidence.tdx_event_log().unwrap()[0].preimage.is_some());
}

#[test]
fn sev_snp_with_report_data_patches_report_and_stack() {
let mut report = vec![0x11; 1184];
Expand Down
6 changes: 5 additions & 1 deletion dstack/gateway/rpc/proto/gateway_rpc.proto
Original file line number Diff line number Diff line change
Expand Up @@ -804,12 +804,16 @@ message ForceReleaseCertLockRequest {
message CertAttestationInfo {
// Certificate public key (DER encoded)
bytes public_key = 1;
// TDX Quote (JSON serialized)
// TDX Quote (JSON serialized). Empty when the node has no Intel TDX, or when
// the quote request failed; `attestation` is the evidence then.
string quote = 2;
// Node that generated this attestation
uint32 generated_by = 3;
// Timestamp when this attestation was generated
uint64 generated_at = 4;
// Versioned attestation (JSON serialized `AttestResponse`). Present on every
// platform, and the only evidence on nodes without Intel TDX.
string attestation = 5;
}

// List certificate attestations request
Expand Down
2 changes: 2 additions & 0 deletions dstack/gateway/src/admin_service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -674,6 +674,7 @@ impl AdminRpc for AdminRpcHandler {
.map(|att| CertAttestationInfo {
public_key: att.public_key,
quote: att.quote,
attestation: att.attestation,
generated_by: att.generated_by,
generated_at: att.generated_at,
});
Expand All @@ -684,6 +685,7 @@ impl AdminRpc for AdminRpcHandler {
.map(|att| CertAttestationInfo {
public_key: att.public_key,
quote: att.quote,
attestation: att.attestation,
generated_by: att.generated_by,
generated_at: att.generated_at,
})
Expand Down
20 changes: 16 additions & 4 deletions dstack/gateway/src/distributed_certbot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -642,7 +642,8 @@ impl DistributedCertBot {
.to_report_data(account_uri.as_bytes())
.to_vec();

// Get quote
// Get quote. GetQuote is Intel TDX only, so this is best-effort: on other
// platforms the versioned attestation below is the only evidence available.
let quote = match agent
.get_quote(RawQuoteArgs {
report_data: report_data.clone(),
Expand All @@ -652,7 +653,7 @@ impl DistributedCertBot {
Ok(resp) => serde_json::to_string(&resp).unwrap_or_default(),
Err(err) => {
warn!("failed to get TDX quote for ACME account: {err:?}");
return Ok(());
String::new()
}
};

Expand All @@ -665,6 +666,11 @@ impl DistributedCertBot {
}
};

if quote.is_empty() && attestation_str.is_empty() {
warn!("no attestation evidence for ACME account, skipping save");
return Ok(());
}

let attestation = AcmeAttestation {
account_uri: account_uri.to_string(),
quote,
Expand Down Expand Up @@ -712,7 +718,8 @@ impl DistributedCertBot {
.to_report_data(public_key_der)
.to_vec();

// Get quote
// Get quote. GetQuote is Intel TDX only, so this is best-effort: on other
// platforms the versioned attestation below is the only evidence available.
let quote = match agent
.get_quote(RawQuoteArgs {
report_data: report_data.clone(),
Expand All @@ -722,7 +729,7 @@ impl DistributedCertBot {
Ok(resp) => serde_json::to_string(&resp).unwrap_or_default(),
Err(err) => {
warn!(domain, "failed to generate TDX quote: {err:?}");
return Ok(());
String::new()
}
};

Expand All @@ -735,6 +742,11 @@ impl DistributedCertBot {
}
};

if quote.is_empty() && attestation.is_empty() {
warn!(domain, "no attestation evidence for cert, skipping save");
return Ok(());
}

let attestation = CertAttestation {
public_key: public_key_der.to_vec(),
quote,
Expand Down
Loading
Loading