diff --git a/CHANGELOG.md b/CHANGELOG.md index 45aa253bb..8ba6ad3fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - sdk: `verify_signature` and `verify_signature_chain` in all four SDKs, replacing the removed guest-agent `Verify` RPC. `verify_signature_chain` is new capability rather than a port: it walks all three links of a `Sign` signature chain -- payload signature, the app root key attesting `"{purpose}:{hex(pubkey)}"`, and the KMS root attesting that app root for this `app_id` -- and requires the chain to anchor at a KMS root public key **the caller supplies**. That anchor has to come from somewhere independently trusted (the `DstackKms` contract's `kmsInfo().k256Pubkey`, or a pinned value); read it from the KMS being checked and an attacker who can answer that query can also mint a self-consistent chain. The four ports are pinned against one committed set of test vectors, `sdk/tests/vectors/signature_chain.json`, generated from the real KMS and guest-agent primitives -- this repo has shipped cross-language crypto drift twice already +- guest-agent: `Attest` accepts `include_boottime_gpu_evidence` and returns the boot-time GPU attestation evidence in `AttestResponse.boottime_gpu_evidence`, so a verifier can fetch the quote and the GPU evidence in one round trip instead of also calling `GpuInfo`. Exposed in the Rust, Python, Go and JS SDKs - sdk: `AppCompose` in the Go SDK gained `init_script`, `storage_fs`, `swap_size`, `event_log_version`, `port_policy` and `verity_volumes`, and `Requirements` gained `gpu_policy` in the Go and Python SDKs - shared API authentication (`dstack-api-auth`) protecting the full VMM HTTP/pRPC/UI surface and unifying Gateway/KMS admin auth: bearer/`X-Admin-Token`/HTTP Basic/bcrypt htpasswd, constant-time verification (#796) - gateway: `Admin.Status` reports `health_gating`, so an operator can see whether this node's health polling is switched on. With it off, instances that opted in sit at `unknown` forever and are all in rotation, which is otherwise indistinguishable on the dashboard from being held out pending a first answer diff --git a/docs/attestation-tdx.md b/docs/attestation-tdx.md index a71533ef1..bab973198 100644 --- a/docs/attestation-tdx.md +++ b/docs/attestation-tdx.md @@ -45,7 +45,7 @@ evaluation, `gpu-attestation`. The `gpu-policy-hash` payload is The `gpu-attestation` payload is JSON containing the verified device count, CC/DevTools state, and `evidence_sha256`. -The guest-agent `GpuInfo` API returns the complete `nvattest` JSON captured during boot. It is not trustworthy by itself. After verifying the TDX quote and replaying the event log to RTMR3, hash the exact UTF-8 bytes of `GpuInfo.attestation` and require the result to equal the `gpu-attestation` event's `evidence_sha256`. See [GPU Security for AI Workloads](./security/security-model.md#gpu-security-for-ai-workloads) for the event schema, ordering, Rego example, and platform differences. +The guest-agent `GpuInfo` API returns the complete `nvattest` JSON captured during boot; `Attest` returns the same bytes in `boottime_gpu_evidence` when called with `include_boottime_gpu_evidence`, so a verifier can fetch the quote and the GPU evidence in one round trip. It is not trustworthy by itself. After verifying the TDX quote and replaying the event log to RTMR3, hash the exact UTF-8 bytes of `GpuInfo.attestation` (or `Attest.boottime_gpu_evidence`) and require the result to equal the `gpu-attestation` event's `evidence_sha256`. See [GPU Security for AI Workloads](./security/security-model.md#gpu-security-for-ai-workloads) for the event schema, ordering, Rego example, and platform differences. ### 2.2. Determining expected MRs MRTD, RTMR0, RTMR1, and RTMR2 correspond to the image. dstack OS builds all related software from source. diff --git a/docs/security/security-model.md b/docs/security/security-model.md index 72791944e..5ba888705 100644 --- a/docs/security/security-model.md +++ b/docs/security/security-model.md @@ -144,7 +144,7 @@ boot-mr-done } ``` -`GpuInfo` returns that complete boot-time `nvattest` JSON in its `attestation` string; it does not run a new attestation. To bind the API result to TDX evidence: verify the quote, replay the event log to the quote's RTMR3, require exactly one pre-`system-ready` `gpu-attestation` event, decode its JSON payload, and compare `evidence_sha256` with `SHA-256(UTF-8(GpuInfo.attestation))`. Only after this comparison should the verifier inspect the returned claims. This exact-byte comparison includes any whitespace or trailing newline in the returned string. +`GpuInfo` returns that complete boot-time `nvattest` JSON in its `attestation` string, and `Attest` returns the same bytes in `boottime_gpu_evidence` when called with `include_boottime_gpu_evidence`; neither runs a new attestation. To bind the API result to TDX evidence: verify the quote, replay the event log to the quote's RTMR3, require exactly one pre-`system-ready` `gpu-attestation` event, decode its JSON payload, and compare `evidence_sha256` with `SHA-256(UTF-8(GpuInfo.attestation))` (equivalently `SHA-256(UTF-8(Attest.boottime_gpu_evidence))`). Only after this comparison should the verifier inspect the returned claims. This exact-byte comparison includes any whitespace or trailing newline in the returned string. A verifier must replay the measured event log, require exactly one `gpu-policy-hash` event immediately after `compose-hash`, and compare its 32-byte payload with the expected policy digest (`SHA-256(JCS({}))` for the omitted/default policy). When MrConfigV3 includes `gpu_policy_hash`, it must match the same digest. When GPU protection is required, the verifier must also require exactly one pre-`system-ready` `gpu-attestation` event with `devices > 0` and, when applicable, the expected deployment count. The raw `attestation.out` file is not trusted by itself; if it is supplied for inspection, its digest must match the `gpu-attestation` event. diff --git a/dstack/cert-client/src/bin/dstack-kms-sign-cert-fixture.rs b/dstack/cert-client/src/bin/dstack-kms-sign-cert-fixture.rs index 3451c5535..e2a09767c 100644 --- a/dstack/cert-client/src/bin/dstack-kms-sign-cert-fixture.rs +++ b/dstack/cert-client/src/bin/dstack-kms-sign-cert-fixture.rs @@ -3,7 +3,7 @@ //! Generate a v2 KMS CSR whose key is bound to fresh guest attestation. use anyhow::{Context, Result}; -use dstack_guest_agent_rpc::{dstack_guest_client::DstackGuestClient, RawQuoteArgs}; +use dstack_guest_agent_rpc::{dstack_guest_client::DstackGuestClient, AttestArgs}; use http_client::prpc::PrpcClient; use ra_tls::{ attestation::{PlatformEvidence, QuoteContentType, VersionedAttestation}, @@ -31,8 +31,9 @@ async fn main() -> Result<()> { let address = dstack_types::dstack_agent_address(); let client = DstackGuestClient::new(PrpcClient::new(address)); let response = client - .attest(RawQuoteArgs { + .attest(AttestArgs { report_data: report_data.to_vec(), + include_boottime_gpu_evidence: false, }) .await .context("failed to obtain key-bound guest attestation")?; diff --git a/dstack/gateway/src/distributed_certbot.rs b/dstack/gateway/src/distributed_certbot.rs index b9193691d..734ab954c 100644 --- a/dstack/gateway/src/distributed_certbot.rs +++ b/dstack/gateway/src/distributed_certbot.rs @@ -12,7 +12,7 @@ use std::time::Duration; use anyhow::{bail, Context, Result}; use certbot::{AcmeClient, Dns01Client}; -use dstack_guest_agent_rpc::RawQuoteArgs; +use dstack_guest_agent_rpc::{AttestArgs, RawQuoteArgs}; use ra_tls::attestation::QuoteContentType; use ra_tls::rcgen::KeyPair; use tokio::sync::Mutex; @@ -658,7 +658,13 @@ impl DistributedCertBot { }; // Get attestation - let attestation_str = match agent.attest(RawQuoteArgs { report_data }).await { + let attestation_str = match agent + .attest(AttestArgs { + report_data, + include_boottime_gpu_evidence: false, + }) + .await + { Ok(resp) => serde_json::to_string(&resp).unwrap_or_default(), Err(err) => { warn!("failed to get attestation for ACME account: {err:?}"); @@ -734,7 +740,13 @@ impl DistributedCertBot { }; // Get attestation - let attestation = match agent.attest(RawQuoteArgs { report_data }).await { + let attestation = match agent + .attest(AttestArgs { + report_data, + include_boottime_gpu_evidence: false, + }) + .await + { Ok(resp) => serde_json::to_string(&resp).unwrap_or_default(), Err(err) => { warn!(domain, "failed to get attestation: {err:?}"); diff --git a/dstack/guest-agent-simulator/src/main.rs b/dstack/guest-agent-simulator/src/main.rs index 95db32586..fd29c43c8 100644 --- a/dstack/guest-agent-simulator/src/main.rs +++ b/dstack/guest-agent-simulator/src/main.rs @@ -13,7 +13,7 @@ use dstack_guest_agent::{ config::{self, Config}, run_server, AppState, }; -use dstack_guest_agent_rpc::{AttestResponse, GetQuoteResponse}; +use dstack_guest_agent_rpc::GetQuoteResponse; use mock_attestation::tdx::TdxGenerator; use ra_tls::attestation::VersionedAttestation; use serde::Deserialize; @@ -102,7 +102,7 @@ impl PlatformBackend for SimulatorPlatform { ) } - fn attest_response(&self, report_data: [u8; 64]) -> Result { + fn attest_cvm(&self, report_data: [u8; 64]) -> Result { simulator::simulated_attest_response( &self.attestation, report_data, @@ -183,9 +183,13 @@ mod tests { fn simulator_attest_response_preserves_legacy_wire_format() { let platform = load_fixture_platform(); let report_data = [0x5a; 64]; - let response = platform.attest_response(report_data).unwrap(); - assert_eq!(response.attestation.first(), Some(&0x00)); - let patched = VersionedAttestation::from_bytes(&response.attestation) + let encoded = platform + .attest_cvm(report_data) + .unwrap() + .to_bytes() + .unwrap(); + assert_eq!(encoded.first(), Some(&0x00)); + let patched = VersionedAttestation::from_bytes(&encoded) .unwrap() .into_v1(); assert_eq!(patched.report_data().unwrap(), report_data); @@ -296,7 +300,7 @@ mod tests { // relying parties on GCP at Attest. let attested = simulator::simulated_attest_response(&gcp_tdx, report_data, true, None) .expect("Attest must work on GCP TDX too"); - let round_tripped = VersionedAttestation::from_bytes(&attested.attestation) + let round_tripped = VersionedAttestation::from_bytes(&attested.to_bytes().unwrap()) .unwrap() .into_v1(); assert!(round_tripped.platform.tpm_quote().is_some()); @@ -312,8 +316,12 @@ mod tests { let original = fixture.clone().into_v1().report_data().unwrap(); let platform = SimulatorPlatform::new(fixture, false, None).unwrap(); let report_data = [0x5a; 64]; - let response = platform.attest_response(report_data).unwrap(); - let patched = VersionedAttestation::from_bytes(&response.attestation) + let encoded = platform + .attest_cvm(report_data) + .unwrap() + .to_bytes() + .unwrap(); + let patched = VersionedAttestation::from_bytes(&encoded) .unwrap() .into_v1(); assert_eq!(patched.report_data().unwrap(), original); diff --git a/dstack/guest-agent-simulator/src/simulator.rs b/dstack/guest-agent-simulator/src/simulator.rs index 92df0950b..b6c731e2d 100644 --- a/dstack/guest-agent-simulator/src/simulator.rs +++ b/dstack/guest-agent-simulator/src/simulator.rs @@ -6,7 +6,7 @@ use std::path::Path; use anyhow::{anyhow, Context, Result}; use dcap_qvl::quote::Quote; -use dstack_guest_agent_rpc::{AttestResponse, GetQuoteResponse}; +use dstack_guest_agent_rpc::GetQuoteResponse; use mock_attestation::tdx::TdxGenerator; use ra_tls::attestation::{ AttestationV1, PlatformEvidence, QuoteContentType, TdxAttestationExt, VersionedAttestation, @@ -59,20 +59,17 @@ pub fn simulated_attest_response( report_data: [u8; 64], patch_report_data: bool, generator: Option<&TdxGenerator>, -) -> Result { +) -> Result { let preserve_legacy = matches!(source, VersionedAttestation::V0 { .. }); let mut attestation = prepare_attestation(source, report_data, patch_report_data, generator, "attest")?; if let Some(event_log) = attestation.platform.tdx_event_log_mut() { cc_eventlog::tdx::fill_v2_preimages(event_log); } - let attestation = if preserve_legacy { + Ok(if preserve_legacy { attestation.try_into_legacy()?.into_versioned() } else { VersionedAttestation::V1 { attestation } - }; - Ok(AttestResponse { - attestation: attestation.to_bytes()?, }) } diff --git a/dstack/guest-agent/rpc/proto/agent_rpc.proto b/dstack/guest-agent/rpc/proto/agent_rpc.proto index 1a3552c5c..65c4442ae 100644 --- a/dstack/guest-agent/rpc/proto/agent_rpc.proto +++ b/dstack/guest-agent/rpc/proto/agent_rpc.proto @@ -53,7 +53,7 @@ service DstackGuest { // Generates a versioned attestation with the given report data. // Returns a dstack-defined attestation format that supports different attestation modes across platforms. - rpc Attest(RawQuoteArgs) returns (AttestResponse) {} + rpc Attest(AttestArgs) returns (AttestResponse) {} // Get app info rpc Info(google.protobuf.Empty) returns (AppInfo) {} @@ -180,6 +180,22 @@ message RawQuoteArgs { bytes report_data = 1; } +// The request to get a versioned attestation +message AttestArgs { + // 64 bytes of report data + bytes report_data = 1; + // Field 2 and 3 carried `include_ccel` and `include_preimages` while this RPC + // took a `RawQuoteArgs`. Both were bools, so reusing either number here would + // make a pre-0.6.0 client's `include_ccel = true` arrive as a request for + // something else entirely. + reserved 2, 3; + reserved "include_ccel", "include_preimages"; + // Also return the boot-time GPU attestation evidence in + // `boottime_gpu_evidence`. This does not sample the GPU now and does not + // answer `report_data`; see that field. + bool include_boottime_gpu_evidence = 4; +} + message TdxQuoteResponse { // TDX quote bytes quote = 1; @@ -196,6 +212,21 @@ message TdxQuoteResponse { message AttestResponse { // The attestation bytes attestation = 1; + // Complete JSON output produced by nvattest at boot, the same bytes `GpuInfo` + // serves. Only `DstackGuest.Attest` populates it, and only when the request set + // `include_boottime_gpu_evidence` and boot-time GPU attestation output exists. + // + // Not bound to `report_data`: nvattest ran at boot against its own nonce, so a + // fresh `report_data` says nothing about it. Bind it by replaying the runtime + // event log and comparing sha256 of these exact UTF-8 bytes against the + // `evidence_sha256` field of the measured `gpu-attestation` event. + // + // This is a historical statement about the boot, not a live one: it does not + // prove the GPU is still attached. Sampling the GPU at attestation time would + // not fix that -- an NVIDIA report binds the device and a nonce but not the TD + // the device is attached to, so a fresh report can be relayed from a genuine + // remote GPU. Only TDISP/TEE-IO device binding closes that. + string boottime_gpu_evidence = 2; } message GpuInfoResponse { diff --git a/dstack/guest-agent/src/backend.rs b/dstack/guest-agent/src/backend.rs index d2347cdef..1c6671454 100644 --- a/dstack/guest-agent/src/backend.rs +++ b/dstack/guest-agent/src/backend.rs @@ -3,7 +3,7 @@ // SPDX-License-Identifier: Apache-2.0 use anyhow::{Context, Result}; -use dstack_guest_agent_rpc::{AttestResponse, GetQuoteResponse}; +use dstack_guest_agent_rpc::GetQuoteResponse; use ra_tls::attestation::Attestation; use ra_tls::attestation::{QuoteContentType, VersionedAttestation}; @@ -11,7 +11,10 @@ pub trait PlatformBackend: Send + Sync { fn attestation_for_info(&self) -> Result; fn certificate_attestation(&self, pubkey: &[u8]) -> Result; fn quote_response(&self, report_data: [u8; 64], vm_config: &str) -> Result; - fn attest_response(&self, report_data: [u8; 64]) -> Result; + /// Attest the CVM itself: the attestation `Attest` and `AttestAppKey` + /// return, with digest preimages filled in. Encoding it is the RPC + /// layer's job. + fn attest_cvm(&self, report_data: [u8; 64]) -> Result; } #[derive(Debug, Default)] @@ -44,12 +47,10 @@ impl PlatformBackend for RealPlatform { }) } - fn attest_response(&self, report_data: [u8; 64]) -> Result { + fn attest_cvm(&self, report_data: [u8; 64]) -> Result { let mut attestation = Attestation::quote(&report_data).context("Failed to get attestation")?; attestation.fill_event_preimages(); - Ok(AttestResponse { - attestation: attestation.into_versioned().to_bytes()?, - }) + Ok(attestation.into_versioned()) } } diff --git a/dstack/guest-agent/src/rpc_service.rs b/dstack/guest-agent/src/rpc_service.rs index e3ee8fc35..b594cee74 100644 --- a/dstack/guest-agent/src/rpc_service.rs +++ b/dstack/guest-agent/src/rpc_service.rs @@ -15,10 +15,10 @@ use dstack_guest_agent_rpc::{ dstack_guest_server::{DstackGuestRpc, DstackGuestServer}, tappd_server::{TappdRpc, TappdServer}, worker_server::{WorkerRpc, WorkerServer}, - AppInfo, AttestAppKeyRequest, AttestResponse, DeriveK256KeyResponse, DeriveKeyArgs, GetKeyArgs, - GetKeyResponse, GetQuoteResponse, GetTlsKeyArgs, GetTlsKeyResponse, GpuInfoResponse, - HealthResponse, RawQuoteArgs, SignRequest, SignResponse, TdxQuoteArgs, TdxQuoteResponse, - WorkerVersion, + AppInfo, AttestAppKeyRequest, AttestArgs, AttestResponse, DeriveK256KeyResponse, DeriveKeyArgs, + GetKeyArgs, GetKeyResponse, GetQuoteResponse, GetTlsKeyArgs, GetTlsKeyResponse, + GpuInfoResponse, HealthResponse, RawQuoteArgs, SignRequest, SignResponse, TdxQuoteArgs, + TdxQuoteResponse, WorkerVersion, }; use dstack_types::{AppKeys, SysConfig, GPU_ATTESTATION_OUTPUT}; use ed25519_dalek::ed25519::signature::hazmat::PrehashSigner; @@ -65,6 +65,16 @@ fn read_gpu_attestation(path: &Path) -> String { } } +/// GPU evidence to return alongside an attestation. Opt-in, so a caller that +/// does not care about GPUs neither pays the disk read nor carries the payload. +fn boottime_gpu_evidence(include: bool, path: &Path) -> String { + if include { + read_gpu_attestation(path) + } else { + String::new() + } +} + #[derive(Clone)] pub struct AppState { inner: Arc, @@ -216,8 +226,8 @@ impl AppState { .quote_response(report_data, &self.inner.vm_config) } - fn attest_response(&self, report_data: [u8; 64]) -> Result { - self.inner.platform.attest_response(report_data) + fn attest_cvm(&self, report_data: [u8; 64]) -> Result> { + self.inner.platform.attest_cvm(report_data)?.to_bytes() } } @@ -443,9 +453,15 @@ impl DstackGuestRpc for InternalRpcHandler { }) } - async fn attest(self, request: RawQuoteArgs) -> Result { + async fn attest(self, request: AttestArgs) -> Result { let report_data = pad64(&request.report_data).context("Report data is too long")?; - self.state.attest_response(report_data) + Ok(AttestResponse { + attestation: self.state.attest_cvm(report_data)?, + boottime_gpu_evidence: boottime_gpu_evidence( + request.include_boottime_gpu_evidence, + Path::new(GPU_ATTESTATION_OUTPUT), + ), + }) } async fn version(self) -> Result { @@ -641,7 +657,12 @@ impl WorkerRpc for ExternalRpcHandler { async fn attest_app_key(self, request: AttestAppKeyRequest) -> Result { let report_data = self.app_key_report_data(&request.algorithm).await?; - self.state.attest_response(report_data) + Ok(AttestResponse { + attestation: self.state.attest_cvm(report_data)?, + // This method attests a key, not the machine. A caller that wants + // the boot-time GPU evidence asks `Attest` or `GpuInfo` for it. + boottime_gpu_evidence: String::new(), + }) } } @@ -748,6 +769,17 @@ mod tests { assert_eq!(read_gpu_attestation(output.path()), attestation); } + #[test] + fn attest_returns_boottime_gpu_evidence_only_when_requested() { + let mut output = tempfile::NamedTempFile::new().unwrap(); + let evidence = r#"{"result_code":0,"claims":[]}"#; + output.write_all(evidence.as_bytes()).unwrap(); + output.flush().unwrap(); + + assert_eq!(boottime_gpu_evidence(true, output.path()), evidence); + assert_eq!(boottime_gpu_evidence(false, output.path()), ""); + } + #[test] fn missing_gpu_attestation_output_reads_as_empty() { let dir = tempfile::tempdir().unwrap(); @@ -951,11 +983,9 @@ pNs85uhOZE8z2jr8Pg== }) } - fn attest_response(&self, report_data: [u8; 64]) -> Result { + fn attest_cvm(&self, report_data: [u8; 64]) -> Result { let attestation = patch_report_data(&self.attestation, report_data); - Ok(AttestResponse { - attestation: VersionedAttestation::V1 { attestation }.to_bytes()?, - }) + Ok(VersionedAttestation::V1 { attestation }) } } diff --git a/dstack/kms/src/main_service/upgrade_authority.rs b/dstack/kms/src/main_service/upgrade_authority.rs index 9340dd650..c315e0128 100644 --- a/dstack/kms/src/main_service/upgrade_authority.rs +++ b/dstack/kms/src/main_service/upgrade_authority.rs @@ -5,9 +5,7 @@ use super::build_boot_info_for_attestation; use crate::config::{AuthApi, KmsConfig}; use anyhow::{bail, Context, Result}; -use dstack_guest_agent_rpc::{ - dstack_guest_client::DstackGuestClient, AttestResponse, RawQuoteArgs, -}; +use dstack_guest_agent_rpc::{dstack_guest_client::DstackGuestClient, AttestArgs, AttestResponse}; use http_client::prpc::PrpcClient; use ra_tls::attestation::{AttestationVerifier, VerifiedAttestation, VersionedAttestation}; use serde::de::DeserializeOwned; @@ -182,7 +180,12 @@ pub(crate) fn dstack_client() -> DstackGuestClient { } pub(crate) async fn app_attest(report_data: Vec) -> Result { - dstack_client().attest(RawQuoteArgs { report_data }).await + dstack_client() + .attest(AttestArgs { + report_data, + include_boottime_gpu_evidence: false, + }) + .await } pub(crate) fn pad64(hash: [u8; 32]) -> Vec { diff --git a/sdk/curl/api.md b/sdk/curl/api.md index be19180f7..84bb3d345 100644 --- a/sdk/curl/api.md +++ b/sdk/curl/api.md @@ -238,6 +238,7 @@ You can submit the returned `attestation` directly to the verifier `/verify` end | Field | Type | Description | Example | |-------|------|-------------|----------| | `report_data` | string | Report data of max length 64 bytes. Padding with 0s if less than 64 bytes. | `"1234deadbeaf"` | +| `include_boottime_gpu_evidence` | boolean | Optional, defaults to `false`. Also returns the boot-time GPU attestation evidence in `boottime_gpu_evidence`. | `true` | **Example:** ```bash @@ -245,21 +246,29 @@ curl --unix-socket /var/run/dstack.sock -X POST \ http://dstack/Attest \ -H 'Content-Type: application/json' \ -d '{ - "report_data": "1234deadbeaf" + "report_data": "1234deadbeaf", + "include_boottime_gpu_evidence": true }' ``` Or ```bash -curl --unix-socket /var/run/dstack.sock http://dstack/Attest?report_data=00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 +curl --unix-socket /var/run/dstack.sock 'http://dstack/Attest?report_data=1234deadbeaf&include_boottime_gpu_evidence=true' ``` **Response:** ```json { - "attestation": "" + "attestation": "", + "boottime_gpu_evidence": "{\"result_code\": 0, \"claims\": [...]}" } ``` +`boottime_gpu_evidence` carries the same bytes [`GpuInfo`](#7-gpu-info) serves, so one call +returns both the quote and the GPU evidence a verifier needs. It is empty unless +`include_boottime_gpu_evidence` was set and boot-time GPU attestation output exists. It is +**not** bound to `report_data` — authenticate it with the `evidence_sha256` +procedure documented under `GpuInfo` below. + ### 7. GPU Info Returns GPU information collected during boot. Currently, this includes the diff --git a/sdk/go/README.md b/sdk/go/README.md index bda275b24..5b7c524de 100644 --- a/sdk/go/README.md +++ b/sdk/go/README.md @@ -575,6 +575,23 @@ instead. - Cryptographic proof of execution environment - Audit trail generation +##### `AttestWithOptions(ctx context.Context, reportData []byte, opts AttestOptions) (*AttestResponse, error)` + +Same as `Attest`, with options. Set `IncludeBoottimeGpuEvidence` to also return the boot-time +GPU attestation evidence, so a verifier gets the quote and the GPU evidence in one round trip. + +```go +resp, err := client.AttestWithOptions(ctx, reportData, dstack.AttestOptions{IncludeBoottimeGpuEvidence: true}) +if err != nil { + log.Fatal(err) +} +fmt.Println(resp.BoottimeGpuEvidence) +``` + +The evidence is the same bytes ``GpuInfo`` serves and is empty unless the flag was set +and boot-time GPU attestation output exists. It is not bound to `report_data`; verify +it with the measured `gpu-attestation` event digest as described under ``GpuInfo``. + ##### `GpuInfo(ctx context.Context) (*GpuInfoResponse, error)` Returns GPU information collected during boot. Currently, this includes the diff --git a/sdk/go/dstack/client.go b/sdk/go/dstack/client.go index 8985cae34..b963deb1c 100644 --- a/sdk/go/dstack/client.go +++ b/sdk/go/dstack/client.go @@ -111,6 +111,20 @@ func (r *GetQuoteResponse) DecodeEventLog() ([]EventLog, error) { // Represents the response from an attestation request. type AttestResponse struct { Attestation []byte + // BoottimeGpuEvidence is the complete JSON output produced by nvattest during guest + // boot. Empty unless the request set IncludeBoottimeGpuEvidence and the guest has + // boot-time GPU attestation output. + // + // It is not bound to reportData: verify it by replaying the runtime event log + // and comparing sha256 of these exact UTF-8 bytes against evidence_sha256 in + // the `gpu-attestation` event. + BoottimeGpuEvidence string +} + +// AttestOptions tunes what an Attest call returns. +type AttestOptions struct { + // IncludeBoottimeGpuEvidence also returns the boot-time GPU attestation evidence. + IncludeBoottimeGpuEvidence bool } // GpuInfoResponse contains GPU information collected during boot. @@ -503,12 +517,19 @@ func (c *DstackClient) GetQuote(ctx context.Context, reportData []byte) (*GetQuo // Gets a versioned attestation from the dstack service. func (c *DstackClient) Attest(ctx context.Context, reportData []byte) (*AttestResponse, error) { + return c.AttestWithOptions(ctx, reportData, AttestOptions{}) +} + +// Gets a versioned attestation from the dstack service, optionally bundling the +// boot-time GPU attestation evidence so a verifier can check both in one round trip. +func (c *DstackClient) AttestWithOptions(ctx context.Context, reportData []byte, opts AttestOptions) (*AttestResponse, error) { if len(reportData) > 64 { return nil, fmt.Errorf("report data is too large, it should be at most 64 bytes") } payload := map[string]interface{}{ - "report_data": hex.EncodeToString(reportData), + "report_data": hex.EncodeToString(reportData), + "include_boottime_gpu_evidence": opts.IncludeBoottimeGpuEvidence, } data, err := c.sendRPCRequest(ctx, "/Attest", payload) @@ -518,6 +539,7 @@ func (c *DstackClient) Attest(ctx context.Context, reportData []byte) (*AttestRe var response struct { Attestation string `json:"attestation"` + BoottimeGpuEvidence string `json:"boottime_gpu_evidence"` } if err := json.Unmarshal(data, &response); err != nil { return nil, err @@ -528,7 +550,7 @@ func (c *DstackClient) Attest(ctx context.Context, reportData []byte) (*AttestRe return nil, err } - return &AttestResponse{Attestation: attestation}, nil + return &AttestResponse{Attestation: attestation, BoottimeGpuEvidence: response.BoottimeGpuEvidence}, nil } // GpuInfo returns GPU information collected during boot. diff --git a/sdk/go/dstack/client_test.go b/sdk/go/dstack/client_test.go index 0ca8d1185..8e82bbef7 100644 --- a/sdk/go/dstack/client_test.go +++ b/sdk/go/dstack/client_test.go @@ -78,6 +78,37 @@ func TestAttest(t *testing.T) { } } +func TestAttestWithBoottimeGpuEvidence(t *testing.T) { + const evidence = `{"result_code":0,"claims":[]}` + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/Attest" { + t.Fatalf("unexpected path: %s", r.URL.Path) + } + var payload map[string]interface{} + if err := json.NewDecoder(r.Body).Decode(&payload); err != nil { + t.Fatalf("failed to decode request: %v", err) + } + if payload["include_boottime_gpu_evidence"] != true { + t.Fatalf("expected include_boottime_gpu_evidence to be forwarded, got: %v", payload["include_boottime_gpu_evidence"]) + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]string{ + "attestation": "deadbeef", + "boottime_gpu_evidence": evidence, + }) + })) + defer server.Close() + + client := dstack.NewDstackClient(dstack.WithEndpoint(server.URL)) + resp, err := client.AttestWithOptions(context.Background(), []byte("test"), dstack.AttestOptions{IncludeBoottimeGpuEvidence: true}) + if err != nil { + t.Fatal(err) + } + if resp.BoottimeGpuEvidence != evidence { + t.Fatalf("unexpected gpu evidence: %s", resp.BoottimeGpuEvidence) + } +} + func TestGpuInfo(t *testing.T) { const attestation = `{"result_code":0,"claims":[]}` server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { diff --git a/sdk/js/README.md b/sdk/js/README.md index 7594e3eb1..f289e6052 100644 --- a/sdk/js/README.md +++ b/sdk/js/README.md @@ -87,7 +87,7 @@ quote.quote // hex-encoded TDX quote quote.event_log // JSON string of measured events ``` -### `attest(reportData)` +### `attest(reportData, includeBoottimeGpuEvidence?)` Versioned dstack attestation that works across TDX / GCP / Nitro providers. Preferred for cross-platform verifiers. @@ -95,6 +95,17 @@ Versioned dstack attestation that works across TDX / GCP / Nitro providers. Pref const { attestation } = await client.attest('app-state-snapshot') ``` +Pass `true` as the second argument to also return the boot-time GPU attestation +evidence, so a verifier gets the quote and the GPU evidence in one round trip. + +```typescript +const { attestation, boottime_gpu_evidence } = await client.attest('app-state-snapshot', true) +``` + +The evidence is the same bytes ``gpuInfo()`` serves and is empty unless the flag was set +and boot-time GPU attestation output exists. It is not bound to `report_data`; verify +it with the measured `gpu-attestation` event digest as described under ``gpuInfo()``. + ### `gpuInfo()` Returns GPU information collected during boot. Currently, this includes the diff --git a/sdk/js/src/__tests__/index.test.ts b/sdk/js/src/__tests__/index.test.ts index c5cc3fa6d..6c0b5e8ce 100644 --- a/sdk/js/src/__tests__/index.test.ts +++ b/sdk/js/src/__tests__/index.test.ts @@ -53,6 +53,16 @@ describe('DstackClient', () => { const result = await client.attest('test') expect(result).toHaveProperty('attestation') expect(result.attestation).not.toBe('') + expect(result.boottime_gpu_evidence).toBe('') + }) + + it('should be able to attest with gpu evidence', async () => { + const client = new DstackClient() + const result = await client.attest('test', true) + expect(result).toHaveProperty('attestation') + expect(result.attestation).not.toBe('') + // Whether evidence exists depends on the host; assert the field is present. + expect(result).toHaveProperty('boottime_gpu_evidence') }) it('should able to get derive key result as uint8array', async () => { diff --git a/sdk/js/src/index.ts b/sdk/js/src/index.ts index dc9f64e8e..7508c8f7a 100644 --- a/sdk/js/src/index.ts +++ b/sdk/js/src/index.ts @@ -101,6 +101,17 @@ export interface AttestResponse { __name__: Readonly<'AttestResponse'> attestation: Hex + + /** + * Complete JSON output produced by nvattest during guest boot. Empty unless the + * request set `include_boottime_gpu_evidence` and the guest has boot-time GPU attestation + * output. + * + * Not bound to `report_data`: verify it by replaying the runtime event log and + * comparing sha256 of these exact UTF-8 bytes against `evidence_sha256` in the + * `gpu-attestation` event. + */ + boottime_gpu_evidence: string } export interface GpuInfoResponse { @@ -285,13 +296,19 @@ export class DstackClient { return Object.freeze(result) } - async attest(report_data: string | Buffer | Uint8Array): Promise { + /** + * Requests a versioned attestation for the given report data. + * + * Pass `include_boottime_gpu_evidence` to also return the boot-time GPU attestation + * evidence in `boottime_gpu_evidence`, so a verifier can check both in one round trip. + */ + async attest(report_data: string | Buffer | Uint8Array, include_boottime_gpu_evidence: boolean = false): Promise { let hex = to_hex(report_data) if (hex.length > 128) { throw new Error(`Report data is too large, it should be less than 64 bytes.`) } - const payload = JSON.stringify({ report_data: hex }) - const result = await send_rpc_request<{ attestation: string }>(this.endpoint, '/Attest', payload) + const payload = JSON.stringify({ report_data: hex, include_boottime_gpu_evidence }) + const result = await send_rpc_request<{ attestation: string, boottime_gpu_evidence?: string }>(this.endpoint, '/Attest', payload) if ('error' in (result as any)) { const err = (result as any)['error'] as string throw new Error(err) @@ -299,6 +316,7 @@ export class DstackClient { return Object.freeze({ __name__: 'AttestResponse', attestation: result.attestation as Hex, + boottime_gpu_evidence: result.boottime_gpu_evidence ?? '', }) } diff --git a/sdk/python/README.md b/sdk/python/README.md index 8788a7269..907145e3f 100644 --- a/sdk/python/README.md +++ b/sdk/python/README.md @@ -100,6 +100,18 @@ print(result.attestation) # hex string print(result.decode_attestation()) # bytes ``` +Pass `include_boottime_gpu_evidence=True` to also return the boot-time GPU attestation +evidence, so a verifier gets the quote and the GPU evidence in one round trip. + +```python +result = client.attest(b'user:alice:nonce123', include_boottime_gpu_evidence=True) +print(result.boottime_gpu_evidence) +``` + +The evidence is the same bytes ``gpu_info()`` serves and is empty unless the flag was set +and boot-time GPU attestation output exists. It is not bound to `report_data`; verify +it with the measured `gpu-attestation` event digest as described under ``gpu_info()``. + ### GPU Info `gpu_info()` returns GPU information collected during boot. Currently, this diff --git a/sdk/python/src/dstack_sdk/dstack_client.py b/sdk/python/src/dstack_sdk/dstack_client.py index 96b40e577..c27571a8c 100644 --- a/sdk/python/src/dstack_sdk/dstack_client.py +++ b/sdk/python/src/dstack_sdk/dstack_client.py @@ -152,6 +152,12 @@ def decode_event_log(self) -> "List[EventLog]": class AttestResponse(BaseModel): attestation: str + # Complete JSON output produced by nvattest during guest boot. Empty unless + # the request set include_boottime_gpu_evidence and the guest has boot-time GPU + # attestation output. Not bound to report_data: verify it by replaying the + # runtime event log and comparing sha256 of these exact UTF-8 bytes against + # evidence_sha256 in the `gpu-attestation` event. + boottime_gpu_evidence: str = "" def decode_attestation(self) -> bytes: return bytes.fromhex(self.attestation) @@ -427,8 +433,13 @@ async def get_quote( async def attest( self, report_data: str | bytes, + include_boottime_gpu_evidence: bool = False, ) -> AttestResponse: - """Request a versioned attestation for the provided report data.""" + """Request a versioned attestation for the provided report data. + + Set include_boottime_gpu_evidence to also return the boot-time GPU attestation + evidence in AttestResponse.boottime_gpu_evidence. + """ if not report_data or not isinstance(report_data, (bytes, str)): raise ValueError("report_data can not be empty") report_bytes: bytes = ( @@ -437,7 +448,13 @@ async def attest( if len(report_bytes) > 64: raise ValueError("report_data must be less than 64 bytes") hex = binascii.hexlify(report_bytes).decode() - result = await self._send_rpc_request("Attest", {"report_data": hex}) + result = await self._send_rpc_request( + "Attest", + { + "report_data": hex, + "include_boottime_gpu_evidence": include_boottime_gpu_evidence, + }, + ) return AttestResponse(**result) async def gpu_info(self) -> GpuInfoResponse: @@ -569,8 +586,13 @@ def get_quote( def attest( self, report_data: str | bytes, + include_boottime_gpu_evidence: bool = False, ) -> AttestResponse: - """Request a versioned attestation for the provided report data.""" + """Request a versioned attestation for the provided report data. + + Set include_boottime_gpu_evidence to also return the boot-time GPU attestation + evidence in AttestResponse.boottime_gpu_evidence. + """ raise NotImplementedError @call_async diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index 80724a22c..75f5efcfa 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -121,6 +121,24 @@ async def test_async_client_attest(): assert len(result.attestation) > 0 +@pytest.mark.asyncio +async def test_async_client_attest_boottime_gpu_evidence(monkeypatch): + evidence = '{"result_code":0,"claims":[]}' + + async def fake_send(self, method, payload): + assert method == "Attest" + assert payload["include_boottime_gpu_evidence"] is True + return {"attestation": "deadbeef", "boottime_gpu_evidence": evidence} + + monkeypatch.setenv("DSTACK_SIMULATOR_ENDPOINT", "http://localhost:0") + monkeypatch.setattr(AsyncDstackClient, "_send_rpc_request", fake_send) + result = await AsyncDstackClient().attest( + "test", include_boottime_gpu_evidence=True + ) + assert isinstance(result, AttestResponse) + assert result.boottime_gpu_evidence == evidence + + @pytest.mark.asyncio async def test_async_client_gpu_info(monkeypatch): attestation = '{"result_code":0,"claims":[]}' diff --git a/sdk/rust/README.md b/sdk/rust/README.md index 2f00b0cec..4cae1a949 100644 --- a/sdk/rust/README.md +++ b/sdk/rust/README.md @@ -103,6 +103,24 @@ println!("{}", info.tcb_info); #### `attest(report_data: Vec) -> AttestResponse` Generates a versioned attestation with a custom 64-byte payload. - `attestation`: Hex-encoded attestation +- `boottime_gpu_evidence`: Boot-time GPU attestation evidence, empty unless requested + +#### `attest_with(config: AttestConfig) -> AttestResponse` +Same, with options. Set `include_boottime_gpu_evidence` to also return the boot-time GPU +attestation evidence, so a verifier gets the quote and the GPU evidence in one round trip. + +```rust +let config = AttestConfig::builder() + .report_data(hex::encode(b"user:alice:nonce123")) + .include_boottime_gpu_evidence(true) + .build(); +let result = client.attest_with(config).await?; +println!("{}", result.boottime_gpu_evidence); +``` + +The evidence is the same bytes ``gpu_info()`` serves and is empty unless the flag was set +and boot-time GPU attestation output exists. It is not bound to `report_data`; verify +it with the measured `gpu-attestation` event digest as described under ``gpu_info()``. #### `gpu_info() -> GpuInfoResponse` diff --git a/sdk/rust/src/dstack_client.rs b/sdk/rust/src/dstack_client.rs index c6f824eeb..eb8e60200 100644 --- a/sdk/rust/src/dstack_client.rs +++ b/sdk/rust/src/dstack_client.rs @@ -4,7 +4,7 @@ // // SPDX-License-Identifier: Apache-2.0 -use anyhow::Result; +use anyhow::{Context, Result}; use hex::encode as hex_encode; use http_client_unix_domain_socket::{ClientUnix, Method}; use reqwest::Client; @@ -152,13 +152,25 @@ impl DstackClient { Ok(response) } + /// Requests a versioned attestation for the given report data. pub async fn attest(&self, report_data: Vec) -> Result { + self.attest_with( + AttestConfig::builder() + .report_data(hex_encode(&report_data)) + .build(), + ) + .await + } + + /// Requests a versioned attestation, optionally bundling the boot-time GPU + /// attestation evidence so a verifier can check both in one round trip. + pub async fn attest_with(&self, config: AttestConfig) -> Result { + let report_data = + hex::decode(&config.report_data).context("Invalid report data encoding")?; if report_data.is_empty() || report_data.len() > 64 { anyhow::bail!("Invalid report data length") } - let hex_data = hex_encode(report_data); - let data = json!({ "report_data": hex_data }); - let response = self.send_rpc_request("/Attest", &data).await?; + let response = self.send_rpc_request("/Attest", &config).await?; let response = serde_json::from_value::(response)?; Ok(response) diff --git a/sdk/rust/tests/test_client.rs b/sdk/rust/tests/test_client.rs index 6bd636c0d..73043099b 100644 --- a/sdk/rust/tests/test_client.rs +++ b/sdk/rust/tests/test_client.rs @@ -6,7 +6,7 @@ // SPDX-License-Identifier: Apache-2.0 use dcap_qvl::quote::Quote; -use dstack_sdk::dstack_client::DstackClient as AsyncDstackClient; +use dstack_sdk::dstack_client::{AttestConfig, DstackClient as AsyncDstackClient}; use dstack_sdk::verify::verify_signature; use sha2::{Digest, Sha256}; @@ -31,11 +31,34 @@ async fn test_async_client_attest() { let result = client.attest(b"test".to_vec()).await.unwrap(); let attestation = result.decode_attestation().unwrap(); assert!(!attestation.is_empty()); + assert!(result.boottime_gpu_evidence.is_empty()); let too_large = client.attest(vec![0_u8; 65]).await; assert!(too_large.is_err()); } +#[tokio::test] +async fn test_async_client_attest_with_boottime_gpu_evidence() { + let client = AsyncDstackClient::new(None); + let config = AttestConfig::builder() + .report_data(hex::encode(b"test")) + .include_boottime_gpu_evidence(true) + .build(); + let result = client.attest_with(config).await.unwrap(); + assert!(!result.decode_attestation().unwrap().is_empty()); + // Whether evidence exists depends on the host, so assert the request + // round-trips and the field is populated from the same source as GpuInfo. + assert_eq!( + result.boottime_gpu_evidence, + client.gpu_info().await.unwrap().attestation + ); + + let too_large = AttestConfig::builder() + .report_data(hex::encode([0_u8; 65])) + .build(); + assert!(client.attest_with(too_large).await.is_err()); +} + #[tokio::test] async fn test_async_client_get_tls_key() { let client = AsyncDstackClient::new(None); diff --git a/sdk/rust/types/src/dstack.rs b/sdk/rust/types/src/dstack.rs index 18144e08b..93789c24e 100644 --- a/sdk/rust/types/src/dstack.rs +++ b/sdk/rust/types/src/dstack.rs @@ -96,6 +96,28 @@ pub struct GetQuoteResponse { pub struct AttestResponse { /// The attestation in hexadecimal format pub attestation: String, + /// Complete JSON output produced by nvattest during guest boot. Empty + /// unless the request set `include_boottime_gpu_evidence` and the guest has + /// boot-time GPU attestation output. + /// + /// Not bound to `report_data`: verify it by replaying the runtime event log + /// and comparing sha256 of these exact UTF-8 bytes against + /// `evidence_sha256` in the `gpu-attestation` event. + #[serde(default)] + pub boottime_gpu_evidence: String, +} + +/// Configuration for a versioned attestation request +#[derive(Debug, bon::Builder, Serialize, Deserialize)] +#[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] +#[cfg_attr(feature = "borsh_schema", derive(BorshSchema))] +pub struct AttestConfig { + /// The report data in hexadecimal format, at most 64 bytes once decoded + #[builder(into)] + pub report_data: String, + /// Also return the boot-time GPU attestation evidence in `boottime_gpu_evidence` + #[builder(default = false)] + pub include_boottime_gpu_evidence: bool, } /// Response containing the complete NVIDIA GPU attestation output.