C8s

C8s is built by Confidential AI as the substrate for private AI: inference, agents, training, and fine-tuning where the infrastructure operator never sees the data. The platform itself is workload-agnostic: anything that runs on Kubernetes can run confidentially.

Links

• Confidential AI, the company behind C8s
• Documentation, the full user-facing docs
• Whitepaper, the C8s architecture paper (also on arXiv)
• Your first confidential cluster, an end-to-end tutorial from bare cloud account to verified confidential workload
• c8s-verify, verify a C8s cluster from a browser
• attestation-rs, the TEE evidence verification service C8s uses
• armTLS, how attested TLS works in C8s — the handshake step by step, the guarantees, and which certificate is used where
• Route evidence, how the router binds its route manifest into attestation evidence and how c8s verify checks it

Features

• Hardware-attested workload identity. The Certificate Distribution

• armTLS mesh. A transparent L4 proxy wraps traffic between workloads in

• Node-as-CVM. Run the whole node as one confidential VM. Supported modes

• Measured boot end to end. Node images boot via IGVM with dm-verity.

• Container image and command-line allowlisting. Every container is

• Attestation-gated secrets. CDS releases an application secret only once

• Encrypted volumes. Data too large to be a secret — model weights, in

• Fail-closed admission. A mutating webhook injects certificate sidecars

• Confidential GPUs. NVIDIA GPUs attached to confidential nodes on

• Verifiable from a browser. A challenge-response protocol and a

• One-command install. c8s install --cvm-mode= brings all of this

Architecture

The unit of trust and attestation is the confidential node.

Node-as-CVM

The entire Kubernetes node is one confidential VM. Pods are ordinary containers inside it. A verifier checks the node's launch digest; everything on the node is inside that one boundary, including the kubelet. Every pod shares that boundary, so node-as-CVM is single-tenant: it isolates the node from the host, not workloads from each other. This is the simplest and densest shape, and the only one available on managed services without nested virtualization (for example Azure AKS).

NODE-AS-CVM
              one launch digest covers the whole node
════════════ TEE boundary (SEV-SNP / TDX encrypted memory) ════════════
┌────────────── Kubernetes node = one confidential VM ──────────────┐
  │                                                                   │
  │   ┌─────────┐   ┌─────────┐   ┌─────────┐      ┌───────────────┐  │
  │   │  pod A  │   │  pod B  │   │  pod C  │  ... │ kubelet, CNI, │  │
  │   │ (runc)  │   │ (runc)  │   │ (runc)  │      │ containerd    │  │
  │   └─────────┘   └─────────┘   └─────────┘      └───────────────┘  │
  │                                                                   │
  │   measured boot: IGVM + UKI + dm-verity node image                │
  └───────────────────────────────────────────────────────────────────┘
═══════════════════════════════════════════════════════════════════════
HOST / HYPERVISOR (untrusted)
  the cloud or bare-metal operator sees only ciphertext

Quickstart

Install C8s onto an existing cluster. The full walkthrough is docs/QUICKSTART.md; the hosted version with provisioning guides is at confidential.ai/docs/c8s.

Prerequisites

• A Kubernetes cluster (vanilla or RKE2) with platform-admin permissions.
• SEV-SNP / TDX confidential VMs as nodes for node-as-CVM

• Helm 3, kubectl, and crane on PATH.
• Go 1.27+ to build the CLI.

Install

# Build and install the c8s CLI
git clone https://github.com/confidential-dot-ai/c8s
cd c8s
make install
Label the node that will run CDS
kubectl label node <cds-node> role=cds
Generate the operator keypair whose public half authorizes allowlist writes
openssl ecparam -name prime256v1 -genkey -noout -out operator.key
openssl ec -in operator.key -pubout -out operator.pub
Install the platform (node-as-CVM) and route the bundled router to your
workload, which serves TLS on 8443 with its injected certificate; install
prints the c8s routes add commands that store the route
c8s install --cvm-mode=bare-metal --hardware-platform=sev-snp --namespace c8s-system \
  --operator-keys operator.pub \
  --workload-ref vllm=vllm/deployment/serving:8443 \
  --upstream vllm

--cvm-mode is required and has no default — bare-metal, gke, and aks are the node-as-CVM shapes. --hardware-platform is required the same way: sev-snp or tdx. An unstated shape would silently mismatch the cluster it lands on, so the install refuses to guess.

--workload-ref adopts an existing workload as a confidential workload and resolves its images into the bootstrap allowlist (see existing workload adoption). If you would rather not trust install-time digest resolution, pass --resolve-digests=false and allowlist the digests yourself — see Managing the image allowlist.

Opt workloads in

Application teams opt in by annotating their pod templates:

metadata:
  annotations:
    confidential.ai/cw: api

The annotation value (api here) is a workload id you choose; the certificate SAN and the c8s- headless Service are derived from it.

The webhook injects c8s get-cert as a native sidecar, which fetches an attestation-bound certificate from CDS and renews it. Certificates land in /etc/c8s/certs.

Production notes

• Pin measurements. The chart's armTLS handshakes accept any TEE-attested

• Pin operator keys. Pass --operator-keys at install time or allowlist

A note on QEMU

• Node-as-CVM needs QEMU 10.1 or newer, built with --enable-igvm.

Verifying a cluster from outside

Anyone can verify that a C8s endpoint really terminates inside attested hardware, without trusting the operator's word for it.

Browser JavaScript cannot inspect certificate evidence during a TLS handshake. The c8s-verify npm package instead runs a challenge-response protocol: the client sends a fresh nonce and an X-Wing encapsulation key, the TEE returns a hardware-signed attestation report binding the complete key exchange in one round trip, and all further traffic flows over a post-quantum over-encrypted channel (X-Wing: X25519 + ML-KEM-768) inside the regular TLS session. A malicious TLS-terminating proxy in front of the real endpoint cannot forge it. The wire contract is PROTOCOL.md.

Operators and CLIs can verify directly: c8s cds verify checks a CDS's attestation and reports the operator keys it pins.

Components

| Component | Description | Docs | |---|---|---| | cmd/cds | Certificate Distribution Service - verifies TEE attestation evidence, signs workload CSRs with an in-process mesh CA, and serves the allowlist and secret-release APIs | operator docs | | cmd/c8s | Operator and install CLI for CRDs, status mirroring, webhook injection, and the embedded Helm chart | operator docs | | cmd/get-cert | CLI tool and init-container for TEE-attested certificate provisioning | README | | cmd/armtls-mesh | Transparent L4 proxy wrapping inter-node K8s traffic in armTLS | README | | cmd/nri-image-policy | NRI plugin enforcing the image and argv allowlist on the host; also the node's admission inventory | allowlist | | internal/cmds/volumed | Encrypted-volume agent — opens volumes into a pod's mount namespace as a node DaemonSet | volumes |

Libraries

| Package | Description | |---|---| | pkg/armtls | armTLS library for hardware-attested mTLS (AMD SEV-SNP, Intel TDX) — see docs/armtls.md | | pkg/armtls/cdsclient | CDS attestation client for certificate provisioning | | pkg/attestclient | High-level client for the CDS attestation flow | | pkg/allowlistclient | CRUD client for the CDS allowlist API | | pkg/allowlist | Allowlist types, argv policy, and secret grants | | pkg/workloadclaims | Sandbox-token fetch and the admission-inventory socket contract | | pkg/overenc | Post-quantum over-encryption channel and its identity transcript | | pkg/operatorauth | Operator-key signing and verification for allowlist and secret writes | | pkg/types | Shared request/response types for the C8s protocols (the attestation-api wire types live in attestation-go/remote) | | pkg/runtimemeasure | TDX image-pin manifests and RTMR[3] measurement replay | | pkg/certutil | Certificate utility functions |

Repository layout

api/               CRD types
cmd/               Binaries: c8s, get-cert, armtls-mesh, nri-image-policy
                   (cmd/cds is only
                   the Dockerfile for the c8s cds subcommand,
                   internal/cmds/cds)
internal/          Operator, webhook, attestation, mesh CA, secret store,
                   embedded Helm chart
pkg/               Public Go libraries (see Libraries above)
node-guest-image/  The node-image definition for node-as-CVM (new home;
                   phase 0 — nothing consumes it from here yet)
docs/              Design and operator docs
samples/           Example manifests
scripts/           Dev and CI helpers
test/              Integration tests: docker-compose get-cert flow
                   (test/integration) and the kind cluster harness
                   (test/integration/cluster, see docs/integration-tests.md)

Build

Requires Go 1.27+.

# Build the c8s binary for the container images (linux/amd64)
make build
Build and install the c8s CLI for your host platform, onto PATH
make install
Run tests
make test
Mutation-test the changes vs origin/main (BASE=<ref> to override)
make mutation-check
Lint (format check + vet)
make lint
Clean build artifacts
make clean

Managing the image allowlist

CDS serves the image-digest allowlist that nri-image-policy enforces on every node. The c8s allowlist command reads and mutates it. By default, router publishes the complete /allowlist API and verifies CDS's attestation before forwarding requests. When router uses the chart default CDS-issued public certificate (router.publicTLS.secretName is empty, discovery mode cds), point the CLI at the same router URL used for application traffic; no port-forward is required.

ROUTER=https://<router-host>
Reads are unauthenticated
c8s allowlist export --url "$ROUTER" \
  --measurements <router-launch-digest> > allowlist.json
c8s allowlist diff allowlist.json --url "$ROUTER" \
  --measurements <router-launch-digest>
Writes are signed with the operator key. 'add' admits an image under any
command line; 'apply' or 'derive' pins one or grants secrets.
c8s allowlist add sha256:<digest> registry.example.com/app@sha256:<digest> \
  --url "$ROUTER" --measurements <router-launch-digest> \
  --operator-key operator.key
c8s allowlist upload allowlist.json \
  --url "$ROUTER" --measurements <router-launch-digest> \
  --operator-key operator.key

For a bare-metal install, pass --cvm-mode bare-metal to allowlist upload; its required component set uses the measured host attestation service. GKE, AKS, and uploads without a mode require the attestation-api image. --require overrides the required set.

--measurements identifies the trusted build of the endpoint you connected to. For the default public route, use the router launch digest; the CLI reads router's discovery document and verifies its attestation automatically. Direct CDS URLs remain supported, in which case pin the CDS launch digest.

An empty set accepts any attested endpoint. Reads run with a warning; anything that signs with the operator key — every c8s allowlist write and c8s secrets put/explain — is refused, because the credential and its payload would go to whatever answered. A plaintext --insecure dev endpoint is exempt: it already declares that nothing about it is attested.

Do not point this CLI at router when router.publicTLS.secretName is set. That front door uses WebPKI (public_tls.mode=webpki), and its public certificate is not cryptographically bound to the discovery attestation, so the CLI deliberately refuses it. Use a direct CDS armTLS URL and the CDS launch digest; if CDS is not otherwise routable, use a local port-forward:

kubectl port-forward -n c8s-system svc/c8s-cds 8443:8443 &
c8s allowlist export --url https://localhost:8443 \
  --measurements <cds-launch-digest>

Writes are authorized by an operator EC keypair whose public half CDS pins at install time. Generate one and pin it:

openssl ecparam -name prime256v1 -genkey -noout -out operator.key
openssl ec -in operator.key -pubout -out operator.pub
c8s install --operator-keys operator.pub   # plus your other install flags

Installing without --operator-keys leaves allowlist writes disabled and CDS unable to store a route manifest, so the router routes nothing. c8s install refuses that on the default path unless you pass --force to acknowledge. Supply the private key to the CLI by flag (--operator-key) or environment (C8S_OPERATOR_KEY). Write tokens are short-lived and bound to the request body, so a captured token cannot be replayed against a different payload. The private key remains on the operator machine: router forwards only the signed request and CDS verifies it against the pinned public key.

Set router.allowlist.enabled=false to remove the built-in public route. A direct CDS connection, including a local port-forward for debugging, can still be passed explicitly with --url.

Verify CDS's root of trust — its launch measurement and the operator key set it serves — before exposing public ingress, using your own install inputs (the operator public-key bundle). The key list is fetched over the attested serving certificate, so it cannot be substituted in transit (docs/armtls.md):

c8s cds verify https://<cds>:8443 \
  --measurements <launch-digest> \
  --operator-keys operator.pub

A swapped key set fails this closed. The check protects the verifier that runs it, so run it continuously (CI), not only at bootstrap.

Two caveats worth knowing before production: revocation is currently coarse (remove the key from cds.operatorKeys and re-install), and this check protects only verifiers that run it. For GitOps consumers, c8s render-values --operator-keys operator.pub embeds the PEM content (the chart value takes content, never a file path); the chart wiring is described in docs/operator.md.

Manage the route manifest

c8s routes (show, digest, put, add, rm) reads and writes the signed route manifest that CDS stores at /routes, with the operator key used for allowlist writes. Point --url at CDS directly, for example through a port-forward, because the router does not relay /routes. The commands only store the manifest: the router does not apply it yet. See Route manifest.

Docker images

All images are published to GHCR on pushes to main. Release-worthy merges also receive stable vX.Y.Z, X.Y.Z, and X.Y aliases in that same workflow run. Per-role image names remain stable, but each image copies the same multi-mode c8s binary and sets an appropriate entrypoint. See docs/releases.md for the version policy.

The measured node-guest-base artifacts follow the same root release with platform-qualified exact aliases such as rke2-tdx-v0.1.0 and rke2-snp-cdi-v0.1.0; they intentionally have no ambiguous bare or moving SemVer alias.

| Image | Base | Notes | |---|---|---| | ghcr.io/confidential-dot-ai/c8s-operator | distroless | Multi-mode c8s binary for operator/install and non-node roles | | ghcr.io/confidential-dot-ai/cds | distroless | | | ghcr.io/confidential-dot-ai/get-cert | distroless | | | ghcr.io/confidential-dot-ai/armtls-mesh | debian-slim | Needs iptables | | ghcr.io/confidential-dot-ai/nri-image-policy | debian-slim | | | ghcr.io/confidential-dot-ai/volumed | debian-slim | Needs cryptsetup/veritysetup |

The chart also deploys ghcr.io/confidential-dot-ai/attestation-api, the TEE evidence verification service, which is built and published from attestation-rs.

Known gaps and open items

C8s is built around a strong threat model, and we would rather list the holes than let you discover them:

• Measurements are not pinned by default. Until cds.measurements and

• CDS is a singleton. The mesh CA key lives only in CDS process

• Secrets and volume keys live only in CDS memory. There is no persistent

• Mesh peers are verified by CA chain, not per-peer measurement. Leaves

• A workload's sandbox identity is CA-vouched, not hardware-bound. The

• **The image allowlist gates digest and command line, not the rest of the

• Init containers cannot consume a released secret. The secret volume is

• **Encrypted volumes are attached at node boot, and a force-deleted volume

• GPU attestation is not wired end to end. GPU passthrough into

• The browser over-encryption channel does not stream. Requests and

• Operator key revocation is coarse. No CRL/OCSP; revoking an operator

Roadmap

The direction of travel:

• GPU attestation end to end. Require GPU evidence at certificate

• In-TEE volume encryption. Stream plaintext into a CVM that generates the

• Encrypted RDMA. Encrypted GPU-to-GPU and node-to-node RDMA for

Standing on the shoulders of the community

C8s exists because a lot of excellent open work came before it, and we want to be loud about that:

• Confidential Containers

• The Confidential Computing Consortium

Where we fix or extend something upstream, we aim to contribute it back.

Contributing and security

Please, before you open an issue or a PR, read CONTRIBUTING.md. It is short, and it explains the contribution terms (you sign the CLA on your first PR), the review bar, our policy on LLM-assisted contributions, and the requirement that commits be signed.

And before you report anything security-shaped, read SECURITY.md. C8s is trust infrastructure: attestation bypasses, policy bypasses, and certificate mis-issuance are security issues. Do not open public issues for them. Email [email protected] instead.

For anything else: [email protected].

Licence

C8s is licensed under the GNU Affero General Public License v3.0. Contributions are accepted under the terms in CONTRIBUTING.md.

Related repositories

• confidential-dot-ai/c8s-verify-js - browser-side cluster verification library (npm: c8s-verify)
• confidential-dot-ai/attestation-rs - TEE attestation evidence verification service (publishes the attestation-api image)
• confidential-dot-ai/attestation-go - TEE attestation evidence verification library for go