Private inference

Attested nodes and encrypted prompts: how the private tier works and where it stands.

Status: live on one attested node. The node runs on Phala Cloud in an Intel TDX confidential VM whose measurement is the approved Arcloak node image, and serves qwen2.5:7b through Phala's confidential inference API, where the model runs on an NVIDIA GPU in confidential-compute mode. While that node is attested, private/qwen2.5:7b is listed by GET /v1/models and offered by the playground; no other private model is listed yet.

How it works

A model addressed as private/<catalog id> (for example private/llama3.1:8b) is served only by a node that presented a hardware attestation: an Intel TDX quote for its confidential VM plus an NVIDIA GPU quote, bound to the node's address and to the public key it generated inside the enclave, and matched against an image measurement Arcloak has approved. The API verifies that attestation when the node connects and again every 24 hours.

Your request is encrypted in your own process to that enclave key with HPKE (RFC 9180): DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, ChaCha20-Poly1305, info string arcloak-private-v1. The API routes the ciphertext to the node you chose and never sees a prompt. The node decrypts inside the enclave, runs the model, and streams the answer back encrypted to an ephemeral key you made for this one request. Billing keeps its shape: x402 upto, a ceiling from an estimate over the ciphertext length, settlement from the node's token counts, at three times the base rate of the model's tier (Pricing).

What is protected, and what is not

  • Protected: the prompt and the answer. They are hidden from Arcloak, from the node operator's host, and from anyone reading the API's logs or database, who see ciphertext and token counts only.
  • Verified by Arcloak: the node's TDX quote and measurement (the enclave key was generated inside a TDX VM running the approved Arcloak node image), and NVIDIA GPU evidence carrying Arcloak's nonce, checked through NVIDIA's remote attestation service. Both are checked when the node connects and every 24 hours.
  • Not yet verified by Arcloak: Phala's gateway quote and its per-response receipts, which are what tie the specific GPU that answered to the GPU that attested. Buyers who want that link can check it against Phala's confidential AI docs. Arcloak's own checks prove the enclave key and the image, and that an attested GPU answered Phala's gateway with our nonce.
  • Still visible: that you made a request, when, to which model, its size (the ciphertext length), token counts and timing, and your payer address on Arc. Token counts are plaintext because they are what you pay for.
  • Not covered: the model's own behaviour. Attestation proves which image runs, not that the model answers well.
  • Local only: a node started with ATTESTATION=dev presents an HMAC instead of hardware quotes, and the playground marks such a turn dev attestation: not hardware. Such nodes never appear on the public API.

Find an attested node

GET /v1/private/nodes with ?model=<catalog id> lists the nodes that currently serve the base model with a verified, unexpired attestation:

{"nodes":[{"address":"0x7d82…e487","enclaveKey":"0x…","attestation":{"kind":"dstack","measurement":"private-node-2026-09","verifiedAt":"2026-09-29T10:00:00.000Z","expiresAt":"2026-09-30T10:00:00.000Z"}}]}

enclaveKey is the node's X25519 public key, 32 bytes as hex. A dstack entry also carries quote, the raw TDX quote as hex, for buyers who verify it themselves. Pick one node and keep its address: the request goes to exactly that node. The query also accepts model=private/<id>.

Request format

POST /v1/chat/completions with model: "private/<catalog id>" carries no messages. Instead:

  • node: the address you picked.
  • private: { "enc": "<base64>", "ct": "<base64>", "clientKey": "0x<64 hex>" }, where enc is your HPKE encapsulation to the node's enclaveKey, ct is the sealed JSON {"messages":[…],"temperature":0.7} (the same messages shape as an open request; temperature optional), and clientKey is the public half of an X25519 key pair you generated for this request. The ciphertext may be at most 1 MiB.
  • max_tokens and stream: true as usual. Private requests must stream.

The answer's delta.content fields are base64 ciphertext sealed to your clientKey; the first chunk also carries arcloak.enc, the node's encapsulation. Open every chunk, in order, through one recipient context made from that enc. The usage chunk, the x402.payment event and [DONE] are unchanged.

import { CipherSuite, HkdfSha256 } from '@hpke/core'
import { DhkemX25519HkdfSha256 } from '@hpke/dhkem-x25519'
import { Chacha20Poly1305 } from '@hpke/chacha20poly1305'

const suite = new CipherSuite({ kem: new DhkemX25519HkdfSha256(), kdf: new HkdfSha256(), aead: new Chacha20Poly1305() })
const info = new TextEncoder().encode('arcloak-private-v1').buffer
const b64 = (ab: ArrayBuffer) => Buffer.from(ab).toString('base64')
const unb64 = (s: string) => Uint8Array.from(Buffer.from(s, 'base64')).buffer as ArrayBuffer
const hex = (ab: ArrayBuffer) => `0x${Buffer.from(ab).toString('hex')}`
const unhex = (h: string) => Uint8Array.from(Buffer.from(h.slice(2), 'hex')).buffer as ArrayBuffer

const { nodes } = await (await fetch('https://api.arcloak.net/v1/private/nodes?model=llama3.1:8b')).json()
const node = nodes[0]
const client = await suite.kem.generateKeyPair()
const sender = await suite.createSenderContext({ recipientPublicKey: await suite.kem.deserializePublicKey(unhex(node.enclaveKey)), info })
const ct = await sender.seal(new TextEncoder().encode(JSON.stringify({ messages: [{ role: 'user', content: 'Say hello from Arc.' }] })).buffer as ArrayBuffer)

const body = {
  model: 'private/llama3.1:8b',
  node: node.address,
  private: { enc: b64(sender.enc), ct: b64(ct), clientKey: hex(await suite.kem.serializePublicKey(client.publicKey)) },
  max_tokens: 64,
  stream: true,
}
// Send `body` through the paying fetch from the buyer quickstart. One recipient context opens the whole stream:
// it is created from the first chunk's `arcloak.enc` and reused for every later chunk, in order.
//   let recipient: Awaited<ReturnType<typeof suite.createRecipientContext>> | undefined
//   for each chunk with delta.content:
//     recipient ??= await suite.createRecipientContext({ recipientKey: client, enc: unb64(chunk.arcloak.enc), info })
//     const text = new TextDecoder().decode(await recipient.open(unb64(chunk.choices[0].delta.content)))

The playground does all of this in the browser when a private/ model is selected, and shows an "Attested" chip with the measurement name and the verification time. "Show proof" under the chip lists the node address, the enclave key the prompt was sealed to, the SHA-256 of the raw TDX quote, the verification and expiry times, and links to the node listing and to a quote checker; "Copy quote" gives the raw quote to verify elsewhere.

Errors

  • 400 invalid_request: the model is private/… but the body carries messages or lacks stream: true, or an open model got private or node. The message says which.
  • 503 node_unavailable: no attested node serves the model, or the node you named is gone.
  • 503 private_decrypt_failed or private_encrypt_failed: the node could not open your request (wrong key, damaged ciphertext) or could not seal its first chunk. The payment is cancelled; nothing was charged.
  • private_encrypt_failed as a stream error event after the payment event: the node could not seal a later chunk. Only the delivered part was billed. private_decrypt_failed would be relayed the same way if a node reported it after its first chunk; the node opens the request before it streams anything, so this does not occur today.
  • private_required: reserved. The body mix-ups above answer invalid_request; this code is not sent yet.
  • attestation_failed: reserved for a node whose attestation stops verifying mid-session. Not produced yet: such a node keeps serving open models only, and a private request to it answers node_unavailable.