Google Has Encrypted Prompts Too: Prompt Encryption SDK Officially Open-Sourced

On July 6, Google announced three things at once: the open-source release of the Prompt Encryption SDK, the preview launch of Confidential G4 VM based on NVIDIA RTX PRO 6000 Blackwell, and the addition of Intel Trust Authority attestation and Hopper GPU support to Confidential Space. Together, these three announcements tell the same story: moving confidential computing from the CPU era into the GPU inference era, and for the first time bringing “the prompt itself” into the encryption boundary.

For teams building enterprise AI applications, this is a significant update. Over the past year, there has been extensive discussion around AI security, but most solutions have remained at the transport encryption (TLS) and storage encryption (KMS) layers. The moment a prompt enters the model, it exists in plaintext — complete prompts and responses sit in the inference server’s memory, visible to cloud providers, operators, or even compromised host machines. Google now wants to close this “data-in-use” gap using hardware TEE plus an SDK.

What Problem Does Prompt Encryption SDK Actually Solve?

Here’s the short version: the SDK establishes an attested encrypted channel between the client and the inference server running inside a TEE (Trusted Execution Environment).

The workflow roughly looks like this:

1. Before sending a request, the client asks the target inference server for an “attestation report” — a cryptographically signed snapshot generated by the TEE chip of the current runtime environment, including measurements such as the hypervisor, guest OS, and image hash.
2. After receiving the report, the client verifies whether the signature comes from trusted hardware and whether the measurements match the expected image hash. If anything doesn’t match, the SDK immediately throws an AttestationError, and the prompt is never sent.
3. After verification succeeds, the client encrypts the prompt using the public key bound to the report. The ciphertext is only decrypted once it reaches the inside of the TEE, and model outputs are likewise encrypted inside the TEE before being returned.

The key point in this mechanism is step 2 — the trusted entity is not the abstract concept of “Google Cloud,” but the exact binary currently running on that specific machine. Even if Google’s own operators tried to SSH in and dump memory, they still could not access plaintext data because the host machine does not possess the decryption keys.

Google has directly published the SDK on GitHub (google/prompt-encryption-sdk) with a Codelab included. The Python client usage is roughly: run a setup.sh script to start the server inside Confidential Space, then use ConfidentialSDKClient on the client side to send requests, with the SDK automatically handling the attestation handshake. The engineering integration overhead is relatively low, but the inference server must be deployed inside Confidential Space — which is also part of Google’s strategy: the SDK is free and open source, but the confidential compute infrastructure still runs on Google’s confidential VMs.

Confidential G4 VM: Blackwell Finally Enters Confidential Computing

An SDK alone is not enough — there also needs to be TEE hardware capable of handling large-model inference. The newly launched Confidential G4 VM (preview) uses NVIDIA’s RTX PRO 6000 Blackwell Server Edition GPU together with the 5th Gen AMD EPYC Turin CPU, relying on AMD SEV-SNP for memory encryption and virtual machine isolation.

One technical detail here is particularly worth noting: over the past two years, the biggest bottleneck in confidential computing has been data transfer between CPU and GPU. In traditional architectures, even if the CPU side runs inside a TEE, data still needs to be decrypted once it is transferred over PCIe to the GPU — effectively leaving the door half open. Starting with Hopper H100, NVIDIA extended TEE support to GPUs, and the Blackwell generation further expanded “near-unencrypted performance” to rack-scale systems (which is also a key selling point of NVIDIA’s Vera Rubin NVL72). The RTX PRO 6000 is a workstation/inference-oriented model within the Blackwell family, featuring 96GB of GDDR7 VRAM, making it suitable for medium-scale confidential inference workloads and easier to deploy than HGX B200 systems costing hundreds of thousands of dollars.

From a product positioning perspective, Google’s strategy here is very clear:

• A3 series (H100): confidential solutions for training and large-scale inference, already generally available for some time;
• G4 series (RTX PRO 6000 Blackwell): focused on inference and enterprise fine-tuning, now in preview;
• C3 series (Intel AMX): CPU-only confidential inference, suitable for small models and embeddings.

Together, these three tiers cover confidential inference workloads ranging from embeddings to 70B-scale models.

Confidential Space Upgrade: A Passport for Cross-Organization Collaborative Training

If Prompt Encryption SDK is designed for consumer-facing API services, then Confidential Space is aimed at B2B and B2G data collaboration scenarios.

This upgrade includes two major additions:

1. Integration with Intel Trust Authority as an independent third-party attestation service. Previously, attestation reports were either verified by Google itself or by clients operating their own verification services. Now there is a neutral third-party authority providing trust guarantees — crucial in multi-party collaborative training scenarios where no participant fully trusts another, but all parties trust Intel.
2. Added Hopper GPU support. Previously, Confidential Space only supported CPU-side TEEs, making large-model training impractical. Now it can use H100 clusters for federated training, enabling scenarios such as hospitals jointly training cancer imaging models or banks collaboratively building anti-money laundering models.

The two typical use cases Google highlighted are representative:

• Healthcare: multiple hospitals jointly train lesion detection models using their own MRI/CT datasets. Confidential Space ensures that no hospital’s raw images leave its local trust boundary, with only aggregated gradients entering the TEE;
• Finance: multiple banks share suspicious transaction information to train AML models without exposing customer privacy, while still allowing the model to learn industry-wide patterns.

These kinds of collaborations have been attempted before using differential privacy and homomorphic encryption, but either accuracy degraded too much or computational overhead became impractical. The TEE approach essentially trades cryptographic overhead for hardware trust, keeping performance loss within 10% and making deployment more feasible from an engineering standpoint.

Several Issues Still Need a Clear-Eyed View

Despite the excitement, several objective limitations need to be acknowledged:

First, the attestation ecosystem is still not standardized. Google uses AMD SEV-SNP plus NVIDIA Confidential Compute, while Azure is pushing its own approach based on Intel TDX. Current client SDKs are effectively tied to individual cloud vendors. Companies deploying across multiple clouds either need to build their own abstraction layer or wait for cross-vendor standards from the Confidential Computing Consortium (CCC).

Second, Prompt Encryption SDK currently only covers the path from the client to the inference server. If you are using a RAG architecture, the vector retrieval portion still requires separate consideration — the SDK does not handle that.

Third, performance overhead still exists. TEE memory encryption, attestation handshakes, and key negotiation all introduce additional cost. In high-concurrency short-prompt scenarios (such as customer service bots), first-token latency will increase noticeably. Google has not published benchmark data yet, so real-world numbers will have to wait until GA and broader community testing.

Fourth, this approach is positioned against solutions like OpenAI’s ZDR (Zero Data Retention) and similar promises from Anthropic. Cloud providers are effectively saying: instead of trusting contractual guarantees from model vendors, trust hardware-enforced isolation. But for most enterprises, whether legal contracts or technical TEEs matter more depends on the industry — healthcare, finance, and government sectors lean toward the latter, while internet companies often prioritize the former.

Practical Recommendations for Developers

If you are building enterprise AI applications — especially for healthcare, finance, or government customers — this update is worth acting on in at least three ways:

• Pull down the SDK and run through the Codelab to understand the attestation verification flow. This is likely to become a required capability in enterprise procurement over the next one or two years;
• Evaluate your model deployment path. If you use self-developed or open-source models (such as Llama, Qwen, or DeepSeek), Confidential Space plus G4 VM can host them directly. If you rely on external APIs, you need to verify whether the model provider supports deployment inside a TEE;
• Reevaluate your data flow architecture. TEEs only protect the “data-in-use” portion. Client-side input collection, logging retention, and caching strategies still need corresponding updates, otherwise security gaps remain wide open.

Incidentally, our OpenAI Hub aggregates APIs for mainstream models such as GPT, Claude, Gemini, and DeepSeek, with OpenAI-compatible direct domestic access. Transport and storage encryption at the API layer are already standard, and confidential inference at the TEE level is currently under evaluation. If your business has strict compliance requirements around “data in use,” stay tuned for future updates.

A Bigger Perspective

Viewed on a longer timeline, Google’s latest moves represent a compliance turning point for AI infrastructure. Over the past two years, competition centered on model capability, context length, and multimodality. Over the next year, the key issue in the enterprise market will shift toward whether models can enter the trust boundary of enterprise data centers. Whoever first fully integrates TEEs, attestation, and key management into a developer-friendly engineering stack will capture the most valuable markets in finance, healthcare, and government.

Google has taken an early lead this time, though Microsoft and AWS are also developing confidential GPU solutions and likely will not lag far behind. The real question is how the open-source community responds: when mainstream inference frameworks like vLLM and SGLang begin supporting TEE mode natively will determine whether this approach becomes a standard enterprise capability or remains confined to private deployments for a small number of very large customers.

References

• ITHome: Google Upgrades Google Cloud Confidential Computing Products with New NVIDIA RTX PRO 6000 Blackwell Virtual Machines — The first Chinese-language report covering this update, including G4 VM, Prompt Encryption SDK, and Confidential Space upgrades
• google/prompt-encryption-sdk - GitHub — Official SDK repository containing the Python client source code, Codelab scripts, and attestation verification examples