Google DeepMind launched a pilot to conduct double-blind evaluations of its proprietary artificial intelligence models, the company announced on August 27, 2026.
The initiative evaluates a Gemini Flash Lite model against confidential benchmarks inside a privacy-preserving cryptographic environment. DeepMind partnered with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons to carry out the tests, targeting the issue of benchmark contamination where models encounter evaluation questions prior to testing.
Cryptographic safeguards
External evaluations historically required a compromise between testing integrity and intellectual property protection. Evaluators had to provide their test prompts directly to the AI developer, which risked exposing the questions in advance. Alternatively, the model developer had to transfer model weights to the evaluator, creating intellectual property risks.
Google DeepMind said the new process relies on Confidential Space within Google Cloud’s Confidential Computing portfolio. Under this architecture, Google cannot view the evaluator’s confidential testing prompts, and external evaluators cannot inspect the Gemini model weights. While companies have previously relied on zero-logging protocols and contractual agreements to protect testing data, the pilot introduces technical cryptographic verification.
DeepMind researchers William Isaac, Sol Messing, and Kristian Lum noted that the setup allows independent groups, civil society organizations, and national AI Safety and Security Institutes to test frontier systems. The company pointed to sensitive domains such as cybersecurity and government evaluations as key use cases, and published a technical report detailing the pilot's methodology and findings.
