Multiverse Computing introduced ProvenanceGuard, a post-generation verification system designed to catch claims attributed to the wrong tool in Model Context Protocol agents, according to reporting published by Hugging Face.
Existing factuality systems like RAGAS, MiniCheck, AlignScore, and SummaC pool retrieved evidence together before assessing support. That pooling creates a failure mode the researchers call cross-source conflation: an agent states a claim that is factually true somewhere in the gathered evidence, but credits the wrong source. ProvenanceGuard targets that gap by keeping tool outputs and source identifiers attached to claims throughout evaluation.
The verification pipeline
ProvenanceGuard operates as an external layer over black-box agents without requiring retraining. The system inspects captured traces, splits the generated response into discrete claims, routes each claim to its most relevant source, and uses natural language inference to verify support. It then verifies whether the supporting tool matches the citation stated or implied in the answer before issuing a per-claim verdict and an overall allow or block decision.
The researchers evaluated a local offline configuration using MiniLM for source routing, a DeBERTa model for inference checks, and a local language model for claim extraction. The verifier checks literal values, including dates, numbers, and identifiers, blocking sentences that lack exact backing in the text. Blocked answers can pass through a RARR-style repair module for grounded rewrites or fallback text.
Test results and medical trials
Researchers tested the system on 281 real traces from a medical agent that queried patient records and research literature. Human experts evaluated 361 claims drawn from 40 held-out test answers, marking 139 claims as unsupported. ProvenanceGuard caught 138 of those 139 claims, letting one through. It also flagged 67 claims that human reviewers considered supported, sending them for further review.
For claims with identifiable sources, ProvenanceGuard selected the correct source about 86 percent of the time. In a separate test featuring multiple similar sources, the system recorded an F1 score of 0.846 for blocking unsupported claims, but identified the exact source correctly in 50.3 percent of cases. When researchers intentionally swapped named sources across 50 supported statements, the system flagged all 50 attribution errors.
Testing on the repair pipeline showed that an initial full-trace run resolved all 173 blocked answers, with 144 settling on fallback text. On reconstructed multi-source test traces, a second repair run resolved 59 blocked answers with two terminal fallbacks. The local setup ran with an overhead of roughly half a second per answer, with inference and routing queries completing in tens of milliseconds.
Nvidia has merged ProvenanceGuard's source-aware verification approach as an optional grounding step in NVFlow for its finance agent, checking generated outputs against retrieved SEC filings. The researchers also presented ProvenanceGuard as a poster at the Agentic AI Summit 2026 at UC Berkeley.
