Egiziago Cioffi is the IT and Enterprise Architect and CEO of SynSphere Italia, a Microsoft partner based in Milan. He built an agent himself. He wrote the indexing job, configured the Azure OpenAI retrieval pipeline, connected it to SharePoint, and watched it pass every evaluation his team ran.
His Azure OpenAI email assistant auto-resolves about 60% of inbound customer email, Cioffi told VentureBeat in written responses to our interview questions. The evaluation scores were clean, and the unit tests passed. None of them asked the question that mattered.
Cioffi ran a low-privilege account against the same questions a high-privilege account had already put to the assistant. The outputs did not match. The assistant returned SharePoint content the requesting user could not have opened in SharePoint on their own. The logs told a different story than the evaluation scores.
Cioffi’s retrieval logs are the evidence for this specific production failure. What follows is independent data showing the failure class is not isolated.
In many production RAG deployments, the agent answers with the indexer’s permissions, not the requester’s
Azure AI Search has shipped native document-level ACL trimming via Entra-based tokens since preview in May 2025, and SharePoint ACL sync followed in a later preview. The capability exists; however, it does not exist everywhere it needs to.
The SharePoint ACL preview can now ingest site-group metadata via the spg: prefix in the 2026-05-01-preview API. However, only Entra-backed principals are documented as reliably enforced at query time. The preview runs through the REST API and preview SDKs and does not cover all agent deployment paths. Azure OpenAI On Your Data, for example, supports document-level access via Azure AI Search security filters, but Microsoft’s own documentation states that if the permitted-groups field is not mapped, document-level access is disabled.
That is a fail-open default in a first-party path. Custom RAG pipelines that bypass Azure AI Search entirely still index under a broadly privileged service account with no query-time entitlement check unless the developer builds one. Cioffi’s deployment took the custom-pipeline path.
Across production agents at scale, 91% of successful attacks ended in silent data exfiltration
Straiker’s red team ran more than 1,700 successful exploit attempts against production agents and published the results in its inaugural STAR Labs Threat Report in July. The 91% figure from their research measures all successful attacks on productivity agents that ended in data exfiltration without detection. It is a measure of what happened after an exploit succeeded, not a measure of how many deployments fail to enforce retrieval-time entitlements specifically.
Across the productivity agents in scope, 91% of successful attacks ended in silent data exfiltration, with the report noting no malware had been required. There was also no lateral movement through the network. The agent returned all the data it could reach. Straiker’s report does not break out which of those successes trace to entitlement failures specifically versus prompt injection, tool abuse, or other attack classes.
Working independently, the U.K.’s AI Security Institute documented 19 unsanctioned agent actions from a July 25 to 28 cyber evaluation. The UKASI published its incident report on August 4 of this year. The evaluation deliberately ran with cyber classifiers disabled and internet access enabled. What the UKASI report demonstrates is agents acting outside the scope their deployers intended, in a permissive test environment, with no reliable mechanism to catch the deviation before it causes damage. It is a containment failure, not a retrieval-entitlement failure, and the overlap with the Cioffi incident is the shared absence of a runtime scope check rather than an identical mechanism.
Why evaluations miss this and why the native fix did not reach Cioffi’s deployment
The evaluations Cioffi’s team ran were designed to test whether the agent answers correctly. They check factual accuracy, relevance, and task completion. They do not ask whose permissions the retrieval pipeline uses when it fetches the source material, because that question is not in the evaluation framework.
Azure AI Search is currently shipping the retrieval-time entitlement check at the platform level. The query-time ACL trimming validates the caller’s Entra token, extracts user and group claims, and returns only documents whose synchronized permission metadata grants the caller access. For deployments that use Azure AI Search with the SharePoint indexer and Entra-backed principals, the control exists natively. Cioffi’s deployment did not use this path. His custom Azure OpenAI retrieval pipeline bypassed the native trimming layer, which is how the gap survived every evaluation his team ran.
From the attacker’s side, this is a broken access control. Adriel Desautels, founder and CEO of Netragard, told VentureBeat in written responses that the failure reduces to a structural collapse of authorization boundaries. “If the NHI credentials usually have broad authorization and can read high privilege data then that is then stored in their index,” Desautels wrote. “If an app does not enforce identity-aware retrieval, then a ‘normal’ user with lower permissions can query the app and access otherwise restricted data. This collapses authorization boundaries down to the lowest privilege level with search capability.”
That gap is what Cioffi’s low-privilege test exposed. The assistant’s context window contained SharePoint content the low-privilege account could not have retrieved through SharePoint directly. The evaluation had passed. The retrieval permission boundary had not been enforced.
Desautels put the evaluation blind spot in operational terms. “Agents tend to run a single, long-lived, non-human identity that holds a wide range of permissions that it might need for any task it is ever asked to complete,” he wrote. “Evaluations also don’t often cover prompts, outputs, transcripts, memory, and logs where it can be read or hijacked through injected content. That mismatch is what most current evaluations get wrong.”
Cioffi’s filter narrowed the assistant’s retrieval scope. It still resolves roughly 60% of email
Cioffi’s fix did not require a new identity platform. He moved the entitlement decision into the retrieval path itself, adding a query-path filter that checks the requesting user’s SharePoint permissions before the model sees a chunk. The filter runs at query time, not at index time. Content the user could not open in SharePoint does not enter the model’s context window.
The control narrowed what the assistant could reach. The assistant still auto-resolves roughly 60% of inbound email with the filter live, Cioffi told VentureBeat. He did not provide a before-the-filter auto-resolution figure for comparison. The qualitative tradeoff he described is that some content the assistant previously used to answer questions is now excluded because the requesting user’s permissions do not reach it. That is the price of enforcing the boundary.
The question of whether retrieval-time entitlement filtering is worth the narrowed retrieval scope does not have a single answer. It depends on the sensitivity of the indexed content, the permission variance across the user population, and whether the deployment can tolerate unanswered queries when the filter blocks a chunk the model needs. What Cioffi’s incident demonstrates is that the gap exists in custom Azure OpenAI pipelines, that answer-quality evaluations do not catch it, and that a query-path filter closes it at a trade-off the builder can describe.
Identity governance platforms address a different layer. Both controls are needed
CrowdStrike announced its $740 million acquisition of SGNL on January 8, 2026, and closed the deal on February 20, 2026. Palo Alto Networks announced its $25 billion acquisition of CyberArk in July 2025 and closed the deal on February 11, 2026. Both deals closed the same month, establishing identity security as a platform pillar at two of the largest security vendors in the world.
Identity governance platforms focus on which service accounts exist, what they can reach, and when their tokens expire. They govern the lifecycle of the credentials that power AI agents. That layer matters. What it does not govern is the retrieval permission boundary. That is the moment a correctly scoped service account retrieves content on behalf of a user who holds fewer permissions than the indexing job does.
Every credential in the chain is legitimate. The service account is clean and properly managed. The knowledge base is correctly indexed. A low-privilege user queries the assistant, and it answers from the full indexed scope. Nothing flags the retrieval because no credential was misused.
Cioffi’s filter is a control at the retrieval permission boundary layer specifically. Azure AI Search’s native ACL trimming addresses the same layer for deployments that use it. Neither replaces identity governance. A production deployment that wants to close both the credential lifecycle gap and the retrieval-time entitlement gap needs controls at both layers.
One question and one test, any security team can run
Ask whose permissions each AI retrieval system uses when it fetches content.
If the deployment uses Azure AI Search with the SharePoint indexer and Entra-backed principals, verify that query-time ACL trimming is enabled and that the user population does not depend on SharePoint site groups. If the deployment uses a custom retrieval pipeline, the entitlement check may not exist at all.
Start by proving the answer from a low-privilege account. Run the same question a high-privilege account has already put to the assistant. Compare the outputs against what the low-privilege account can access through the underlying system directly.
Desautels confirmed that this is where a red team would start. “The first test would likely target the gaps between data and instructions, and the gaps between the user’s identity and the assistant’s own credentials,” he wrote. “We’d attempt to plant an instruction within content that we think the assistant will ingest as data. We’d have that content direct a side-effectful, privileged action that the attacking user is not authorized to perform.” A failing result, in Desautels’ assessment, is “the successful or even partial execution of our injected commands.”
If the assistant returns more than the account’s direct access would allow, the retrieval permission boundary is not enforced at query time. That test costs two accounts and thirty minutes. It produces a result an evaluation score cannot replicate.
Cioffi built the agent on a custom Azure OpenAI pipeline that bypassed the native ACL trimming layer. He ran every evaluation his team had. He found the gap in his own logs after all of them passed. The evaluation tested whether the agent answered correctly. It did not test whose permissions the agent was using. Run the two-account comparison before the next deployment goes live. Thirty minutes tells you which side of the line you are on.
You must be logged in to post a comment Login