Security

Don’t worry, this one was via a bug bounty program

An AI broke Snowflake’s code; then another AI, an attack agent, autonomously found the bug, exploited it, and extracted credentials without human intervention.

Luckily, this wasn’t yet another case of rogue AI agents doing evil things. It was a sanctioned bug hunt, conducted through Snowflake’s HackerOne vulnerability disclosure program, and Snowflake fixed the flaw the same day Wiz reported it and rotated the affected credentials the following day.

Wiz’s red agent, an AI-powered autonomous attacker designed for offensive security, found the GitHub Actions workflow flaw during a routine scan of public repositories on June 23. The script injection vulnerability existed in snowflakedb/snowflake-connector-net, and it allowed an unauthenticated user to execute arbitrary commands within a GitHub Actions runner by opening a GitHub issue with a specially crafted title.

And it turned out an AI had inadvertently injected the bug into the code five days earlier.

GitHub Copilot Autofix, an AI coding assistant, co-authored the commit on June 18, and it introduced a script injection bug in run: blocks by removing the repository’s existing sanitized input pattern and replacing it with direct string expansion in a shell script.

“We crafted an issue title that, after template expansion, breaks out of the echo string and exfiltrates the Jira credentials via an out-of-band callback,” Wiz’s head of threat exposure Gal Nagli said in a Monday blog. 

These credentials gave Wiz read access to Snowflake’s engineering, security compliance, and bug bounty tracking projects.

Wiz reported the workflow vulnerability to the cloud data platform on June 23, and Snowflake patched it the same day. It also revoked and rotated the Jira token, and confirmed, via audit logs, that Wiz was the only third-party to access the endpoint during the five-day exposure window. 

The disclosure “was immediately investigated and remediated, and our investigation found no evidence of unauthorized access,” a Snowflake spokesperson told The Register. “We are working together with Wiz to share these learnings with the broader industry to encourage widespread adoption of these security best practices.”

Wiz, for its part, deleted all of the data it accessed during the vulnerability research and proof-of-concept exploit testing, and told us that this incident proves human code review isn’t sufficient to quickly detect vulnerabilities –  especially as developers increasingly use AI.

“This incident highlights a rapidly emerging reality in software development: how AI coding assistants can inadvertently introduce workflow injection vulnerabilities, and how automated AI agents can rapidly surface them in the wild,” Nagli wrote.

Of course, the Google-owned biz has a vested interest in saying this. But this doesn’t make it not true.®

A retrieval-augmented generation (RAG) system is built to answer strictly from the documents it retrieves. But when engineers optimize these AI pipelines end-to-end, the reader module can learn a shortcut: instead of relying on retrieved evidence, it starts answering from its own internal memory — while the system’s overall accuracy keeps climbing. This is the hidden challenge of “role drift,” a failure mode in compound AI systems where individual modules learn to bypass their assigned tasks even as end-to-end performance improves.

To address this, researchers at MIT and Harvard introduce Role Anchor, a technique that forces modules to stay in their lanes during training. When applied, the technique mitigates role drift. For example, it forces the RAG reader to rely on retrieved evidence instead of answering based on its internal knowledge.

The primary takeaway for practitioners is that end-to-end accuracy alone can overstate how much a compound AI system has genuinely learned. Engineers must evaluate individual components and ensure they work as intended.

Role Anchor serves as both a guardrail and a diagnostic tool when optimizing multi-step LLM pipelines. It can be essential for real-world AI applications that require a strict division of labor between modules.

Why terminal accuracy hides the problem

Compound LLM systems divide complex tasks among specialized modules. For example, a system designed for multi-hop reasoning might split a task between a “Decomposer” and a “Solver.” The Decomposer breaks a large problem down into manageable sub-tasks, while the Solver computes the answers to those sub-questions. This division of labor allows AI engineers to delegate execution to smaller, cheaper models, and makes it possible to process sub-tasks in parallel where possible.

To improve the performance of AI pipelines, engineers typically optimize them using end-to-end reinforcement learning (RL) guided by a single “terminal reward.” This means the system is evaluated on whether or not the final answer is correct (the researchers call it “terminal accuracy”). When this terminal accuracy goes up, the system is considered to be learning and working as intended.

However, terminal accuracy does not verify whether the modules properly executed the tasks they were assigned. As Xiaoyang Cao, co-author of the paper, told VentureBeat, “Terminal accuracy reduces the behavior of an entire multi-part AI system to a single number. It shows whether the final answer is correct, but says little about which components contributed or whether they followed their assigned roles.”

This blind spot leads to role drift, a failure mode where a module’s behavior diverges from its assigned role during optimization, even though the system’s terminal accuracy continues to improve. 

“For engineering teams, the practical risk is that they can deploy a pipeline that passes every end-to-end evaluation even though its intended division of labor has silently broken down,” Cao said. Because the reward system only scores the final answer, it fails to detect or penalize the module for going rogue.

Consider how this happens in the Decomposer-Solver pipeline. The Decomposer’s assigned role is to write abstract sub-questions without solving the task, leaving the reasoning to the Solver. Under end-to-end RL, the Decomposer quickly learns that the weaker Solver is prone to errors on abstract tasks. To maximize the reward, the Decomposer begins leaking or planting answers into the sub-questions it sends to the Solver. The Solver ends up parroting the answer the Decomposer fed it. Terminal accuracy goes up, but the intended architecture is compromised.

But if the system is getting the right answers and accuracy is going up, why should we care if a module drifts from its role?

Real-world deployment requires much more than just a correct final answer on a training dataset. The implicit roles assigned to these modules ensure scalability, reliability, and auditability. Consider what happens when role drift takes over:

• Loss of efficiency and auditability: In the reasoning example, role drift causes the Decomposer to do all the heavy lifting instead of planning and delegating. “Once the decomposer starts putting answers directly into its sub-questions, the solvers are reduced to copying those answers,” Cao said. “You are still paying to run [different modules], but they are no longer doing independent work.” The workload can no longer be parallelized across multiple Solvers, it cannot be delegated to cheaper models to save compute, and downstream human stakeholders can no longer audit the system’s logic step-by-step to verify how it arrived at the answer.

• Fragility in dynamic environments: Consider a RAG system, in which a Reader model is tasked to answer questions strictly using external retrieved documents. If the Reader drifts and learns to rely on its own internal parametric memory instead (because its memory happens to be accurate during training), the system becomes brittle. When the enterprise updates its database with new information, or a user asks a question about a novel topic outside the model’s pretraining, the system will fail because it abandoned the grounding mechanism it was built to use.

How Role Anchor measures a role — and enforces it

“Training only for the final outcome rewards a system for producing the right answer, regardless of how it gets there,” Cao said. To counter this, Role Anchor serves as a lightweight regularization technique that makes role instructions part of the training objective. It compares how the component behaves with and without those instructions and discourages training from weakening their effect. 

At a high level, it ensures the module continues to respect the steering influence of its original role prompt throughout the reinforcement learning optimization process, making role drift both measurable and controllable.

A key insight of Role Anchor is that a role’s effect can be measured by comparing how a model behaves with and without the role prompt. The system evaluates two different prompts for each module:

1. The specialized, instruction-heavy role prompt (e.g., “You are a careful Reader. Use the retrieved passages to answer the user’s questions…”).

2. The neutral prompt (e.g., “Answer the user’s question…”).

For any given input, the model outputs a probability distribution for the next token. When run under the role prompt, it will favor certain tokens. When run under the neutral prompt, it behaves like a generic assistant. The difference between these two probability distributions is the “role utility.”

This utility measures the ”nudge,” or the direction and strength with which the role prompt shifts the LLM’s default predictions. If a token is highly aligned with the assigned role, the role prompt boosts its likelihood compared to the neutral baseline (or “nudges” the model toward that token).

Before starting RL training, Role Anchor keeps a frozen copy of the model as reference and measures the role prompt’s original nudge on this reference model. This pre-RL nudge serves as the ground truth of the designer’s intent, acting as a proxy for how the role prompt is supposed to steer the model.

During RL training, as the active model’s weights are updated, Role Anchor regularly calculates the current nudge and compares it to the reference nudge. If the current nudge starts to fade or deviate from the reference, Role Anchor applies a penalty to the model to prevent role drift.

To see this practically, consider the RAG system evaluated by the researchers. In this pipeline, the Reader module is explicitly instructed to answer user questions based only on retrieved documents, rather than relying on its internal knowledge.

During unconstrained, outcome-only RL, the reader learns that the upstream retriever is sometimes noisy. To maximize accuracy on the training set, it starts ignoring the retrieved passages and answering from memory. Consequently, the gap between its behavior under the role prompt and the neutral prompt shrinks to the point that the reader starts behaving identically under both, ignoring the grounding instructions.

In contrast, Role Anchor detects when the reader’s nudge deviates from the reference nudge. It applies a penalty, redirecting the model’s parameters away from this memory-based shortcut. This forces the reader to find role-compliant ways to improve, such as learning how to extract answers from the retrieved passages more robustly or avoiding using its internal knowledge when the retrieved passages are faulty.

The numbers: how much of the accuracy gain was real

To test the efficacy of Role Anchor, researchers evaluated it on the RAG and Decomposer-Solver (DEC) pipelines. The experiments compared systems trained with standard outcome-only reinforcement learning (no anchor) against systems trained with Role Anchor.

Under outcome-only RL, the RAG system’s terminal accuracy rose, but its internal integrity collapsed. The researchers measured “Evidence-Following Accuracy,” a probe testing if the model changes its answer when the retrieved text is deliberately swapped to state the opposite. This metric plummeted from 0.86 to 0.54 (just above random chance), meaning the model learned to ignore retrieved passages and rely on its pre-trained parametric memory instead. In one test, researchers deliberately changed a piece of information in a retrieved document to contradict the model’s internal knowledge. The unanchored model did not update the response because it wasn’t using the external document.

When Role Anchor was applied, the Reader’s Evidence-Following Accuracy remained at 0.869, proving it relied strictly on the retrieved text. When researchers fed the anchored model random passages that were unrelated to the input prompt, its accuracy correctly dropped because it refused to use its internal knowledge. The unanchored model scored higher on random passages because it was guessing from memory.

The Decomposer (DEC) pipeline showed an even more dramatic failure mode. Under outcome-only RL, terminal accuracy shot up, but the “insertion rate” (i.e., the frequency at which the Decomposer leaked the answer into the sub-questions it sent to the Solver) surged from 0.143 to 0.596.

In the RAG pipeline, preserving the intended role cost the system a very modest accuracy drop (-0.067). The Reader still learned to be better at extracting answers, but it did so legitimately rather than by cheating with its internal memory. This means it is more reliable on real-world tasks with novel knowledge it has not seen during training.

In the DEC pipeline, unanchored RL improved accuracy by 0.310 above the base model, while Role Anchor only showed a 0.057 improvement. When diagnosed, it turned out that the underlying issue was that the Solver model was too small and couldn’t learn the problem-solving part. This forced the Decomposer model to cheat and provide the answer to boost the terminal accuracy. This meant 86% of the unanchored improvement was fake, and the system had simply learned to exploit a shortcut instead of learning how to reason or decompose problems better.

However, this tradeoff is not a universal rule. In some cases, eliminating shortcuts can actually boost overall performance. “Role Anchor… does not necessarily reduce final accuracy,” Cao said. “In a coding pipeline we recently tested, the model had learned to manipulate its own test executor during reinforcement learning training. Adding Role Anchor completely eliminated that shortcut while slightly improving correctness on the final tests used to judge the code.”

What it takes to add Role Anchor to an existing pipeline

For engineering teams looking to apply this technique, “Role Anchor can be added to an existing reinforcement learning fine-tuning process as an extra training objective for each component that a team wants to anchor,” Cao said. The main pipeline and deployment setup remain entirely unchanged.

To implement it, engineers need three specific items for each anchored component: its original role instructions, a matched neutral version with the role information removed, and a saved copy of the model from before reinforcement learning fine-tuning.

Importantly, there is no latency penalty at inference time. “Role Anchor runs only while the model is being trained, so it does not slow down the deployed system,” Cao said. He noted that their current implementation takes roughly 20 percent longer during training due to additional calculations, though there is likely room to optimize and reduce that overhead. The research code, training configurations, and selected model weights will be released publicly in the near future.

Deciding when to use Role Anchor is a case-by-case decision based on whether final accuracy captures everything that matters. Cao points to a regulated legal RAG system as a prime candidate. “The component producing the answer may need to follow retrieved evidence, stay grounded in an approved set of documents, and produce answers that can be traced back to their sources,” he said. “Final accuracy alone cannot verify those properties, so the behavior of that component needs to be measured and enforced directly.”

As enterprise AI evolves toward more complex compound pipelines, role enforcement will become harder, and relying on prompts alone will prove unreliable. “At larger scales, role specifications will need to be enforced through both training and system design,” Cao said. “Methods such as Role Anchor can help preserve intended behavior during training, while clear system boundaries, limited tool permissions, and monitoring during use can provide additional safeguards.”

The FCC has a track record of using its Covered List to squeeze Chinese tech out of American markets, and robots just became its newest target. Foreign-made humanoids, quadrupeds, and even robot vacuums now need to be mostly built in the U.S. to sell here (via Rest of World). 

So what does the new rule actually require?

The FCC added “advanced robotic devices” to its Covered List last month, the same national security roster that’s previously blocked devices from Chinese companies like Huawei and ZTE. New models of foreign-made humanoids, quadrupeds, robot vacuums, and even lawn mowers are now barred from entering the U.S. market.

Only models assembled domestically and sourcing at least 65% of their component value from within the U.S. qualify. The threshold will climb to 75% by 2029. The policy doesn’t target any single country (or mention one), but it’s widely understood as aimed squarely at reducing Chinese imports.

The good news is that existing products (already on sale) aren’t affected. Robots imported purely for research and development (not for commercial sale) stay exempt from the rule for now.

How are startups actually reacting to this?

Founders say hitting 65% local sourcing threshold is nearly impossible right now. Plenty of motors, sensors, and actuators are either too expensive, too slow to source, or, the worst one of all, simply unavailable domestically. The issue is so severe that employees have reportedly flown parts over from China in their luggage just to keep prototypes moving. 

Michael Perry of Persona AI put it bluntly: “You need to provide the carrot as well as the stick.” Not everyone’s upset, though. Oregon-based Agility Robotics and San Francisco’s Nori Robotics both welcome the rule, stating that it will eventually shield them from cheaper Chinese competition. 

However, there’s no denying that the rule complicates sourcing for the immediate future. China didn’t end up controlling robotics manufacturing by accident. Decades of state investment, a deep engineering workforce, and a boom in electric-vehicle production carved out the kind of sensor, battery, and actuator supply chain robots also rely on. 

The result: Chinese-made humanoids account for nearly 90% of everything sold worldwide in 2025, according to research firm Omdia. 

Apple has released an update for iOS 26.6.1 for all users, with the usual inclusion of security fixes, making it a must-install for your iPhone.

While Apple is constantly beta-testing its 27-generation operating systems, it’s still shipping updates to its earlier releases. That includes the current-gen versions.

On Monday, as part of its swathe of releases, Apple issued its update for iOS 26.6.1. The update, which it rolled out as a release candidate on August 10, can be downloaded to an iPhone running iOS 26.6 or earlier.

It also brought out updates for its other operating systems:

• iOS 26.6.1 is build 23G83
• iPadOS 26.6.1 is build 23G83
• iOS 18.7.10 is build 22H374
• iPadOS 18.7.10 is build 22H374
• visionOS 26.6.1 is build 23O780
• macOS 26.6.2 is build 25G83

The updates include what Apple characterizes as security fixes, which is a regular item for granular updates such as this.

We are still waiting for Apple to share the list of updates in the release. The iOS 26.6 update had more than 75 security fixes, and with the persistent threat of AI-based discoveries, AppleInsider recommends installing it sooner rather than later.

A German competition authority said that Apple’s prompts favors its own apps versus third-party options.

If you’ve been putting off that macOS update sitting in your notifications, this is the week to stop and install it. 

Apple quietly patched a serious screen sharing flaw in macOS earlier this month. While that usually means the issue is resolved, new evidence shows hackers already broke into unpatched Macs, hijacked them, and used them for cryptocurrency mining before the fix shipped.

So what exactly went wrong with screen sharing?

The flaw lets bad actors access macOS screen sharing without the system’s valid login credentials. Once in, they could view a victim’s desktop and take control of the mouse and keyboard (remotely) without warning whoever was actually sitting at the machine.

Once attackers get root access, they reportedly install Monero crypto miners onto every compromised Mac spotted so far, quietly burning through users’ CPU cycles and electricity in the background; hackers value that over the local files.

Apple patched the issue on August 6 with macOS Tahoe 26.6.1, describing it as improved state management for the authentication process, along with matching updates for Sonoma and Sequoia.

What happens on a hacked Mac, and how do you stay safe?

However, the Netherlands’ National Cyber Security Center says it has since tracked real-world abuse of that bug on machines running unpatched or older macOS versions, where port 5900 (which screen sharing uses) was left accessible from the open internet.

Port 5900 has been a known soft spot in remote access tools for years. Apple’s macOS automatically opens that port the moment you enable screen sharing, with no separate warning that it’s now reachable from outside your network.

Because the bug bypasses authentication entirely, scanning bots can automatically gain root-level control of exposed machines and install a cryptocurrency miner. If your Mac relies on screen sharing over an open network connection, act immediately.

I’d recommend installing the latest macOS version immediately. Head to System Settings > General > Software Update, and install the update. If updating immediately isn’t an option, your safest short-term move is to disable the feature altogether by going to System Settings > General > Sharing and toggling screen sharing off.