DeepSeek has piqued the interest of the developer community by releasing an early version of its open source agent harness. This happens as harnesses have become increasingly important to those working with machine learning models.

“Powered by the Cordis meta-framework, DeepSeek Harness is an agent harness built around one core idea: Everything is a plugin,” the China-based AI biz said. “Models, tools, skills, sessions, sandboxes, filesystems, loops, orchestration, and UI are ALL implemented as plugins, and can be mixed, matched, replaced, and extended.”

The term “harness” came into common use this year to describe a longstanding software function – middleware or a mediation layer that handles the input passed to an AI model and the output returned from it. Harnesses oversee prompts, context management, tool orchestration, the agent loop, state management, error handling, safety, permissions, and related concerns.

Claude Code serves as a harness for Anthropic’s Claude model family and Codex performs a similar function for OpenAI’s GPT model family. And there are many other model harnesses, including Aider, Cline, Goose, OpenCode, OpenHands, and Pi, to name a few. 

The term isn’t precise: It may be used to refer just to the agent loop and tools, or it may be extended to a broader set of concerns related to orchestrating different tools, services, and capabilities like sandboxing, subagents, and so on. Google Antigravity, for example, consists of the Antigravity Agent Runtime (harness) that can be accessed through the Agent SDK, the Antigravity 2.0 desktop application, and the Antigravity CLI.

Vague definitions aside, AI model harnesses are now where much of the competition is happening, particularly as models proliferate and become commoditized. The harness often implements the user interface, a source of user inertia, and once developers configure their tooling and become accustomed to doing things a certain way, it becomes more burdensome to switch to a competing product, even if the interface consists mainly of a command line.

What’s more, various studies have suggested that model performance (and cost) varies significantly with the harness used, due to different design choices. For example, the Pi coding agent relies on a minimal system prompt of about 200 tokens. Claude Code by comparison uses a system prompt of around 10,000 tokens (or did until last month when Anthropic trimmed the system prompt by about 80 percent). The same model will produce different results with different harnesses.

DeepSeek Harness is noteworthy because of its innovative design, and because it shows Chinese AI labs moving to compete beyond model benchmarks and pricing.

First, it treats everything as a plugin. It uses the plugin system from its underlying Cordis framework, which is designed to make it possible to add and remove components dynamically without wreaking havoc.

“Plugins provide every agent capability, including models, tools, skills, sessions, sandboxes, storage, loops, scheduling, and the UI,” the DeepSeek Harness website explains. “Cordis services and events let the plugins work together. Developers can select, swap, or extend any capability in configuration without changing the DeepSeek Harness source code.”

A DeepSeek paper [PDF] by researchers Yifan Shi, Wei Zhang, and Tianyi Cui explains the function of Cordis in more detail. Cordis is designed to support dynamic composability – adding plugins and removing them on the fly without breaking the application.

The paper refers to this as temporal composability – removing a component and reverting its effect upon removal – and spatial composability – allowing components to manage dependencies upon other components.

It cites as an example the plugin system used by Microsoft’s Visual Studio Code. VS Code, the authors explain, runs all of its extensions in a shared process called the extension host. Once activated, they cannot be removed on the fly; the host has to be restarted. 

While VS Code provides a way for extensions to declare dependencies between extensions, it’s seldom used. DeepSeek Harness supports plugin dependencies.

The DeepSeek researchers argue temporal and spatial composability are necessary in a system where modification can occur continuously with little or no human oversight. It’s a way of avoiding forced restarts and crashes when components appear and disappear.

DeepSeek Harness also supports another useful feature: chain of thought traces.

“Everything the model sees is recorded in an append-only session log: system prompts, reasoning, tool calls and results, subagent scheduling, and every context injection,” the DeepSeek Harness website says. “In the Trajectory view, you can inspect these records by source. Resume, fork, search, and replay all operate on the same event stream.”

DeepSeek R1 made waves when it was released last year and it was trained to use chain of thought reasoning. This involves breaking down prompts into a series of “thoughts” and reflecting on those steps before emitting a final answer.

Access to this intermediate reasoning turns out to be useful for assessing whether a model is reasoning well, whether its responses are accurate, how additional “thinking” affects output, and so on. 

Anthropic provides some access to thinking when extended or adaptive thinking is available (it varies by model). But increasingly the biz has been hiding model reasoning by summarizing chain of thought traces. That appears to be due in part to concerns that chain of thought traces can be used for copying models through a standard research process called distillation. 

Earlier this year, Anthropic said it had implemented classifiers for the “detection of chain-of-thought elicitation used to construct reasoning training data.” The company also does not display raw chain of thought. It explains that “the text in a thinking block is a summary of Claude’s reasoning.” Accessing raw thinking requires contacting Anthropic sales personnel.

Except for its open source models, OpenAI has also chosen to hide chain of thought reasoning, which the company uses for model monitoring. “After weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users,” the biz said two years ago when it introduced its o1 reasoning model.

With the newly released DeepSeek-V4-Pro and V4-Flash, the API provides thinking mode enabled by default. And as the open source model ecosystem matures, having access to chain of thought looks likely to become another opportunity for competitive differentiation.

“I don’t think the DeepSeek Harness is perfect but this is for sure the first time I have been looking at something new in the space and felt quite inspired to revisit some of our choices,” said Armin Ronacher, co-founder of AI biz Earendil, which now steers the development of the Pi agent, in a social media post. “I love that part about Open Source a lot!” ®

Anthropic wants to make AI-generated text easier to identify, and on paper, I have very little reason to complain. The company is experimenting with an invisible watermark that can be baked directly into text generated by Claude.

It sounds like a sensible idea. AI-generated text is everywhere, and knowing where something came from could certainly help. Moreover, Anthropic isn’t simply hiding a marker somewhere inside a document. Its approach changes how Claude selects words to create a statistical pattern that can later be detected.

But there is one detail that bothers me. Anthropic is testing just how persistent that watermark can be, even after the text has been modified.

That is where I can already smell trouble.

Claude touched my writing. Did it actually write it?

Think about translation for a moment. Let’s say someone writes an entire essay themselves in Spanish and asks Claude to translate it into English. The ideas are theirs. The research is theirs. The arguments are theirs. Claude’s only job is translation.

Yet the resulting text could still carry Claude’s watermark.

The same question applies to proofreading. What if someone writes something themselves and asks Claude to fix the grammar? What about shortening a paragraph, changing its tone, cleaning up dictated text, or simply making an awkward sentence easier to read?

These aren’t fringe uses for AI anymore. People increasingly turn to assistants like ChatGPT, Gemini, and Claude for everyday tasks that have little to do with generating original work. A watermark can tell you that Claude was involved with a piece of text. It cannot tell you whether Claude actually wrote it. Anthropic makes the same point, saying the watermark shows Claude’s involvement, not who created the original work.

Now imagine explaining that distinction to a professor after their detection software has just flagged your essay.

We already know how messy AI detection can get

I wouldn’t worry nearly as much if our track record with AI detection were particularly good. It isn’t.

MIT Sloan’s guidance is quite straightforward about existing AI detectors. It says they have high error rates and can lead instructors to falsely accuse students of misconduct.

We’ve already seen what that looks like in practice. Students have found themselves defending work they say they wrote themselves after automated systems identified it as AI-generated. In one case documented by The Guardian, a student’s essay was flagged as entirely AI-generated despite the student saying they had only used approved spelling and grammar assistance. The appeal was eventually accepted.

To be clear, Claude’s watermark is fundamentally different. Conventional AI detectors look at writing and essentially estimate whether an AI might have produced it. Anthropic is deliberately planting a detectable signal in Claude’s output. In theory, that should make its system considerably more reliable. But reliability isn’t the only problem here. Interpretation is.

We’re using AI to prove we didn’t use AI

Things have already reached a slightly ridiculous point.

Students worried about AI detection are turning to so-called AI humanizers, which rewrite text specifically to make it less likely to trigger detectors. Some students are even using these tools on work they wrote themselves because they’re worried about false positives. Detector companies, naturally, are developing ways to identify humanizers.

Read that again.

A human can write something, worry that an AI will think an AI wrote it, feed it through another AI to make it look more human, and then have yet another system determine whether the AI made it look human.

It’s a technological ouroboros.

Making Claude’s watermark resilient enough to survive editing and translation is technically impressive. Previous research has shown that translation can defeat some text-watermarking techniques, so solving that weakness would represent meaningful progress.

I just don’t think making the signal harder to remove solves the more important problem.

A watermark needs context

There are good reasons to watermark AI-generated content. It could help identify mass-produced misinformation, undisclosed synthetic text, or AI-written material that later ends up in training datasets.

The problem is that AI assistants now do far more than generate content from scratch. People use them to translate text, proofread documents, summarize research, help with code, improve accessibility, or simply clean up an email before sending it. In that context, detecting AI involvement does not automatically tell you who actually created the work.

All of those interactions involve AI to wildly different degrees. If Claude writes an essay from scratch, knowing that is useful. If Claude translates an essay someone spent three weeks researching and writing themselves, knowing Claude was involved tells you considerably less.

The watermark may be perfectly capable of answering “Did Claude touch this?” My concern is what happens when people start treating the answer as proof of “Did Claude write this?”

Anthropic can build the smartest watermark in the world. Unless the people using it understand that difference, I suspect we’re going to have some problems.

Flipping the two-inch screen open is all it takes to start recording with the DJI Osmo Pocket 3, priced at $399 (was $499). That simple motion turns the camera on and gets you rolling in about two seconds, whether you hold it upright for vertical clips or keep it horizontal for wider scenes. The screen itself rotates freely and stays bright enough to read outdoors at 700 nits, so framing a quick walk-and-talk or a family moment never feels like a chore.

The image quality is much improved because of this device’s one-inch sensor. Coupled with a 20mm equivalent f/2.0 lens that can zoom in from only 20 centimeters out to infinity, the camera does an excellent job of capturing clean details even as the light fades, while 4k video can run at 60 frames per second in standard mode and 120 frames per second in slo-mo without losing resolution. You have ten-bit color options, including D-Log M, which gives you even more editing freedom, and the basic profile generates results that appear highly polished straight out of the box. Even in low-light conditions, this item beats most pocket cameras, as noise is maintained under control even when photography in a restaurant or outside in the evening.

Sale

DJI Osmo Pocket 3 Vlogging Camera with 1” CMOS & 4K/120fps Video

• Capture Stunning Footage – Osmo Pocket 3 vlogging camera features a 1-inch CMOS sensor and records in 4K resolution at an impressive 120fps. Capture…
• Effortlessly Frame Your Shots – Get the ideal composition with Osmo Pocket 3’s expansive 2-inch touch screen that rotates for both horizontal and…
• Ultra-Steady Footage – Say goodbye to shaky videos! Osmo Pocket 3’s advanced 3-axis mechanical stabilization delivers superb stability. Enjoy smooth…

The 3-axis mechanical stabilization ensures that everything looks as smooth as possible. Walking at a reasonable pace, up a flight of stairs, or following someone around a room results in silky smooth film with no artificial warping. Activetrack 6 locks on to a face, person, or object and keeps it in the center even when the gimbal pans and tilts, and full-pixel phase detection focusing snaps in and stays locked even when your subject moves closer or further away.

The sound quality is also respectable, even for everyday conversations, thanks to the camera’s three built-in mics, which capture crisp stereo sound and effortlessly combine with up to two DJI mic mini or mic 2 transmitters. When you clip a wireless mic to your shirt, the signal comes thru loud and clear; no additional adaptor is necessary, allowing solo vlogging or easy interviews a one-handed process.

Battery life is adequate for a casual afternoon of shooting. The internal battery provides up to 166 minutes of run time in ideal settings at 1080p, and in the real world, you should anticipate to get an hour or so depending on the resolution and tracking you’re doing, and don’t worry if you run out of juice; the battery recharges quickly. It will charge you back up to 80% in roughly 16 minutes with a competent USB-C charger. Plus, if you need more space, you can simply replace out the card for a larger one, as microSD supports up to 1TB, so your only true limit is the card in your pocket.

At 179 grams and 14 cm tall, this device is small enough to fit into a jacket pocket or bag without adding any bulk, and if you need more grip or a place to put it, simply screw on the accompanying handle with a quarter-inch thread.

The big picture: As various governments grapple with how to rein in the effects of social media on children, many, including several European countries, are considering an outright youth ban. However, France has just demonstrated that balancing such legislation with constitutional freedoms remains tricky.

France’s highest court struck down a law this week that would have banned children under 15 from accessing social media networks such as Facebook and X. The legislation had previously sailed through the lower chambers of government with unusually broad bipartisan support.

In addition to blocking social media for children and young teenagers, the bill would have banned smartphone use in schools. Resources such as Wikipedia, GitHub, and scientific directories would have been exempt.

The National Assembly had approved the bill in January, voting 130 to 21, before the Senate passed it in July and the Assembly voted in favor again the same day by a margin of 279 to 81. President Emmanuel Macron, championing the legislation as a defense against American tech companies and Chinese algorithms, sought to fast-track the bill into force before the beginning of the school year in September.

However, France’s Constitutional Council ruled that the law infringed on freedom of expression and communication. The president now hopes to revise and reintroduce the ban before spring. If it passes, it will likely be one of his final major accomplishments before his term ends next year.

The law would make France the first European Union country to ban children from social media. The Norwegian and British parliaments plan to introduce similar legislation sometime this year, and Greece aims to enforce a ban in 2027. Spain is also weighing the issue.

Australia attempted to set an early example late last year, but recent data suggests that keeping kids off social media is easier said than done. Studies suggest that as many as 85% of under-16s who had accounts before the ban can still access them. While some underage users circumvented age gates with VPNs (which France might also target), most never encountered any restrictions.

While regulators accuse social media companies of dragging their heels on the issue, verifying users’ ages involves steep technical, security, and privacy-related challenges. Prior studies have shown that facial scans struggle to estimate teenagers’ ages, and uploading government IDs invites the risk of data breaches.

The Seattle Seahawks are about to give some football fans something they still can’t count on from most live sports broadcasts: Dolby Vision HDR delivered directly through the team’s own streaming platforms.

Beginning with Seattle’s 2026 preseason coverage, games streamed through the Seahawks Mobile App and Seahawks.com will use Dolby OptiView with Dolby Vision on supported devices. Team press conferences and other live digital coverage will also use the platform throughout the season. Local preseason viewers can additionally receive Dolby Vision through KING 5 using ATSC 3.0 on compatible equipment.

That makes Seattle the first individual NFL team to integrate Dolby OptiView with Dolby Vision into its own digital experience. And it raises an obvious question.

If the Seahawks can do this for preseason football, why doesn’t every NFL game look this good?

Related Reading: WTF Is Dolby Vision?

What Dolby OptiView Actually Does

Dolby OptiView is not simply another HDR badge. The platform combines live video delivery and playback technology with configurable latency, synchronized streams and support across web browsers, mobile devices, smart TVs and other streaming platforms. The Seahawks say the implementation will provide richer visuals, lower latency and a more immersive streaming experience.

Dolby Vision adds dynamic HDR processing designed to provide brighter highlights, deeper contrast and richer color on compatible displays. There is one specification worth making clear: Dolby Vision does not automatically mean 4K. Neither Dolby nor the Seahawks has specified that the Seahawks streams will be delivered at 4K resolution. Dolby Vision can be used with HD or 4K production workflows.

That distinction matters when consumers have been trained to lump 4K, HDR and Dolby Vision together as though they were the same thing. They aren’t.

Live sports remain one of the areas where the difference between an expensive modern television and the signal being fed into it can be painfully obvious.

You can own a premium OLED or Mini LED TV capable of spectacular HDR performance, yet the quality of the game still depends on how it was captured, encoded, distributed and streamed. Seattle’s approach attacks more than one part of that chain.

Dolby Vision can improve HDR presentation, while OptiView is designed to reduce latency and keep streams synchronized. Anyone who has heard a neighbor celebrate a touchdown several seconds before it appears on their television understands why latency matters almost as much as picture quality during live sports.

The biggest beneficiaries are therefore Seahawks fans watching supported preseason streams on compatible Dolby Vision devices, especially those who have already invested in televisions capable of displaying HDR properly.

They don’t need another TV. They need somebody to send their existing TV a better signal.

So Why Isn’t the Entire NFL Doing This?

The technology itself is only part of the problem.

The NFL’s regular-season television rights are divided among multiple major media partners, including CBS, FOX, NBC, ESPN/ABC and Amazon under agreements that run through the 2033 season. Those broadcasters and streaming services control the production and distribution of huge portions of the regular-season schedule.

That makes a team-controlled preseason game very different from Sunday Night Football, Monday Night Football, a CBS or FOX Sunday afternoon game, or Thursday Night Football on Prime Video.

Seattle has much greater control over the digital production and distribution chain for its own preseason and team-produced content. It cannot simply decide that every regular-season Seahawks game will suddenly be streamed in Dolby Vision through Seahawks.com.

There are rights agreements, broadcasters, production trucks, cameras, HDR workflows, encoders, apps and millions of playback devices involved.

The NFL is not starting from zero, either. Dolby says the league already uses Dolby OptiView technology within NFL+ for video playback and monitoring. What Seattle is doing differently is adding Dolby Vision to a team-level OptiView implementation.

So this is less a case of the Seahawks discovering technology the NFL has never heard of and more a demonstration of what can happen when one organization has greater control over the entire viewing experience.

The Rest of the NFL Should Be Watching

Seattle’s experiment could become an important test case.

If fans respond positively, other NFL clubs have little reason not to investigate similar technology for preseason games, press conferences, team-produced programming and other content they control directly. The larger opportunity belongs to the league and its broadcast partners.

NBCUniversal has already demonstrated that premium live sports can be delivered at scale. Super Bowl LX and the 2026 Winter Olympics were presented in 4K HDR on NBC and Peacock, and Peacock has been expanding Dolby Vision and Dolby Atmos across its live sports programming.

The question is no longer whether premium HDR can work for live sports, but how quickly it can be implanted and whether the manufacturers and league can promote the benefit to viewers. Because it’s not like anyone watches professional football on Sunday.

MLB and the NBA Are Already Moving

Baseball and basketball do not actually need Seattle to tell them that this is coming.

Peacock announced plans to expand Dolby Vision and Dolby Atmos across Sunday Night Football, NBA and MLB coverage throughout 2026. The platform also streamed Telemundo’s Spanish-language FIFA World Cup coverage in Dolby Vision and Dolby Atmos this summer.

That is important because baseball and basketball are particularly good showcases for modern HDR displays. A baseball game offers bright daylight, stadium lighting, uniforms and huge differences between shaded and illuminated areas. Basketball combines brightly lit courts, dark seating areas, saturated uniforms and constant motion. Apple is so desperate to sell the Vision Pro that immersive 8K baseball is coming later this month. Professional sports in Dolby Vision is most certainly an easier sell.

What About the NHL?

Hockey might be one of the most interesting candidates of all.

The 2026-27 NHL season will continue to be distributed nationally in the U.S. through ESPN/ABC and TNT Sports, including HBO Max, while Sportsnet remains a major Canadian partner.

There has not been a comparable league-wide Dolby Vision announcement from the NHL.

That feels like an opportunity.

Hockey presents television cameras with an unusually difficult image: a huge bright sheet of ice surrounded by darker seating areas, fast-moving players, dark pucks and rapidly changing camera angles. Better HDR production and display handling could be particularly valuable in maintaining highlight detail and contrast.

The NHL and its broadcasters should be paying attention to what Seattle, Peacock and Dolby are doing.

The Bottom Line

The Seahawks are not suddenly giving every Seattle game the Dolby Vision treatment. For now, the big consumer-facing change applies primarily to preseason games and team-produced streaming content, with geographic and device restrictions applying.

But that almost misses the larger point.

Seattle has demonstrated that a professional sports team can control its own streaming experience and deliver Dolby Vision, lower latency and synchronized playback directly to fans using devices many of them already own. That should put pressure on everyone else.

Consumers have spent years buying OLED and Mini LED televisions capable of extraordinary HDR performance. Streaming services and sports leagues are finally starting to provide live content capable of taking advantage of them.

Related Reading:

Players can now play classic console games natively on Mac thanks to a growing scene of fan-made decompilations and recompilations. Here’s how to get started.

If you’ve been around the block a time or two, you’re probably familiar with emulation. It’s the process in which a piece of software emulates an environment that another piece of software can run on.

But there are problems with emulation. The first is that it relies on an emulator, which can come with its own host of problems.

The first is that no matter how good an emulator is, it’s not going to perfectly emulate original hardware. This can create glitching or unexpected behaviors.

It also adds an entire additional level of processing. Not only are you running a game, but you’re also recreating the game’s original hardware.

But emulation isn’t the only way. There’s always the option to play native ports, too. Read Only Memo has collated a list of such projects, and we thought they merited a closer look.

There are two main methods of reaching a native port: decompilation and recompilation. I’ll explain them in more detail below, but I want to give a quick shout-out to Redditor SoulWager, who explained it in a delightful way.

We can think of both processes as baking a cake, but the path to the final product is notably different.

Recompilation is like baking a recipe you know in a new kitchen. You more or less know everything that goes into it, but you might need to change a few things to get the desired result.

Decompilation is like starting with a finished cake and reverse engineering the recipe. You might not wind up with the same ingredients, but you’ll have a similar end product.

Native ports can offer better performance and tighter integration with modern hardware than emulation. They can also make features like higher resolutions, widescreen support, modern controllers, and modding significantly easier to implement.

So let’s talk about the differences between the two.

Decompilation

Decompilation is, effectively, when a developer reverse engineers a game’s compiled code to recreate the source code. The eventual goal is to reproduce the original game’s behavior while also creating code that can be adapted to other platforms.

It confers some unique advantages over static recompilation, namely that it provides full debugging capabilities. It is also better for preservation and makes asset replacement much easier.

It is a time investment, though, and little bits of weirdness can happen. One developer noted that this seemed to happen quite a bit with audio.

Static recompilation

Static recompilation takes a different approach. Rather than recreating the game’s source code, a developer will translate the existing code into something that runs on a modern platform.

This has some benefits over decompilation. The first is that it’s generally much faster to do, which is a benefit if you’ve got a long list of projects to get through.

One developer noted that audio worked right out of the box when statically recompiling Star Fox 64. They also noted that static recompilation can preserve undefined behavior from the original game.

Of course, it’s not without its drawbacks, either. The memory layout can make certain types of modding more difficult. It also means that the game must be recompiled after every code change.

There’s no clear superior option here, especially for the end user. It’ll probably boil down to the individual game.

So, if a game has both a decompiled and a recompiled game, you might need to play both to see which you like better.

Some assembly required

Of course, it’s not just as easy as hopping on over to GitHub and downloading a port and playing Mario 64 within 20 minutes.

These projects generally don’t include the original game’s assets, because a platform (in this instance, Nintendo) would stomp them into the dirt with a cease-and-desist first, and a full lawsuit later.

Instead, you’ll need to provide a copy of the game data. Now, how you do this is up to you, and we really can’t tell you how to do that.

Projects worth checking out

All that said, we figured we’d highlight some of the more interesting, Mac-compatible projects out there. Again, we suggest that you check out Read Only Memo’s write-up, too, as they’ve done an excellent job uncovering a plethora of playable ports.

It’s also a great time to remind you of Mac Source Ports. We’ve mentioned them before, and they’re a great way to play native app builds of old games that never got an official release on Mac.

There is a step in the development process for large language model (LLM)-assisted tooling that most teams skip because it’s tedious, time-consuming, and doesn’t produce results visible to end users: Verifying that what the model is saying is actually correct. Not fluent, not coherent, not topically relevant — correct in the sense of accurately identifying the right answer to the specific problem the tool was built to solve.

The gap between “this output sounds right to me” and “this output is verifiably correct” is where most LLM-assisted enterprise tools fail quietly. They pass internal review because the output sounds right. They fail in production because those people weren’t reviewing against ground truth — they were reviewing against their intuition about what a good answer looks like.

This distinction matters more as LLM-assisted tools move from productivity accessories to components that influence real business decisions. If your AI-assisted tool is shaping how an analyst investigates a data quality issue, how a compliance reviewer decides whether to escalate a flagged record, or how an operations team triages a validation failure — the accuracy of its output has real consequences. “Seems reasonable” is not an adequate evaluation standard for that.

What qualitative evaluation actually catches

The standard evaluation approach for LLM output in enterprise tooling is qualitative: A sample of outputs is reviewed by someone with domain knowledge, judged against a mental model of what a good answer looks like, and the prompt is adjusted if too many outputs seem off.

This catches a specific class of problems: Outputs that are obviously wrong, poorly formatted, or off-topic. These are real issues worth catching. They’re also the easy ones.

What qualitative evaluation consistently misses is the class of outputs that are wrong in ways that are difficult to see without checking against something external. An explanation that confidently identifies the wrong root cause, in language that sounds authoritative, based on reasoning that sounds plausible — this passes qualitative review. It fails the moment someone with the right context checks it against what actually happened.

In a system whose value proposition depends on accuracy, “sounds plausible” is not the same as “correct.” The two can diverge significantly, and qualitative review won’t tell you when they have.

What an actual eval harness looks like

The alternative is building an evaluation harness that scores model output against labeled ground truth — a set of cases where the correct answer is known, against which you can measure accuracy rather than coherence.

I built this while developing a root-cause explainer for data migration drift: A tool that takes a detected drift event and generates a ranked explanation of what most likely caused it. The first prototype produced fluent, specific-sounding explanations that passed qualitative review. When I tested it against cases where I already knew the root cause, the explanation was wrong often enough to matter.

The eval harness I built works in three parts.

First, a synthetic ground truth dataset: Cases where the correct answer is known by construction. This meant introducing specific, controlled causes into a test pipeline — schema changes, transformation logic bugs, source system behavioral shifts — recording exactly what I introduced, and running the model against the resulting drift events. The correct answer for each case was the cause I had deliberately introduced.

Getting the synthetic scenarios realistic enough to be useful required more care than I expected. Early versions were too clean — the drift signal was obvious in ways that real production drift events aren’t. Adding realistic noise, overlapping signals, and cases where multiple plausible causes were present simultaneously was what made the synthetic set actually predictive of real-world performance.

Second, a scoring function that evaluates ranked output. Binary correct/incorrect isn’t sufficient when the model produces a ranked list of likely causes rather than a single answer. An explanation that correctly identifies the root cause as the third most likely candidate is meaningfully different from one that identifies it as the most likely. The scoring function evaluated two dimensions: Presence — did the correct answer appear in the output at all — and rank — how prominently was it featured relative to incorrect candidates. These were combined into a weighted score that rewarded both finding the right answer and ranking it appropriately.

Third, systematic evaluation across the full synthetic dataset rather than spot-checking. Running the harness across the complete set reveals patterns that spot-checking misses: Which categories of problem the model handles reliably, which it consistently gets wrong, and which combinations of signals produce the highest rate of confident incorrect explanations.

What the evaluation revealed

The results were more informative than any qualitative review could have been.

Schema change scenarios scored well — the model was reliable at identifying upstream schema changes when the evidence was present and distinctive. Transformation logic bugs were harder — the model consistently identified the right general category but misattributed the specific change that caused the problem, particularly when multiple changes had been made close together. Overlapping-signal scenarios were the hardest — cases where two different causes occurred close in time produced the highest rate of confidently wrong explanations.

That last finding is the one that qualitative review would never have surfaced. The model’s expressed confidence didn’t correlate with its accuracy — it was most confident in the cases where it was most wrong. Without the eval harness measuring against ground truth, that pattern would have been invisible.

The practical implication for enterprise AI deployment

For teams deploying LLM-assisted tools in enterprise contexts — particularly tools that influence how people investigate problems, triage alerts, or make routing decisions — the eval harness question to answer before production deployment is: Have we measured accuracy against cases where we know the right answer, or have we only reviewed whether the outputs seem reasonable?

If the answer is the latter, the tool has been tested for fluency and coherence but not for correctness. Those are different properties. For tools that shape business decisions, correctness is the one that matters.

Building the synthetic ground truth dataset is the hard part and the part most worth investing in. It forces you to define precisely what “correct” means for your specific use case — which turns out to be a useful exercise independent of the evaluation itself. The scoring function and the harness infrastructure are relatively straightforward once you have that definition. Without it, you’re measuring something other than what you’re trying to guarantee.

Arun Mishra is an enterprise architect.

Welcome to the VentureBeat community!

Our guest posting program is where technical experts share insights and provide neutral, non-vested deep dives on AI, data infrastructure, cybersecurity and other cutting-edge technologies shaping the future of enterprise.

Read more from our guest post program — and check out our guidelines if you’re interested in contributing an article of your own!

On August 20, 2019, Apple officially launched Apple Card with any US citizen able to apply. Since then it’s been a hit, but it’s faced controversy over everything from low reward rates to alleged sex discrimination — and Apple’s partner wanting to pull out.

In every sense bar one, Apple Card is typical of the products Apple has been most famous for making. First it was rumored for years, then it launched to fanfare and criticism, plus it was even thought of first by Steve Jobs. The difference is that apart from its titanium card, it’s not hardware, and apart from its app, it’s not really software either.

Apple Card belongs to the moment when Apple most clearly pivoted from being a hardware company that also did software, to one it’s arguably a services firm. Apple was heading that way for a long time, and it’s not as if its hardware and software are going away.

But Apple’s March 25, 2019 special event marked a turning point. On that day, Apple eschewed its regular launching of new MacBooks, iPads or education products.

Instead, those products were hurriedly announced in press releases ahead of the event, so that the day itself could be devoted to services. Most of the event was the announcement of what was then called Apple TV+, but Tim Cook also confirmed that the rumored Apple Card was coming.

This is another departure from the Apple of old. It’s now rare for the company to reveal anything and conclude with the words “available today,” but most of that event left us waiting a long time.

While Apple News+ was announced and actually launched, Apple TV+, Apple Arcade, and Apple Card were all given the full launch treatment — bar an actual date.

Apple Card pre-history

We would end up waiting 148 days before Apple Card was available. But in truth, we’d really been waiting for something more like 15 years.

Following the departure of the GE Capital Apple Credit Card, used to exclusively buy Apple products, Steve Jobs proposed an Apple-branded credit card. It wasn’t a casual thought, either, as the company went very far down the line of making it a reality.

Original marketing materials for what would have been an Apple Card in 2004. Notice the name on the card: Richard O Croswell was an executive at MBNA at the time.

According to Apple’s then advertising agent, Ken Segal, plans were sufficiently advanced that an ad campaign was mocked up and ready to go. Seemingly Jobs couldn’t get the terms he wanted out of card companies like Mastercard, and he dropped the entire plan.

Tim Cook may not necessarily have got exactly the terms Apple wanted either, but he did negotiate a deal with Goldman Sachs that at least appeared to work for both companies.

While Goldman Sachs had worked with Apple since Steve Jobs returned to Cupertino, it appears to have specifically begun negotiating about what would become Apple Card from sometime in 2018. The key difference between then and 2004, though, is that there was now Apple Pay.

Goldman Sachs reportedly wanted a consumer product, instead of its business-to-business financial offerings, and Apple had this extremely successful payment system. Launched in 2013, Apple Pay is still expanding and sees transactions across the US and more than 50 countries.

Waiting to launch

Based on the existing wide reach and high adoption of Apple Pay, the one group that didn’t wait for the actual launch of Apple Card was the financial press. Investment banks such as HSBC predicted that the “large potential captive market” would mean a windfall for Apple and Goldman Sachs.

Jennifer Bailey, Apple’s Vice President of Internet Services, and Apple Pay, announcing Apple Card

Basing calculations on the idea that around half of Apple’s then 146-million adult users in the US could qualify, HSBC predicted net income up to $300 million in the first year and up to $1.5 billion by 2025.

Those were guesses, but what we learned for sure shortly after launch was that Goldman Sachs’ investment in Apple Card was serious, working out at $350 per user.

And Goldman Sachs specifically told its investors in October 2019 that Apple Card had gone quite well. It was “the most successful credit card launch ever,” said the company.

Tim Cook said much the same in an Apple earnings call in July 2020. Asked about how buyers had adapted to the coronavirus lockdowns, Cook said that it was clear from Apple Card how things had moved.

“We saw changes in consumer spending as the shutdowns occurred and store closures occurred, we could see that across the Card,” he said. “It affected the categories that you would guess the most, like travel and entertainment etc.”

“But overall,” he continued, “if you sort of pull the lens out on the Apple Card, we’re very happy with the number of people that have [one].”

“We believe based on what we’ve heard that it’s the fastest rollout in the history of credit cards and so we feel very good about that,” he concluded.

The controversies begin

Back when that rollout was beginning, Apple may have opened applications to all US citizens in August, but there were many people using it before then. They consisted of various trial runs, some Apple staff, and selected beta users. And some of them had problems.

Specifically, very shortly after the launch, they were already able to say that after a month’s use, the titanium Apple Card showed wear. Apple’s legion of support documents for Apple Card user quickly added one about how to take care of it — and clean the titanium.

Speaking of titanium, though, it turned out that the card wasn’t entirely made of that material, not absolutely entirely. For a start, the white finish is a coating, but people with a scanning electron microscope and not enough real work to do, dug further.

The University of Berkley reports that Apple Card is 90% titanium. The remainder is an alloy that chiefly comprises aluminum.

Vastly more serious was the controversy that began in November 2019 where Apple and Goldman were accused of sex discrimination. The first accusation came from David Heinemeier Hansson, then best known for creating Ruby on Rails, but now perhaps better known for complaining about the App Store and its treatment of his Hey email app.

Although Apple promotes the card as having been “created by Apple,” it was always really curated by Goldman Sachs. That company, along with unspecified other partners, was the one that determines eligibility for the card and sets credit limits.

“We have not and never will make decisions based on factors like gender,” responded Goldman Sachs Bank CEO Carey Halio said in statement. “In fact, we do not know your gender or marital status during the Apple Card application process.”

As Goldman Sachs denied any gender bias in its credit ratings, Apple co-founder Steve Wozniak claimed to have seen a similar disparity between him and his partner. And then US Senators got on the case.

Goldman Sachs’s Halio asked Apple Card holders to contact the company if they had received unexpected credit limits. “If you believe that your credit line does not adequately reflect your credit history because you may be in a similar situation, we want to hear from you,” she said. “Based on additional information that we request, we will re-evaluate your credit line.”

The New York Department of Financial Services announced in November 2019 that it would be formally conducting an investigation. In March 2021, it reported that the investigation “did not produce evidence of deliberate or disparate impact discrimination but showed deficiencies in customer service and transparency.”

You can now pay for an iPhone in instalments using Apple Card

Positives and negatives

In good news about Apple Card, December 2019 saw Apple add the ability for users to buy iPhones on their cards and pay in instalments. Apple Card offers 3% Daily Cash on purchases made from Apple, and despite the instalment plan, buying an iPhone counts as one purchase.

So this was both a long-term aid to spreading the cost of a new iPhone, years before the introduction of Apple Upgrade, and it was an immediate benefit, too. Then in June 2020, Apple extended the programme to mean you can buy almost any Apple devices this way, just not an Apple Watch or an iPod touch for some reason.

There was also the way that to help people during the COVID-19 situation, Apple allowed Apple Card users to defer their March 2020 payments. The company continued to do that through the next several months.

Similarly, depending on your point of view, Apple also launched a way to help people whose poor credit scores meant they were refused an Apple Card — or Apple found a way to get more users. In July 2020, the company launched “Path to Apple Card,” a program that helps them assess and improve their financial situation.

Included in the four-month program is advice that ranges from how reducing debt and paying on time helps. At the end of the program, users are invited to apply again for an Apple Card.

This launched alongside a new Apple Card financial health website. So there’s definitely an educational element here, but arguably it was prompted by allegations that Apple and Goldman Sachs were accepting applicants with “sub prime” credit records.

Apple iterates on the Apple Card

We’ll never know what changes or improvements Apple and Goldman Sachs may have done to their internal processes across the first year of Apple Card. However, as that year was coming to an end, Apple did launch a couple of new features that were surprisingly absent in the original release.

Specifically, it made some online changes. Whereas before, you could only pay off your Apple Card via your iPhone, from July 2020, Apple added a way to do it online.

Then Apple added a feature you could’ve sworn must’ve been there from the start. As of August 12, 2020, you can now use Apple Card to buy from the online Apple Store.

Apple Card 2.0

Although Apple could offer better rewards for using Apple Card, otherwise it’s hard to see what could be added next.

Right now Apple Card appears to be working for potentially millions of people, perhaps undertaking billions of transactions. It’s bringing those people some rewards, and some convenience, while presumably earning Apple a nice amount.

If it’s only tenuously hardware, and tenuously software, this Apple Card service represents the new Apple in every way — bar one.

Apple Card remains exclusively available in the United States. That’s despite an early promise from Goldman Sachs, made before the card had even officially launched.

“We’re going to start in the US,” said Goldman Sachs International CEO Richard Gnodde, “but over time, absolutely, we will be thinking of international opportunities for it.”

Tim Cook said similar things, but even after seven years and JP Morgan Chase beginning to take over from Goldman Sachs, there’s been no official word of a wider rollout.

Tumbleweed and regrets

As it turned out, it took until 2023 before there would even be one rumor of Apple Card coming to any other country. Multiple sources then reported that Tim Cook held talks with banks in India while he was there for the opening of the country’s first two Apple Stores.

There have been small adjustments to the terms and conditions of Apple Card, including one where buyers can only use its monthly instalment feature on iPhones that come with a carrier instead of simply being unlocked.

Much more welcome was the long delayed launch of a high yield savings account in April 2023. Apple Card users could use this account as a regular savings one, and also have their Daily Cash automatically go into it.

That saw $1 billion in its first four days, and then a total of $10 billion saved into it just over its first three months.

Not everyone wins

But even as all of this was seemingly going very well for Apple, it doesn’t appear to have been great for Goldman Sachs. The company has pulled out of all its other consumer projects and while that’s not all down to Apple Card, the costs of that were a contributor.

From around mid-2023, it started to be first rumored and then reported that Goldman Sachs wanted to get out of its Apple Card deal.

At one point, Apple issued a statement that was notable for taking quite a few words in order to say nothing.

“Apple and Goldman Sachs are focused on providing an incredible experience for our customers to help them lead healthier financial lives,” said the company. “The award-winning Apple Card has seen a great reception from consumers, and we will continue to innovate and deliver the best tools and services for them.”

Those tools and services would not, though, include a previously planned stock trading app. Apple and Goldman Sachs reportedly abandoned that plan in September 2023.

The future of Apple Card

Goldman Sachs and Apple had a deal to continue working together until 2029. But in January 2026, Apple confirmed the rumors that Apple card was moving.

It will take at least a couple of years before the transition is complete, but Goldman Sachs is going and JPMorgan Chase is in.

“We’re incredibly proud of how Apple Card has transformed the credit card experience for customers by delivering innovative tools that empower users to make healthier financial decisions,” said Jennifer Bailey, Apple’s vice president of Apple Pay and Apple Wallet at the time. “Chase shares our commitment to innovation and delivering products and services that enhance consumers’ lives. We look forward to working together to continue to provide a best-in-class experience and exceptional customer service with Apple Card.”

It’s expected to take until early 2028 before Chase has fully taken on Apple Card. That length of time points to the complexity of the operation, and may also be a clue why Apple needs the backing of a firm like Chase.

Apple Card head Jennifer Bailey is retiring after overseeing the deals with Goldman Sachs and JPMorgan Chase

Apple clearly has the funds to run Apple Card entirely by itself, and at times it was rumored to be considering it. Since launching Apple Card, Apple briefly launched a Buy Now Pay Later (BNPL) service, which it did back itself.

Apple Pay Later, as it was called, rolled out in October 2023 and was reportedly a success.

But presumably it wasn’t enough of a hit, as Apple axed it less than a year later, in June 2024.

What happens next

For now, no existing Apple Card user should notice any difference, and they’re unlikely to see any extra benefits. That said, there was an unusual offer in May 2026 that offered AirPods Pro 3 for free for people joining Apple Card, but it was much more limited than it seemed.

Depending on how well that worked for Apple, there may be more to come. And Apple is at least hinting at there being “an exciting future ahead.”

Apple’s Eddy Cue said that in a note sent to all employees to announce the retirement of Jennifer Bailey, vice president of internet services and Apple Pay. “She also led the development and growth of… Apple Card, and led our work in Services fraud prevention and partner operations,” said Cue.

Unsurprisingly, then, Bailey was involved in the deals with Goldman Sachs and Chase. Her successor will be announced in October 2026, but it’s still going to be more than a year until there can be any change with Apple Card.

There are credit cards that offer better rewards, but none that are so tightly integrated into the Apple ecosystem.

No other credit card has the same immediate access to particularly clear details of a user’s account. No other credit card makes managing the account as straightforward.

Apple is now definitely a services company, but it’s this combination of services with hardware and software that gives Apple Card an edge.

Whether you are trying to keep your invoices organized for tax season or just want a digital backup of important papers, scanning documents on your iPhone is one of those small habits that pays off later. 

That’s why document scanning apps have become so popular on the App Store. But what most people don’t realize is that you do not need to download a single third-party scanner app. Your iPhone already has several built-in tools that do the job well, and today I will show you how to use them.

Scan documents using the Notes app

If Apple Notes is your main repository of notes and documents, you will be happy to know that you can scan your invoices directly inside the Apple Notes app. It gives you two ways to scan a document, and honestly, either one works great depending on what you are trying to do. 

If you just want to scan something without worrying about which note it lands in, long-press the Notes app icon on your Home Screen and tap Scan Document. This launches the built-in scanner right away. From here, you can either let the app automatically detect your document’s edges and scan it, or tap the shutter button to capture it manually.

If you aren’t happy with how it turned out, you can use the Retake button to try again. Once the scan looks good, you can fine-tune the boundary by dragging the corner handles. If you have multiple pages to scan, repeat this same process for each one. 

When you are all done, tap the yellow checkmark button, and the app will bundle the pages into a single PDF and drop it into a new note. If you would rather have separate files instead of a combined PDF, tap the button after each individual scan instead of waiting until the end. 

If you want to save a document inside a specific note, you can do that too. Just open the note where you want to save the scanned document and tap the paperclip attachment icon in the keyboard toolbar. Now, tap the Scan Documents button and follow the steps to scan your documents. 

Scan documents using the Files app

The Files app follows a very similar routine, and honestly, it might be the better home for your scans if you are trying to keep things organized by folder. Long-press the Files app icon on your Home Screen and choose Scan Documents to jump straight into the scanner. 

From here, everything works the same way as it did in Notes. You capture the page, adjust the corners if needed, and repeat for as many pages as you need. The only extra step here is that once you tap Save, you will need to pick exactly which folder you want the file saved to before confirming in the top-right corner.

If you already know exactly where you want the scan to live, there is an even faster route. Open that specific folder inside the Files app, tap the three-dot menu in the top right corner, and select Scan Documents. Your file will save directly into that folder without any extra navigating.

Scanning documents is easy

Personally, I use the Files app to scan and save invoices and documents. Each type of document has its own folder, so everything is neatly organized and I can quickly find them. If you have been paying for a third-party app until now, I highly recommend you try the native option and see if you can remove a subscription from your list.

A friend from my old hackerspace was in grad school for electrical engineering. He had a professor who would ask, when something went wrong with a student project, “Have you fully characterized the system?” It’s a good, if lofty, goal, but it also became an inside joke around the hackerspace because YOLO was our MO about 95% of the time. Head crashes on the 3D printer – “not fully characterized”. Forgot to take out the trash last weekend? Was the system fully characterized?

It’s maybe also the difference between theory and practice: In theory, there’s no difference between theory and practice, and all systems can be fully characterized. But in practice, it’s hard to fully characterize a system that you don’t yet fully understand.

Case in point: we have nine small saplings growing in our front yard, and I have to water them. It’s boring moving the hose from tree to tree, so I thought I’d take a length of hose, stopper it at one end, and drill enough holes in it so that it could irrigate all of the trees at once. I kinda characterized the system: I figured out how much water flows per minute through our hose, and divided that up into a reasonable outflow in my mind, and drilled holes that ended up being way too large.

Why? Because a length of hose has a resistance to flow, and the water came pouring out of the first few holes, while the last few were dry. It wasn’t a constant pressure system like I thought it would be. I hadn’t even thought that the drag in the hose would matter, so there was no way I would have tried to measure it. But how would I characterize this resistance anyway? You could make a hose with too-large holes and measure the falloff. (Oops, that’s exactly what I did.)

In retrospect, professional drip irrigation systems always have holes that are tiny relative to the pipe diameter, which avoids this pressure-drop phenomenon, which means that they don’t have to worry about characterizing the hose resistance. So that’s what I ended up doing. I cut the hole size in half, and later widened up some of the downstream holes until it looked about right. Not even close to fully characterized, but it works.

So now, in addition to the engineer’s “have you fully characterized the system?”, I have the hacker’s “can you avoid characterizing parts of the system?” in my mind. And a holey chunk of hose in the trashcan.

Supercon News

Just briefly, in case you missed it: Tickets are on sale now for Supercon Ten, and we’ve extended the call for participation by another two weeks. If you’re a Hackaday fan, you owe it to yourself to join us at our annual gathering.