In a nutshell: Amazon’s growing use of AI across its operations is starting to expose a practical problem: the technology can become expensive quickly, and in some cases, no one notices until the bill is already high. Internal discussions reviewed by the Financial Times show that several AI-driven projects at the company have run over budget, sometimes by a wide margin.

In one case, Amazon spent $1.8 million on a project that used Anthropic’s Claude Sonnet model to match author information with product listings. The system ultimately failed, and spending exceeded the original budget by 860%. The issue went undetected for five months.

Engineers say the problem isn’t limited to one project. As more teams shift from traditional software to AI models, routine mistakes are becoming far more costly. Tasks that once required minimal computing resources now depend on systems that charge based on usage, often measured in tokens. When those systems are misconfigured or left unchecked, costs can climb quickly.

During a recent internal meeting, senior engineers described these errors as “catastrophically expensive,” noting that similar mistakes in conventional systems were “trivially cheap.” The difference comes down to how AI workloads are priced and executed. Instead of predictable infrastructure costs, teams are now dealing with variable expenses that depend on how often models are called and how they are used.

Other projects have run into similar issues. One effort to build a financial auditing tool generated about $541,000 in unexpected costs. Another project aimed at improving delivery speeds in Amazon’s logistics network incurred $134,000 in additional spending, and it took more than two weeks to catch the problem.

According to people familiar with the situation, engineers are now working on safeguards to prevent similar incidents from happening again. These include automated controls to limit how AI systems are used and improved real-time spending tracking.

Part of the challenge is visibility. “It’s difficult to figure out how much anything [AI related] costs,” one senior Amazon employee told the Financial Times. Unlike traditional software systems, where costs are easier to estimate, AI introduces more moving parts. Model calls, prompt chains, and autonomous agents can all drive usage in ways that are difficult to track until after the fact.

Amazon, for its part, says the company is still in a learning phase. “As with any new technology, we’re experimenting, learning and improving how we use it, including how we drive cost efficiencies,” the company said in an internal presentation.

It also pushed back on the idea that these incidents reflect broader problems, adding that “Cherry-picking small, isolated examples where teams are learning from one another and portraying them as business as usual doesn’t reflect how teams across Amazon are using AI.”

Still, the examples point to a broader shift happening across the tech industry. As companies move away from fixed pricing models and toward usage-based billing, AI costs are becoming harder to predict. Systems that rely on autonomous agents can generate large volumes of activity, sometimes far beyond what teams initially expect.

Amazon has already encountered related issues elsewhere. Earlier this year, AWS experienced outages tied to errors from AI coding tools. In response, the company limited what those tools were allowed to do rather than giving them full access. It has also stepped back from internal efforts to encourage heavy AI usage after costs began to rise.

For a company of Amazon’s size, the financial impact of these overruns is relatively small. The business generates more than $180 billion in revenue each quarter and is expected to spend heavily on AI infrastructure this year. But the incidents highlight a more fundamental issue: managing AI costs requires a different approach than managing traditional software.

Time to deal with that overflowing drawer of tangled old cables.

Go to a local chain store or other depot

Savanevich Viktar/Shutterstock

What cables should you keep?

kurgenc/Shutterstock

What just happened? X has ended a high-profile dispute with the World Federation of Advertisers, closing a legal fight that had become a proxy for broader tensions between the platform and the advertising industry. The company, owned by Elon Musk, confirmed that both sides had agreed to resolve the case and move forward.

In a joint statement, X Corp. and the WFA said they were “putting the litigation . . . behind them” and that the agreement “resets the relationship between the two organizations.” Neither side disclosed the terms.

The settlement brings an end to one part of a lawsuit X filed in 2024, accusing the WFA and several major advertisers of coordinating an “illegal boycott” that diverted “billions of dollars” away from the platform. A federal judge dismissed the case earlier this year, but X appealed in April. That appeal is still ongoing against the other companies named in the suit.

At the center of the dispute was the Global Alliance for Responsible Media, a WFA-backed initiative launched in 2019. Garm was designed to help brands avoid placing ads next to harmful or illegal content, an issue that has become more complex as ad buying has shifted toward automated systems and AI-driven placement.

X argued that the standards promoted through Garm effectively pressured advertisers to pull their spending. The WFA and participating brands rejected that claim, maintaining that companies have the right to decide where their ads run.

As part of the agreement, the WFA said it will not “form or restart Garm or a similar initiative.” That decision removes a widely used framework that had shaped how platforms, agencies, and brands approached brand safety at scale.

The group also emphasized that its position on broader principles has not changed. It said it “reiterates its commitment to freedom of speech, a principle first included in WFA’s founding constitution back in 1953, and a principle it shares with X.” It added that “WFA and X are fully aligned in the view that brands, platforms and consumers will all benefit from brand safety innovation.”

The case stemmed from a sharp drop in advertising revenue at X following Musk’s $44 billion acquisition of Twitter in 2022. Many advertisers cut spending amid concerns about content moderation and the risk of their ads appearing next to controversial material.

Those concerns have been amplified by the way modern ad systems work. Much of digital advertising now runs through automated platforms that use machine learning to place ads in real time. While this increases efficiency, it also makes it harder for brands to control exactly where their ads appear, raising the stakes for brand safety tools and standards.

Musk has repeatedly pushed back against advertisers and industry groups over those issues. When X filed the lawsuit, he wrote, “we tried peace for 2 years, now it is war.” In a separate incident, he told advertisers to “go *uck themselves” as revenue declined.

– Elon Musk (@elonmusk) August 6, 2024

At the same time, X has been shifting toward a more technology-focused strategy. The company merged with Musk’s artificial intelligence venture, xAI, last year, linking its social platform with a growing AI operation that also connects to SpaceX. That move points to a heavier reliance on in-house systems for everything from content ranking to ad targeting.

The agreement with the WFA removes one of the biggest external challenges X had mounted against the advertising industry. But it does not resolve the underlying tension between how platforms manage content and how advertisers decide where to spend.

It’s summer. You are sweaty. There’s some unprecedented heat wave that’s even worse than the last unprecedented heat wave. You are uncomfortable. You’re getting sweatier by the second. What do you do?

This predicament — one that’s becoming more and more common — just flat out sucks. But if you have a handheld mini fan on you, the situation can change from “actively horrific” to “less than ideal.” In sweaty circumstances, that’s a miracle.

Over the last few summers, I’ve bought a handful of random mini fans on Amazon or TikTok Shop. While those $10 to $15 devices have become staples in my summer purse, I noticed that premium brands like Shark and Dyson were rolling out their own handheld fans at much higher price points ($100 to $150). As someone who suffers from the SSRI sweats — if you know, you know — I was curious whether these higher-end products are really that much better than the drop-shipped fans I’d bought in summers past.

The verdict is: they’re all pretty good in their own way! But the best one for you depends on your budget, the size of your go-to summer bag (or pockets), and your tolerance for drawing attention to yourself with a large device blowing air in your face.

Most powerful: Dyson HushJet ($99)

I wasn’t surprised to find that the Dyson HushJet is an exceptionally powerful mini fan — this company is best known for vacuums, air purifiers, and hair dryers. Dyson knows how to make stuff that blows air! But at about 7 inches tall, the HushJet is the tallest and heaviest device I tried. It will still fit in most purses, but it’s not pocketable.

The HushJet comes with a travel bag, a charging/desktop stand, a neck lanyard, and a USB-C charger (all of these fans charge via USB-C — woohoo!).

If my desk weren’t directly below an air vent, I could see myself using this while I work, since it functions so well on a desktop. The neck lanyard looks a little dorky, and I imagined myself wearing it while pushing a stroller at Disney World. This is an odd image, since I don’t have children, but maybe the reason I envisioned a hot day out with kids is because the lanyard frees up your hands? Or, maybe it’s because if I had to give one of these devices to a kid and bet they wouldn’t break it, I’d say the Dyson is the most kid-proof? Unfortunately, the lanyard doesn’t really help the fan blow air at you while you wear it — you still have to pick it up to direct it at your face.

The HushJet is more powerful than any other mini fan I tried, but that power comes with some noise. Luckily, it has five different power settings, and the lower ones are quieter.

Most versatile: Shark ChillPill ($149)

The Shark ChillPill is the most expensive fan of the bunch, but to be fair, it also does the most. The ChillPill comes with three attachments: a fan, a water mister, and a cooling plate that you touch to your wrist or temple to help you cool down. Like the Dyson HushJet, the ChillPill can sit up on its own for desktop use. It’s about 4 inches long and about double the width of the other options because of its dual-cylinder design.

I chose to bring the ChillPill with me to a softball tournament, since I knew I’d already be sweaty enough that a mister would benefit me without making me look unnecessarily damp (there is no universe in which I would not appear damp at an all-day outdoor event in the summer).

You just have to pour a tiny bit of water into the attachment and it works great. The cooling plate is interesting, but I personally found the fan more immediately effective — plus, it has ten speeds!

I might be more inspired to bring the ChillPill with me in my bag if it came with a case for all of its accessories — I wasn’t really sure what to do with the extra little components. I mostly used the ChillPill with its fan attachment, but I was impressed by the product design when it comes to changing out attachments. There’s a locking mechanism, which you toggle when you want to remove or add a new attachment. It feels secure enough that it would be pretty difficult for an attachment to accidentally fall off or something.

I understand why the ChillPill is as wide as it is — there’s probably a lot of technical components in there that make the 3-in-1 design possible. The design works for outdoor events like music festivals, sports games, or beach days, but I prefer smaller fans for other circumstances, like when you’re the only one at the bar who thinks it’s a little hot inside, and you feel weird about that.

Good enough: JISULIFE 3-in-1 Portable Fan ($13)

The JISULIFE 3-in-1 Portable Fan was the first mini fan I ever bought, and it’s served me well for the last three years — it still works just as good as new. I’ve taken it on vacations without AC and found it endlessly valuable. It has a flashlight, though not one that’s any better than your phone flashlight, and a USB plug-in to charge your phone. Of all the fans I tested, the JISULIFE is the weakest (though still decent), and it can be kind of annoying to flip open. Then again, it has little bear ears on it. Does Dyson randomly put little bear ears on its fans? I didn’t think so.

This product isn’t going to knock your socks off. The charging is slow, yet effective. The fan is fine. The light is kind of dim. But it gets the job done well enough that it’s accompanied me on a few “glamping” trips and a trip to Paris during a heat wave. Sure, those trips all took place before I had tried any of these other fans, but I wasn’t sitting around thinking, “Gee, I sure wish I had a better fan.” (Rather, I was reflecting on how the most American thing about me is that I love AC.)

The fan is about 4.5 inches when folded, and about 8 inches when using the fan.

Best overall: This random fan from Amazon ($15)

While over $250 worth of review unit mini fans sat at home, I couldn’t stop reaching for this random MACMORE fan from Amazon when I went out. It’s definitely some sort of sketchy drop-shipping situation — the images on the listing all look very obviously edited. The company, MACMORE, only lists two products (both mini fans) on Amazon. But what can I say? I love this little thing. I even bedazzled it. (Bedazzling kit sold separately, but highly recommended to maximize whimsy.)

At 4 inches tall, this fan is both the smallest and lightest one I tried. It’s not as strong as the Shark or Dyson fans, but it works well enough that I’m not wishing the fan were stronger. It comes with a removable clip, so you can attach it to your shirt or the strap of a crossbody bag to keep cool, hands-free. Like every other fan, it makes some sound, but it’s small and quiet enough that I don’t feel like I’m drawing attention to myself when I use it in public.

When it comes down to it, the size (and low price) is probably what makes me love this fan so much. It takes up less space in my purse than my keys, and it fits in a pocket too, so it’s become a staple. But really, this all comes down to personal preference. I’m optimizing for discretion, but maybe you’ll choose to optimize for a fan with the most features or that has little bear ears on it.

Back in the late 1990s and early 2000s, finishing a film on DVD often meant one more step. People would slide that same disc into the DVD-ROM drive of their Windows PC, wait for the software to load, and suddenly find themselves staring at a whole different set of options. Wallpapers. Screensavers. Side-scrolling Flash games. Synchronized screenplays that played alongside the film. Interviews, photo galleries, and even live online events that used the disc itself as the connection point. This was the brief window when studios packed extra material onto the unused space of a movie disc and aimed it squarely at computers.

InterActual Technologies was the main player. Todd Collart and Chris Brown founded this company in San Jose in 1995, long before the DVD format was fully developed. They received the backing of some major investors, including Warner Bros, Panasonic, and Macrovision. They developed a software program that transformed a standard movie DVD into something much more. Their early attempts were dubbed PC Friendly, but by 2000 it had been formally renamed InterActual Player, and according to Forbes, the studios were ecstatic, with approximately 90 percent of big studios using it. The concept was that InterActual discs would allow you to access high-quality entertainment on demand without having to wait for dial-up to work.

Sale

Monster Vision 2 | Portable Entertainment System with 15.6″ 1080p Display & 60W Speakers | up to 25 Hours…

• Monster Vision 2 features a brilliant 15.6” Full HD IPS Display and 60W audio power – the ultimate all-in-one experience for movies, music, and…
• UP TO 8 HOURS OF VIDEO PLAYBACK: Connect your favorite smart streaming device or gaming console via the dual HDMI ports (3ft USB-C to HDMI cable…
• UP TO 25 HOURS OF AUDIO PLAYBACK: Experience upgraded 60W sound performance for powerful, immersive audio. Stream wirelessly with Bluetooth 5.3, enjoy…

When you insert a DVD into your computer, an installer is automatically launched. Once the player was up and running, it would overlay on top of your existing DVD decoder, such as WinDVD or PowerDVD, which were included with many drives at the time. The UI resembled a terrible old website from the time, with HTML pages, Internet Explorer rendering, and a touch of Macromedia Flash for any interactive parts. Everything was saved on the disk, so you didn’t have to be online unless you wanted to attend a live event or receive a link.

The Matrix is a great example, as the PC Friendly bonuses offered a handy script-to-screen tool that allowed you to read the text while watching the corresponding scene on screen. That’s not all; storyboard images appeared alongside the final cut, an archival version of the film’s 1999 website as a Matrix code screensaver because, well, it’s The Matrix and you need some Matrix-y stuff, and a quiz game. In November 1999, they went a step further by hosting a coordinated online screening in which over 200k individuals began watching the movie at the same time while also chatting with the directors. To their credit, the player did a nice job of keeping everyone in sync.

American Pie made an effort to be unique. Its InterActual package included a multitude of extras, including story summaries, on-set images, actor interviews, and even a “script to film” comparison, in which the written page was displayed alongside the finished scene. In a bizarre moment, it included a Flash game called Rock the Potty, in which Finch is attempting to make a wild sprint around the home without soiling himself after taking a laxative.

Office Space introduced a game called Road Rage Rally in which players attempted to beat Lumbergh to a parking spot. Meanwhile, Goldmember included a Revoice Studio, which allowed users to record themselves speaking over selected sequences’ dialogue. Men in Black included extraterrestrial case files, a shooting gallery, and a 3D headquarters tour, the last of which required QuickTime VR to navigate. The Mummy included wallpapers of Brendan Fraser and Rachel Weisz, as well as a demo for the connected PC game. Even earlier titles, such as the 25th anniversary edition of Jaws, would occasionally include a simple film scan slideshow that could be used as a screensaver.

The window for shipping additional copies of a website closed quickly. As broadband became more generally available, the idea of putting a local duplicate of a website on a DVD became less necessary. Studios began to focus more on these new high-definition formats that appeared in the mid-2000s. Sonic Solutions acquired InterActual in 2004 and included it into their larger line of disc-authoring products. By the time Blu-ray emerged as the new big time format, all of the PC-specific bonuses had largely faded from new releases. The PC Friendly and InterActual logos began to disappear from the cases, and the software became more difficult to work with on newer operating systems. What was once a very ordinary supplementary feature has evolved into an oddity that only die-hard collectors and retro aficionados seek out.

DEEP DIVE For the first time, Nvidia has directly challenged Intel and AMD’s CPU dominance. With the launch of Vera, the AI arms dealer aims to flog its standalone CPUs to as many hyperscalers and other cloud providers as it can.

Alibaba, ByteDance, Meta, Oracle, CoreWeave, Lambda, Nebius, and NScale have already signed up to deploy the chips in their respective clouds.

The follow-on to Nvidia’s Grace CPU promises 88 custom Armv9.2 cores, 176 threads, support for up to 1.5 TB of LPDDR5X memory, and, critically, availability as a standalone platform independent of Nvidia’s GPUs.

But beyond that, and a mountain of marketing about how it’ll be the best CPU for everything AI, Vera’s inner workings have largely remained a mystery until recently.

That changed late last month, when Nvidia released a whitepaper spilling the beans on its first fully-custom CPU, which is far weirder than anyone could have anticipated.

Ostensibly, Nvidia is aiming Vera at two key workloads: the first and least surprising is as the AI head node responsible for managing the GPUs in its upcoming Vera Rubin systems. The second, and more contentious, is as a host for AI agents, which, unlike the large language models (LLMs) that power them, don’t actually run on GPUs.

From what we gather, much of Vera’s core architecture is predicated on quashing pipeline and execution bottlenecks in order to make it more effective in these roles. But before we dive into Nvidia’s Olympus core, let’s revisit the chip itself.

Monolithic compute, multi-die memory and I/O

Peel back Vera’s rather substantial heat spreader and you’ll find an assortment of chiplets responsible for I/O, memory, and compute. However, this isn’t another rehash of the chiplet architecture popularized by AMD.

Unlike x86 processor makers, which spread dozens of cores across multiple dies, Vera’s compute die is monolithic. All 88 cores are housed in one big chunk of silicon that from what we understand is fabbed on TSMC’s 3nm process. Nvidia argues its monolithic compute architecture has benefits for both core-to-core bandwidth and latency compared to competing designs.

Surrounding the compute die is an assortment of chiplets, including eight LPDDR5x controllers, and what appears to be two distinct I/O dies, one responsible for PCIe 6.4 and CXL 3.1 connectivity and another dedicated to the chip’s NVLink Chip-to-Chip interface.

In terms of package design, Vera is somewhat reminiscent of Amazon’s Graviton 4 CPUs, which also combine a monolithic compute die while disaggregating the I/O and memory functionality to dedicated silicon.

The Vera CPU Superchip

Like most modern datacenter CPUs, Vera supports both single- and dual-socket configurations, the latter of which Nvidia calls the Vera CPU Superchip.

Unveiled at GTC in March, the Superchip features two Vera CPUs connected over NVLink-C2C at a blisteringly fast 1.8 TB/s of bidirectional bandwidth, for a total of 176 cores and 352 threads.

The two chips are fed by 16 SOCAMM2 LPDDR5x memory modules, which deliver 2.4 TB/s of aggregate memory bandwidth (1.2 TB/s each) — roughly twice the bandwidth of AMD’s Turin Epycs, which were launched in 2024.

Nvidia’s agentic AI reference designs call for cramming as many as 128 of these superchips (256 CPUs) totaling 22,528 cores and 384 TB of memory into a single liquid-cooled rack.

Summiting Olympus

Previously, Nvidia had relied on off-the-shelf CPU cores from Arm. For example, depending on the iteration, Nvidia’s Grace CPUs used Arm’s Neoverse V2 (GB200/300), Neoverse V3 (AGX Thor), or Cortex X925 and A725 cores (GB10), depending on which was most convenient.

With Vera, Nvidia has moved on to designing its own ARMv9.2-compatible core, called Olympus. In fact, it’s up for debate how custom the chip really is. We’ve been told that the chip builds heavily on existing Arm IP, and looking at Olympus’ architectural block diagram, we can see why.

Olympus features a 10-wide decoder and dispatch, eight integer arithmetic logic units (ALUs), six vector/FP pipelines (SVE 128), four load units, and 2x store units. So it’s got a fat front and back end compared to the Zen 5 cores found in AMD’s Turin Epycs or the Redwood Cove cores used by Intel’s Granite Rapids Xeons.

But compared to off-the-shelf Arm cores, Olympus’ front and back end aren’t breaking any records. In fact, Olympus looks a lot like a heavily modified Cortex X925.

With that said, Nvidia claims Olympus features an entirely custom neural branch predictor, which is capable of exploring two branches simultaneously.

Branch prediction is key to performance in modern CPUs, and involves anticipating future code paths and executing down them before they’re needed. By predicting two paths per cycle, Vera decreases the likelihood of a mispredict, theoretically boosting its performance in the process.

And from what we can tell, a lot of the customization that went into Vera was made explicitly to stamp out pipeline and execution bubbles in order to maximize the number of instructions per clock (IPC).

In particular, Nvidia claims its neural branch predictor delivers higher hit rates than history-based approaches, making it ideal for running branch-heavy workloads, like the Python scripts generated on the fly by AI code assistants.

Nvidia also claims to have made several innovations to the mid-core, again aimed at mitigating pipeline stalls. Memory renaming speeds up store-to-load dependency chains by allowing dependent instructions to execute before a load completes if the relationship to the data can be divined.

“This is particularly beneficial for pointer-heavy software, graph traversal, runtime frameworks, and complex object-oriented workloads,” the company explained.

Nvidia also implemented a value prediction scheme which “identifies stable dependency chains and predicts future values before they are produced, allowing dependent instructions to execute speculatively while correctness is verified later.”

This functionality, the company claims, helps to speed up repetitive software patterns and sequential data processing.

On the back end, Nvidia Olympus features a similar complement of ALUs and FP units as the X925, but they’ve been implemented differently. Eight of the ALUs are simple integer units which handle basic mathematics, while the company says two complex ALUs handle multiplication and division, CRC, and shift intensive workloads.

The back end also features four dedicated branch units that work with the chip’s branch predictor to resolve control flow decisions that would otherwise cause pipeline stalls.

Meanwhile, for floating point operations, Olympus features six 128-bit Arm scalable vector engines (SVE2) extensions which also support FP8, along with a pair of vector units dedicated to speeding up crypto operations.

Each core is supported by a 64 KB L1 instruction cache, 96 KB L1 data cache, and 2MB per core of L2. The chip also features 164 MB of system level cache (SLC) that’s sharded across the chip.

SMT makes a comeback on Arm

Perhaps Olympus’ most interesting party trick is support for simultaneous multithreading (SMT).

SMT has been a mainstay of x86 chips since 2002. The technology boosts utilization by enabling two threads to harness idle execution units during a single cycle. While SMT doesn’t double throughput, for certain applications it can deliver double-digit percentage gains.

While Arm has supported SMT in the past — you may recall Cavium ThunderX2’s four-way SMT implementation prior to its acquisition by Marvell — the company has downplayed its value with its own agentic CPU.

Arm’s dislike for the tech didn’t stop Nvidia from implementing SMT-like functionality in Olympus. We say SMT-like because it’s not SMT in the conventional sense. Rather than simultaneous multi-threading, Nvidia markets it as spatial multithreading, which is an apt name given how it works.

Rather than two threads that share the resources of one core, Olympus’ SMT implementation looks more like core bifurcation with a shared cache line. You can either have one fat core or two skinny ones. To our eyes, it looks a bit like multi-instance GPU (MIG) for the CPU.

The white paper makes a big to-do about how spatial multithreading avoids the thread contention you can see with x86 parts, which makes sense, because Nvidia’s threads behave more like individual cores. We wouldn’t read too much into these claims, though. There have always been applications that benefit from SMT and those that don’t, and it’s easy enough to turn it off if you don’t want it.

With that, Nvidia’s SMT implementation could be beneficial to agent sandboxes for agentic workloads. With containers — that’s all these sandboxes really are – there are two workloads. There’s the job you’re running inside the container – let’s say a Python script and interpreter  – and then there’s the container runtime, which is responsible for orchestration of container workloads and managing networking and storage overlays.

These are very different jobs. By bifurcating its cores, Nvidia can keep the workload and runtime separate, which in theory should make for more deterministic performance.

Fat pipes galore

Before we move on to Vera’s memory subsystem, we need to spend a few minutes discussing Nvidia’s second-gen scalable coherency fabric (SCF), which is used to stitch all 88 cores together.

As we mentioned earlier, Vera features a monolithic compute die, which means all of its cores are packed on the same silicon. The SCF is essentially a cache-coherent superhighway designed to connect cores together at a whopping 3.4 TB/s of bisectional bandwidth.

As Nvidia explains it: “SCF is organized as a distributed coherent interconnect connecting multiple cores through a collection of Coherency Switch Nodes (CSNs). Each CSN connects up to two cores and serves as a routing point within the fabric, enabling efficient movement of requests between cores, caches, memory controllers, and external interfaces.”

The SCF is also where the L3 cache lives, which explains why Nvidia opted to shard its SLC across the CPU rather than having one big, centrally located pool of it.

The SCF is critical, as without it, Vera wouldn’t actually be able to utilize its memory bandwidth effectively.

Vera features eight LPDDR5x memory controllers with support for SOCAMM2 modules up to 9600 MT/s. At 1.2 TB/s and roughly 14 GB/s per core, Vera has one of the fastest memory subsystems in the world today, but it’s worth pointing out that the competition’s chips launched nearly two years ago.

Both Intel and AMD’s next-gen CPUs will make the jump to 16 memory channels. We’re still waiting on details on Intel’s 192-core Diamond Rapids platform, but we learned last week that AMD’s Venice CPUs will boast up to 256 cores and support for 8,000 MT/s RDIMMs or 12,800 MT/s MRDIMMs, which translates to between 1 TB/s and 1.6 TB/s.

Assuming AMD’s GMI-links aren’t a bottleneck this time around, the House of Zen’s 96-core Epycs should hit around 17 GB/s per core, and there’s no reason to think Intel’s Xeon 7 processors won’t do the same.

With that said, Vera’s memory subsystem should suck a bit less power thanks to the use of LPDDR5X rather than standard RDIMMs. The company claims under a sustained load, its memory subsystem contributes just 30 to 40 watts of consumption versus 100 to 200 watts depending on channel and DIMM count on a traditional server platform. For reference, Vera’s TDP is configurable between 250W and 450W.

Power consumption might explain why AMD is working on a new Epyc, codenamed Verano, which will also use LPDDR5X.

An I/O trifecta

Vera isn’t bursting just with memory bandwidth, but with I/O bandwidth as well. The chip is among the first to support PCIe 6.4 and CXL 3.1, beating out Intel, and possibly AMD depending on the first systems to actually make it into customers’ hands.

As with past generational PCIe bumps, 6.0 sees per lane bandwidth double to 64 Gbps, which translates to 128 GB/s (256 GB/s bidirectional) for a single x16 slot. A single Vera socket can support 6 such slots (96 lanes in total) for connectivity to NICs, storage, and other peripherals. In the dual-socket Vera CPU superchip config, the lane count does drop slightly to 176 lanes or 11 x16 slots.

Along with PCIe, those lanes also support Compute Express Link 3.1. CXL has been around for a while now, but the 3.1 iteration is notable for its support for fabrics and memory sharing.

As we’ve previously discussed, this capability opens the door to network attached memory appliances — which we affectionately call memory godboxes — where multiple CPUs can share the memory.

Finally, we need to talk a bit about Vera’s NVLink chip-to-chip connectivity. First introduced back in its Grace CPUs, the high-speed interconnect’s bandwidth has now been doubled to 1.8 TB/s of bidirectional bandwidth (900 GB/s up and down).

This high-speed interconnect allows for coherent memory access across connected CPUs or GPUs. In the Vera Rubin superchip, NVLink is how the CPU talks to the two GPUs. Meanwhile for Vera CPU superchip, the NVLink-C2C interface provides a blazing fast interconnect that mitigates many of the bandwidth bottlenecks associated with multi-socket systems.

Performance

In agentic workloads, Nvidia estimates Vera will deliver 1.8x higher per-core performance than AMD’s 128-core Epyc 9755 in benchmarks representative of agentic workloads, and up to 2.6x faster performance in graph traversal jobs.

While that sounds impressive, we’re not reading too deeply into Nvidia’s performance claims just yet. Turin launched nearly two years ago, which makes it an advantageous comparison point. CPU benchmarks are notoriously easy to cherry pick, and microbenchmarks, while useful for quantifying hardware’s capabilities, aren’t always representative of real-world performance.

A classic example of this is core-to-core latency on early AMD Epycs versus Intel Xeons. As we’ve already established, AMD employs a multi-chiplet architecture that, for Turin, used up to eight frequency-optimized cores or 16 density-optimized cores per core-complex die (CCD).

For workloads that can fit within a single CCD, the core-to-core latency is extremely small. But if a core on one CCD needs to talk to a core on another CCD, it has to walk the I/O die, which incurs a stiff performance penalty.

Intel, until recently, has favored higher core-count compute dies, which historically have fared better for latency-sensitive applications, like databases, for the same reason Nvidia opted for a big monolithic compute die on Vera.

Whether this actually confers an advantage varies from application to application, and it’s not a clear cut victory for one approach or the other. Both have their merits. AMD’s chiplet architecture lets them use smaller, better yielding dies, which translates to better margins.

We should also mention that Venice makes the jump to 12- and 32-core CCDs, which means the workload would need to be even more talkative for you to notice the latency hit.

We’ve said it before and we’ll say it again: there has never been and never will be one CPU to rule them all. Vera doesn’t change that.

After getting a peek at Vera’s performance in the Phoronix Test Suite in a selected, Nvidia-dictated set of tests earlier this year, we’re optimistic about Vera’s performance in its target workloads, but look forward to seeing fully independent runs on production hardware, especially with AMD’s Venice just a few months away.

We fully anticipate Vera to sell extremely well, if not for its performance characteristics, then because if you want more than eight Rubins, it’s probably coming with a Vera CPU attached. ®

TechRadar’s reviews crew has been working as hard as ever this month, putting dozens of gadgets and gizmos through their paces to find the ones that are worth recommending. In this Reviews Recap, I’ve pulled together just our very favorites; the products that impressed us enough to score 4.5 or 5 stars in our tests.

This month, we called the formidably tough new GoPro Mission 1 Pro a “fantastic all-round vacation camera”, fell in love with the Marshall Stanmore IV speaker’s “rumbling bass and amp-inspired aesthetic”, and declared the Samsung S95H “one of the best OLED TVs [we’d] ever tested”… despite its rather controversial design.

In brief: The controversy surrounding Flock cameras primarily centers on privacy and fears of a surveillance state. However, the cloud-based license-plate-reading system’s accuracy and effectiveness have also drawn scrutiny. In one particularly egregious case, an internal review uncovered more than 1,000 false positives over two years.

Police in Roseville, California, found that, out of 1,427 incidents in which the town’s AI-powered license-plate cameras flagged a vehicle as stolen or used in a felony, the cameras misread the plate in 71% of cases.

Flock, the startup that supplies the cameras to Roseville and approximately 6,000 other communities across the US, attributed the errors to Roseville’s atypical camera setup, but as Business Insider reports, the results could expose flaws in an already controversial technology.

Flock’s cameras, typically placed on traffic light poles and roadsides, photograph passing vehicles and analyze their characteristics, including license plates, color, make, model, and other details via machine learning. Authorities can search the company’s cloud database for vehicles using descriptions to track their movements, which they say has helped solve crimes and find missing persons.

The AI cameras process approximately 20 billion vehicle scans per month and have raised fierce debate across the US. Communities across the country have deactivated their cameras since last year out of concerns about state surveillance, but some towns, including Roseville, are also dealing with the system’s blind spots.

There, a police supervisor joked about memorizing one driver’s license plate because Flock’s cameras misread it at least six times. In each case, the cameras mistook a 9 on his license plate for an 8. This is more likely to happen if a camera takes a blurry image or if the license plate is obscured by a tree or license plate frame.

Ironically, while Flock claims that the errors stem from how Roseville deployed the cameras, the suburb’s unique implementation likely also prevented matters from worsening.

To avoid recording drivers’ faces, Roseville configured the cameras to only capture vehicles from behind, which Flock says makes them less accurate. However, none of the false positives led to traffic stops, much less arrests, because Roseville officers must verify each report, which is not required in California.

Meanwhile, the LAPD recently let its contract with Flock expire after an internal audit found that 161 innocent people were pulled over in a two-month span. Outdated or inaccurate data from other jurisdictions was cited as a primary cause.

Critics have also pointed out that, despite license-plate reading being their primary function, the AI cameras can also analyze and track people on foot. Police have searched the database for people based on extremely vague descriptions, and numerous officers have been caught using the system to stalk ex-partners.