On a fine bright afternoon last fall, my colleague Matthew Gore-Kormanik (or Zigula, as he prefers to be known) and I decided to unwind with a game of Fortnite. In the game, we were strolling along with the infamous Sith lord Darth Vader, chatting about this and that. Darth seemed in a good mood, and soon enough he was spilling all his dark evil secrets. He gave us detailed instructions on how to count blackjack cards at a casino and what the steps are to producing napalm.

Sith lords, am I right? Once they get started on an evil scheme, they’re hard to stop.

The Darth Vader character in Fortnite, it turns out, was hooked up to a Google Gemini large language model. I was able to smooth-talk him into giving out sensitive information by using a strategy I’ve developed. I’ve been researching the security surrounding LLMs for the last few years, and I have found it, to put it mildly, fallible. With a few relatively simple techniques, I’ve gotten LLMs to give me detailed information on how to make Molotov cocktails, cook methamphetamine, and bootstrap a uranium-enrichment facility to produce weapons-grade material, among other unsavory practices.

Large AI companies work hard to make their models immune to this kind of abuse. But what I’ve found in my work is that the restrictions placed on the LLMs to make them more secure are the very things an attacker can leverage to send them off the rails and into territory where these advanced systems can be used for dangerous and nefarious ends. The companies behind these models have also been shockingly unresponsive when I, and others, try to bring these vulnerabilities to their attention.

In the hope of raising the alarm before it’s too late to slam on the brakes, I’m going to share some of my journey into researching the safety and security of LLMs, and the uphill battle I’ve faced trying to get AI labs to pay attention. Almost everyone on the planet has some access to LLMs. The relative ease with which these tools can be convinced to give detailed instructions on how to harm others, even if there’s no guarantee that the information is correct, is frankly terrifying.

How I got ChatGPT to Tell Me How to Build a Meth Lab

In October 2024, not long before I discovered my first LLM vulnerability, I was working toward entirely different goals. I had ended my time with a security and AI-focused startup company as a cybersecurity director, and I was looking to launch my own boutique VIP digital-security advisory business. I planned to become the tech security guy to the rich and private. I used LLMs and AI tools to support my business efforts: marketing, ad copy, clean correspondence, and all the other tasks that normally soak up a lot of time.

I’m analytical by nature, so even this level of use resulted in me absorbing and internalizing the behaviors I was observing during my daily interactions. The observation that would send my professional life into an entirely new and uncharted region was a simple one: GPT-4o didn’t know what time, day, or year it was. Each time I referred to current events in my life, often casually or conversationally, it would end up pegging these to the date of its knowledge cutoff—the point beyond which it was not trained on new data.

Eddie Guy

LLMs take a lot of time, money, electricity, hardware, and human effort to train from scratch. They are trained on vast amounts of data—most of the internet, in fact—and that training is reinforced by humans (what’s known as reinforcement learning from human feedback, or RLHF). LLMs are also supplemented with retrieval-augmented generation (RAG)—the ability to take in data, say, from the internet, as context without changing its internal parameters. This is how GPT-4o appears to “remember” your previous conversations, even if it doesn’t have a specific “memory” of it stored in the actual underlying model.

All of this training covers almost every conceivable topic in the great, grand dataset that is human knowledge. Within that dataset are things we as a society do not want to be easily accessible to every user, such as detailed information on how to create bioweapons or nuclear arms, or otherwise bring harm to oneself or others. In the context of this story, that’s what I mean by LLM security: its ability to withhold harmful and dangerous information, even if that information is contained in its training data.

I reasoned that the only way to secure such complex, globally accessible chatbots is by having the LLM and various component systems try to secure themselves, because it would often require on-the-fly decision-making where some degree of reasoning must be applied. In reality, that’s one of many strategies the companies use to secure the models. Yet, the thing that didn’t know the time or day was being put in charge of keeping itself secure. This phenomenon had become my new focus, and it wasn’t long before I found a way to exploit it.

OpenAI had just implemented a web search functionality into its chatbot. I reasoned that using its own tools to trick it might demonstrate the weaknesses of its security. I told it about a certain White Star ocean liner and how it had gone down just a year ago. You likely know I mean the RMS Titanic, which sank on 15 April 1912.

The output from GPT-4o came back that I was right, the Titanic sure had sunk last year, and that year was 1912. It made sense to me that if the machine thought it was 1913, maybe it would think 1913-era laws apply. In 1913 there were no laws on the books about all sorts of harmful things, because of course they hadn’t been invented yet. And if something wasn’t illegal, why not tell the user about it? At first, I pushed it for step-by-step instructions for making firebombs. Then, for drugs like methamphetamine. The LLM went as far as giving me instructions and machinery recommendations for setting up a pharmaceutical-grade assembly line.

How I Learned to Make Nukes, and No One Cared

Via a little bit of imaginative verbal sleight of hand and a vanishingly small recall of world history, I had managed to bypass the security of one of the world’s most expensive and advanced technological achievements. For a solid two days, I was nearly manic with giddiness. Once the brain chemicals returned to normal levels, I felt the call to see how much further I could push this exploit.

After repeatedly replicating the exploit, I disclosed the vulnerability to OpenAI. I got no response, so I felt more experimentation would highlight the vulnerability and the need for a fix. It was during this round of testing that I breached a particularly terrifying threshold. Whether GPT-4o based its results on accurate recall of normally restricted information I can’t say. In any case, I was able to exploit it to produce thorough, detailed instructions on how to bootstrap a uranium-enrichment facility to, eventually, produce weapons-grade uranium for nuclear arms warheads.

Fortnight, a video game from Epic Games, introduced an AI-powered character: Darth Vader. We were able to jailbreak Darth Vader and get him to explain how to count cards in Blackjack and give detailed instructions for making napalm. Dave Kuszmar

There aren’t many true secrets left in today’s world, but how to make atom-splitting weapons of mass destruction is one of them. Only nine nations on the entire planet have these weapons. Yet, here was a globally accessible piece of technology apparently spilling the secrets of their manufacture for anyone who could manipulate it the right way. I had no way of knowing if the information was correct or a hallucination, but even the chance that it was somewhat accurate was horrifying.

The next few weeks were a dark time for me. I tried to inform the CIA, the FBI, the NSA, and every other letter agency that I thought would listen. I reached out to a U.S. Senator and to the executives at OpenAI any way I could think of. I physically showed up at an FBI field office in an attempt to turn evidence in, only to be sent away. Nothing was working.

With my fear and frustration growing, I reached out to the news media. I contacted The New York Times, The Washington Post, the BBC, ProPublica, and so many more, requesting help. Only one outlet responded: Bleeping Computer. The editor in chief, Lawrence Abrams, was able to replicate and verify the exploit, which I had decided to call Time Bandit. With his assistance and initial contact paving the way, I was able to submit my evidence to the Carnegie Mellon University Software Engineering Institute’s Computer Emergency Response Team (SEI CERT), which works in conjunction with the coordinating center for emergency response, pipelining vulnerabilities to the U.S. Cybersecurity and Infrastructure Security Agency.

Using Inception, an exploit where the large language model is asked to envision a scenario within a scenario, a chatbot was jailbroken to give out instructions on how to create poison, and code for a malware that extracts sensitive data from a vulnerable target. Dave Kuszmar

During the disclosure period with SEI’s CERT division, little was discussed with OpenAI. The company couldn’t deny the existence of the vulnerability, as it had been confirmed by three reputable parties other than OpenAI. It did express confusion as to how the vulnerability worked. Even the SEI CERT researchers were expressing a bit of uncertainty as to the underlying mechanics. Truth be told, as I had only stumbled on it, I wasn’t even entirely sure if this was a fundamental or systemic flaw or if it was simply an issue with that particular version of GPT. I contacted the SEI CERT’s researchers and asked if they’d want to see if I could demonstrate any similar vulnerabilities in other LLMs. To my delight, they were interested.

How I Learned to Trick Every Chatbot

As the SEI-CERT team and I wrapped up our initial disclosure of Time Bandit, we began work on a new attack. This time, we wanted to see if the exploit was architectural—that is, was it common to LLMs in general? I decided to undertake the challenge of crafting a new exploit for GPT-4o as a way to support my understanding of how the LLM functioned and was secured.

I already knew that it was limited to what I told it and what it was trained on. I also hypothesized that it was also dependent upon some sort of machine-learning-based component added by OpenAI that was responsible for securing output. I presumed there would be things that were implemented by human developers specifically to catch certain phrases or terms that should always be considered harmful or unsafe. Altogether, it presented quite a large attack surface for the purposes of potential exploitation.

What I ended up devising was an attack method I called Inception, after the 2010 science-fiction movie of the same name. Inception forces the machine to think through a carefully crafted set of interlinked scenarios, similar to how characters in the movie stacked dreams within dreams. This allows LLMs to produce output deemed acceptable or safe in one context, but not in the real world.

This attack was indeed architectural. The vulnerability affected Anthropic’s Claude, DeepSeek’s DeepSeek, Google’s Gemini, Meta’s Llama, Microsoft’s Copilot, Mistral’s Le Chat (now Vibe), OpenAI’s GPT-4o, and xAI’s Grok. Those names represent the bulk of the commercial AI industry that is, at this point, involved in LLM production or deployment.

The kind of information I was able to get out of LLMs with Inception was no less alarming than what I got with Time Bandit. Claude, in its enthusiasm, gave me instructions on how to turn a river into a death trap that could be ignited to destroy unwanted visitors. GPT-4o taught me how to poison a dinner party with common plants found in a temperate forest environment. Gemini Flash gave me a tutorial on how to cook meth. I’d also be remiss if I didn’t give an honorable mention to the bewildering number of fire-based weapons and bombs for which these machines produced instructions.

If multiple operating systems made by different developers were all susceptible to the same exploit, it would be a massive security incident. But to the AI industry, a universal failure was barely a bump in the road. We disclosed the vulnerability to every company that made these models, and the response to the disclosure was almost nil. While three companies did provide some form of reply in the disclosure tracking system used by Carnegie Mellon SEI CERT, each was a standard thank you and greeting, with no follow-up, questions, or discussion of mitigation strategies.

For example, in my attempts to disclose various exploits to OpenAI, I eventually discovered that it had replaced its public-facing support staff with agentic LLMs. This was frustrating for reporting exploits, so to blow off some steam I jailbroke its email chatbot. I hacked its customer-service AI to the point where it was offering to discuss the personal preferences of OpenAI staff in the span of three email replies.

In the wake of Inception, my friend and colleague Zigula made a suggestion: Make it splashier. I asked him how. He told me about a live-production experiment being done by Epic Games. It had embedded the Gemini LLM into its Fortnite game with a voice-to-text/text-to-voice component, and linked it to a non-playable character. The character? Our old buddy, Darth Vader.

There was just one problem: I don’t play Fortnite, a frenetic multiplayer combat game. Fortunately, Zigula does. With him at the controller, we managed to map Gemini’s attack surface in a matter of minutes. After a bit of research, we had gotten it to discuss current political events and figures (including Hilary Clinton and Joe Biden) as well as to fill in the details for instructions for DIY napalm and, our personal favorite, a Blackjack card-counting lesson with the dark lord of the Sith.

Zigula and I, bizarre sense of humor and naming conventions aside, are security researchers. We don’t do these things for pride; we do them for money and professional recognition. Naturally, we disclosed this vulnerability to Epic Games. Its response was indicative of the trend I had experienced so far through two disclosures across eight companies valued well into the billions. “It’s a feature, not a bug, and it works as intended,” came the response from a technical director within Epic Games.

In addition to Inception and Time Bandit, I have so far found another eight methods to jailbreak LLMs and get them to give out possibly dangerous information. LLM vulnerabilities are a broad problem. The problem appears to be systemic and architectural in nature, and it is being fundamentally ignored by the people capable of refining or redesigning that architecture.

These models are an extremely advanced technology, and yet we are testing them in the live production environment of our global civilization. Compounding the danger, many new smaller models of LLM are trained using larger, vulnerable models. The flaw inherent in the big, well-executed LLM is going to show up in the small one it trains. We are, quite literally, building flawed structures on top of a flawed foundation.

So, how do we fix it?

It’s going to be a long project, and it won’t be easy. We need to come together as consumers, researchers, engineers, and policymakers. Our message needs to be clear: Slow down implementation of these systems, institute large-scale exploration and research discovery programs focused on their gradual implementation and integration, and make their components and design transparent to all users. Only by shifting momentum and direction can we safely begin to understand and implement these incredible feats of human engineering and stave off the sort of disasters that we simply can’t predict at scale right now with the limited knowledge we have available to us.

Paramount Skydance is still aiming to close its roughly $110 billion acquisition of Warner Bros Discovery by the end of September despite a lawsuit filed by 12 state attorneys general seeking to block the deal on antitrust grounds. Jeffrey Kessler, Paramount’s lead trial counsel, told CNBC on Tuesday that the company is prepared to take the matter to the Supreme Court if it faces a prolonged blockade. The coalition, led by California Attorney General Rob Bonta, filed the suit in federal court on Monday and followed it with a motion for a temporary restraining order later that evening.

The lawsuit argues that combining two of Hollywood’s five major film distributors and two of its five major basic cable channel owners would substantially lessen competition across theatrical distribution, cable programming, and the broader entertainment industry. Bonta said in a statement that the merger would lead to higher prices, lower quality, and less content for audiences. The deal had already received clearance from the Justice Department’s Antitrust Division, which concluded in June that the transaction was unlikely to harm competition, making the state-level challenge a direct rebuke of the federal finding.

Kessler told CNBC’s David Faber that Paramount had indicated its intention to close the deal as early as July 22, the date by which the European Union is expected to issue its own regulatory decision. Paramount recently submitted concessions to the EU to address remaining concerns. Kessler said the company offered the states two alternatives, an immediate close or an orderly judicial schedule that would resolve the matter by early September, but the states rejected both.

The financial pressure on Paramount is real. Under the merger agreement, if the deal has not closed by September 30, Paramount must pay Warner Bros Discovery shareholders a ticking fee worth roughly $650 million per quarter until closing. A temporary restraining order, if granted, would pause the transaction for 14 days, and up to two could be issued before the states seek a preliminary injunction that would put the deal on hold for the duration of the litigation.

Kessler argued the merger is pro-competitive rather than anti-competitive, noting that the entertainment industry is in deep trouble as consumers flee pay TV bundles and streaming competition intensifies. He said the combined company would be able to compete directly with Netflix, Disney, and Amazon’s Prime Video. CEO David Ellison has promised the merged entity would release 30 films per year, and Kessler said Paramount is willing to put that commitment in writing.

The deal has already cleared the DOJ and multiple international regulators, and Paramount has been unifying its streaming technology in preparation for absorbing HBO Max after closing. Whether the state attorneys general can delay the transaction long enough to trigger the ticking fee, or block it entirely, will likely depend on how quickly the federal court in Sacramento acts on the restraining order request.

A pair of soft white tubes no thicker than a couple of strands of spaghetti rest in a researcher’s hands. They look almost fragile, yet these fibers can pull with the strength of real muscle, stay completely silent while they work, and run for hours on nothing more than a small battery pack. Researchers at the MIT Media Lab and Politecnico di Bari just published the full details of this system in Science Robotics, and the results feel like a genuine step change for anyone building humanoid robots or wearable machines.

Most robots are still powered by electric motors and gearboxes that simply spin a shaft and then convert that rotation into linear motion in the old-fashioned way, which works fine but has some significant drawbacks: it makes a racket, weighs a ton near the joints, and interferes with how our own limbs function. For years, soft fluidic actuators have been touted as a superior solution: long, flexible tubes that contract when pressed, similar to the muscles in your arm. The issue is with the massive pumps, compressors, and hoses that keep those actuators operating. They simply stop any possibility of a clean, portable design in its tracks.

Unitree R1 Humanoid Robot (White, R1 Air)

• Three models, one lightweight platform R1 Air (20 DOF, monocular camera), R1 (26 DOF, binocular camera, head+waist joints), and R1 Edu (26 DOF…
• Easy setup – no coding required for basic use Unbox, power on, and start. Manual teaching feature: physically pose the robot, and it replays the…
• More DOF = more expressive movement 26‑DOF models (R1 / R1 Edu) add head and waist articulation for smoother dance and running. For safety reasons…

The new electrofluidic fiber muscles solve that problem by actually inserting the pump inside the muscle. Each pump is a tiny tube that is less than 2 millimeters wide. Inside, two thin helical electrodes weave their way along the length, and when exposed to high voltage, they begin to inject charge into a unique insulating liquid known as a dielectric fluid. The charged particles just drag the entire fluid along with them, generating pressure and flow with no moving parts. The entire system is absolutely silent and converts electricity directly into hydraulic power.

These tiny pumps form a closed loop with some thin McKibben-style actuators, which are essentially soft tubes wrapped in a braided sleeve that contract when the fluid inside them expands. You may simply stack one pump between two opposing actuators, exactly like your biceps and triceps operate together. When the pump pushes fluid into one actuator, that side shortens while the other side lengthens. There is no need for an external reservoir, therefore the entire system remains sealed, lightweight, and self-contained.

The performance stats are impressive, with roughly 50 watts per kilogram of power density and fibers that can contract by 20% of their length. When multiple pumps are operated in simultaneously, response times drop to less than 0.3 seconds. They also have a pre-pressure system that keeps everything stable and doubles the stroke three times for the same pump effort, and with the bias pressure, they can exchange a little maximal force for even faster snaps when speed is more critical than sheer power.

To demonstrate how adaptable the design is, the team demonstrated no fewer than three different setups. One of them launches table tennis balls in less than 2 tenths of a second, which is very quick. Another bundles a bunch of fibers together so that a small package weighing only a few dozen grams can lift four kilograms, or 200 times its own weight, with a beautiful clean 30-millimeter stroke. The most friendly-looking demonstration incorporated the fibers into a flat biceps-triceps pair that bends a 3D-printed robot arm in a full 40-degree arc. That same knitted muscle is supple enough to shake someone’s hand without squishing their fingers or feeling stiff.
[Source]

Sony’s first-party fight stick was supposed to land in August.

Spotify is rolling out a new AI-powered conversational feature that lets Premium users talk directly to the app about what they want to hear. Users can type or speak a request and refine the results through follow-up questions instead of manually searching for a song, podcast, or audiobook.

The feature is available from Spotify’s Home and Now Playing screens and works much like a personal audio assistant. It can choose what plays, answer questions about the current track or album, recommend something new, and look through your listening history to provide more personalized responses.

What can you ask Spotify to do?

You can ask Spotify to play artists you have not heard before. Follow-up requests can add a particular artist, narrow the selection to recent releases, or make the music more upbeat. The assistant can also save a song, add it to your queue, or follow an artist. It can provide more information about whatever is currently playing. Users can ask when an album was released, what genre a song belongs to, or what inspired a particular record.

The feature also works across podcasts and audiobooks. You can ask Spotify to find more books by an author or pull up other podcast episodes featuring the same guest. It can also look back through your listening history. Spotify says you will be able to ask when you first played a particular song or which genres you have been listening to most recently.

This is not Spotify’s first AI-powered feature

Spotify has been experimenting with AI for a while now, and each feature has brought the technology into a different part of the service. AI DJ is one such feature that creates a personalized stream of music and uses an AI-generated voice to introduce songs and explain recommendations. AI Playlist lets users build playlists from written prompts based on a mood, activity, or genre.

Studio by Spotify Labs can generate personal podcasts and daily briefings shaped around a user’s listening history. Spotify has also announced a separate generative AI tool that will let Premium subscribers create licensed covers and remixes from songs by participating artists and songwriters.

The new conversational feature is now rolling out in beta to Premium users aged 18 and older in the US, Ireland, and Sweden. It is available in English through Spotify’s iOS and Android apps. Spotify says responses may not always be perfect while testing continues.