With the rise of AI pentesting, a lot of people are wondering if it’s as good as a human. Having started as a pentester twelve years ago, and having moved through similar roles to working on Intruder’s AI pentester,

I’ve come to understand the advantages and disadvantages of both. Here’s how they compare.

The strict methodology was never the good bit

You may expect AI pentesters to follow strict methodologies in the way that many human pentests do.

At Intruder, we experimented with this approach early on – structured systems where each agent had a specific vulnerability type to look for. What we found was that agents would miss things outside their remit entirely, or miscategorize what they did find as the one thing they’d been told to look for, causing confusing reports.

A complete lack of structure also failed to produce good results, as completely free agents without any guidance couldn’t break down and manage large applications properly. They could find interesting things, but at the cost of missing a load of other vulnerabilities in our targets.

We’ve put a lot of testing into finding the balance between these approaches, and settled on giving agents freedom with broadly defined roles to help them approach and manage large applications, and stricter guardrails to stop them causing damage. 

This does mean that our AI pentester doesn’t tick off the parts of a methodology document that you’d typically see in a human pentest. That may make some people uncomfortable – a strict methodology gives people assurance that someone has gone through your application and tested everything on the list. An unauthenticated SQL injection needs to be found, and a checklist is there to make sure it does.

The methodology gives a lot of people the idea that a human pentest is a perfectly repeatable product, as any tester will be able to follow the methodology, and everything important will get found. The reality is a lot more nuanced than that though, as pentesting is a skilled job, and many of the most impactful findings on a pentest will come from the tester being experienced in noticing weird behaviours and following up on them.

For our AI pentester, we ended up with much more interesting and impactful results by not trying to guide our agents through specific vulnerability types. A good example is a complex authentication bypass the agents found that had been sitting in a codebase through five years of pentests without any human testers picking it up.

The mechanics of that issue mean that you’d never think to write a testing guide that could be strictly followed to identify it, and if we’d tried to limit our AI pentester too much it never would have found it either.

You could say that only finding the common things would still be pretty good, but there’s such an array of vulnerabilities that aren’t common that you find during a pentest. And just because they’re not common, it doesn’t mean that an attacker can’t find them too, especially now AI is being used to enable attacks more.

A strict methodology being carried out by a competent tester only ever provided a low baseline of reliability, and it didn’t even do that perfectly. What makes a pentest good is the part that a skilled tester does outside the checklist, and that’s where AI thrives – following its nose through your application and finding the things that were never going to make a list.

What they know, and what they can’t ask

Our AI pentests take a white box approach, which means the agents get the source code. They work through the entire codebase and build a picture of not just what every page is, but how the application is meant to function and what requests can be made. Doing this to the level of detail that the AI is able to isn’t feasible for a human pentester during a typical time window for a pentest.

The AI pentester also builds up a sense of what the application is for, and a good example of this is access control – it doesn’t need to be told that low privileged users shouldn’t be able to reach admin functionality, or that a particular bit of data is sensitive.

You can give them extra information as well, just like a human tester. For example, people often specify that they’re worried about their customers being able to access each others’ data, and the pentester has done a great job of focusing on these issues and identifying a lot of them.

What the agents can’t do is ask you for anything you didn’t already give or tell them. If I don’t understand what a part of an application does while I’m doing a pentest, I can ask my contact, and those conversations can be very useful for testing the application properly, and assessing risk in the final report.

I can also clarify the scope with my contact if needed, or ask for permission to verify something when I’m unsure if there’s a risk of disruption.

We’ve had to deliberately make our AI pentesting agents cautious on this front to avoid going out of scope.

Neither finds everything

The freedom that lets AI pentesting agents find something like a complex auth bypass is the same thing that makes them less predictable. They decide where to dig, and that won’t be identical every time.

But pentest results have always varied with the person doing them. It’s why many consultancies will rotate your tester each year – different people find different things on the same application.

This comes down to people’s experience, specialisms, and incidental knowledge.

I’ve found issues because of a blog post or specification I happened to read years earlier and was left sitting in the back of my head. And even the things on a checklist get missed sometimes. People are fallible, and covering an entire application methodically over multiple days can be hard.

Consistent coverage is something we’re actively working to improve, but it’s not something that was ever offered perfectly from anyone. It’s why every human pentest report comes with a disclaimer saying as much, even though most people read it as a formality.

The remediation I couldn’t have written

Every pentester has strong opinions about what makes a good report. I’ve built up many of my own over the years, as have the rest of the current and former pentesters at Intruder. These opinions are what we built our report around.

The executive summary is the hardest page of a report to get right. For a lot of people it’s their entire experience of your work, and at a good consultancy a lot of effort goes into it. We’ve worked to get to the point where the agents are now writing genuinely good ones, and I often read them first on our own reports to get a good overview before diving into detail.

Where they’re still off is scoring risk. They rate things higher than I would, partly because they can’t have the conversations with you that would bring a rating down, and partly because LLMs err towards caution when you start talking about cybersecurity.

That’s the right direction to be wrong in, but it does mean you’ll see things flagged as more serious than they are, and it’s something we’re working on.

Remediation guidance is where the agents excel. After a long week on a pentest it’s tempting for a human to write generic advice, and you catch yourself and try to do better. Unfortunately, you’re usually limited by time and usually by not having the source code.

The agents don’t have either problem, so they’ll tell you the line of code to change, and what to change it to on every issue. They also write about mitigating controls that can help reduce the impact of the issue, or provide a temporary fix.

We’re seeing that speed up how quickly people actually fix things. We had a customer with 78 issues who fixed all of them within a few days, which as a human pentester I have never seen happen. I couldn’t have written the level of detail that made those fixes possible in the time I’d have had.

So is AI pentesting as good as a human?

AI pentesting and human pentesting are different, and once you stop trying to rank them the comparison gets a lot more useful.

The agents are finding serious issues in applications that human testers have already been through multiple times. They’ll read a whole codebase, work through every path in it, and write up every instance of what they find with the fix down to the line of code.

And you can run one whenever you want. For what a single human engagement costs you can run four AI pentests, which means you can do deep testing on a regular basis rather than once a year.

That’s a key thing that is missed by human pentesting, and often applications have undergone a lot of untested changes in between pentests.

See what Intruder’s AI pentester finds in your application. Learn more. 

Author

Written by Daniel Thatcher, Security Research Engineer, Intruder

Daniel is a security researcher at Intruder with a background in penetration testing, and a focus on exploiting web applications in black-box scenarios.

Sponsored and written by Intruder.

Devyn Johnston has spent years chasing smaller and smaller gaming machines, and this latest effort pushes that habit into territory that feels almost stubborn. Inside the ZCOOI Mugitya FE9, an RTX 5090 Founders Edition sits in a chassis that measures only 56 mm thick across the top and 79 mm at the base, all while still packing a full desktop processor and a 1000-watt power supply. The case arrives as a set of thick CNC-machined aluminum pieces, each finished in black or silver anodizing with a few gray-gold accents on the nameplate and front panel, and the whole collection ships inside its own aluminum carrying case.

Assembly begins by removing the back panel, front cover, and bottom portion, which are all held together with run-of-the-mill screws, so you can just put in the mini-ITX motherboard. Johnston chose an ASUS ROG Strix X870I for this build, which included Wi-Fi 7, a 2.5 gigabit Ethernet port, and a couple of USB 4 ports. He then topped it off with a 64GB of DDR5-6000 RAM, split evenly between the two memory slots, and paired a 2TB Samsung 990 Pro with a second 2TB 980 Pro for a total storage space of 4TB. The power supply is the Corsair SF1000, which is stealthily installed on the bottom panel, and every single cable has been cut to a custom length to ensure that nothing takes up room inside the box.

Acer Nitro V 16S AI Gaming Laptop | AMD Ryzen 7 260 Processor | NVIDIA GeForce RTX 5060 Laptop GPU…

• AI-Powered Performance: The AMD Ryzen 7 260 CPU powers the Nitro V 16S, offering up to 38 AI Overall TOPS to deliver cutting-edge performance for…
• Game Changer: Powered by NVIDIA Blackwell architecture, GeForce RTX 5060 Laptop GPU unlocks the game changing realism of full ray tracing. Equipped…
• Supreme Speed. Superior Visuals. Powered by AI: DLSS is a revolutionary suite of neural rendering technologies that uses AI to boost FPS, reduce…

A Thermalright AXP90-X47 low-profile all copper unit provides cooling and stands only 47 mm tall, comfortably fitting within the 49mm clearance on AMD boards. Johnston liked the orange fan so much that he replaced it with a black Noctua model, which allows you to see the blur of the fan spinning from the side panel without detracting from the overall beauty of the build. The graphics card is attached using a solid PCI-E riser rather than a fragile cable, and to get it to fit properly, Johnston removed the original bracket and bolted it directly to the chassis, allowing the coolers to rest behind matching holes in the panels.

One minor detail turned out to be the standout. The power button is based on a conventional keyboard key, and the case has four removable keycaps in black, white, transparent, and semi-transparent finishes, allowing you to customize it to suit your mood. Once everything is in place, the chassis weighs 6.5 kg and is supported by one of two available foot types, a narrow bar or a broader set for added stability. Magnets hold the side panels and coverings in place, allowing you to open the item with little more than the included aluminum screwdriver.

He put the finished machine through its paces, running The Witcher 3 Remastered at a smooth 120 frames per second on extreme with ray tracing and DLSS Quality turned on. The graphics card maintained a consistent 230-240 watts while staying cool at 54 °C, but the 7800X3D drew 53 to 60 watts and peaked at 70°C. Resident Evil Requiem drove the GPU to around 500 watts and 71-74 °C, but it still managed to maintain frame rates around 130. Cyberpunk at a resolution of 5120 by 2160 with strong ray tracing pushed the GPU to roughly 500 watts and 71-72 °C in the middle of the action, while the CPU reached nearly 80 °C under load. Downsizing to 1440p resulted in the GPU maintaining a very respectable 430-450 watts and 67 °C. The only issue occurred during the initial shader compilation, when the CPU soared beyond 95 °C, but as the game stabilized, the temperature remained within the range of what a low-profile cooler can tolerate.

When the system is up and operating, noise levels average around 43 decibels, with an occasional electrical hum from the power source that refuses to go away completely. The design does not allow for additional case fans to be installed, allowing the graphics card’s own cooler and copper heatsink to handle the work. He also has to consider that this case was designed specifically to house the Founders Edition 5090 (as well as the 5080 FE or PRO 6000), so swapping out the graphics card later for something new would necessitate tearing everything down and rebuilding, or ditching the case entirely. Johnston appears to consider this setup as a beautiful piece of technology to admire rather than a tool to be used on a daily basis. Yes, it achieves high frame rates while taking up less area than a typical tower.

A complete system would cost roughly $465 for just the case, before you add in all of the additional components, and the issue is, every component must fit perfectly or the entire thing will not close. When everything is fully put together and the last panel is clicked into place, you have a slab of aluminum that houses a full 5090 and an 8 core processor, all held together by magnetic panels and requiring only one keyboard key to boot up the system.

Besides running controllers with outdated firmware, the affected stores likely also failed to put the devices behind a VPN and create a restricted list of IP addresses allowed to log in. These protections are standard but often not followed. In this case, that failure enabled a mass hack that likely cost tens or hundreds of millions of euros. Spain, Portugal, Poland, Germany, and Belgium were also reportedly affected.

“Two things would have changed the outcome,” researchers at Pen Test Partners said Wednesday. “Cyber hygiene would have prevented the access: take the controller off the public internet, put remote access behind a VPN and an IP allowlist, bring third-party kit onto the network in a controlled, owned and documented way rather than leaving it on a contractor’s own connection, change known and default credentials, and patch to a current release.”

Researchers from security firm Claroty warned of the threat of ignoring these common-sense steps last October. The US Cybersecurity and Infrastructure Security Agency did the same in February.

In an email, Copeland acknowledged a vulnerability “identified in a discontinued, end-of-life generation of controllers” and said the company is working with customers to prevent similar incidents in the future. Danfoss representatives didn’t immediately respond to questions. CNN said the company also cited discontinued models.

So far, the party behind the attack has remained silent. That raises the possibility that the motive was vandalism on a mass scale. Another possibility is a nation-state or other international actor aiming to create a continent-wide disruption.

The incident underscores the tension between the financial and security departments in most businesses. Some companies are loath to spend large sums of money on new equipment when there are no tangible benefits or savings. Only after a devastating hack do they realize why the investment was worthwhile. Executives at affected supermarket chains will likely learn the hard lessons. Less clear is whether the cautionary tale will sway other businesses.

Enterprise investment in artificial intelligence has, understandably, been concentrated on compute. Access to GPUs, model performance and the cost of inference have become central considerations as organizations move AI workloads from experimentation towards production.

However, an increasing proportion of the economic and operational challenge sits elsewhere in the infrastructure stack. AI depends on data at every stage of its lifecycle: datasets must be collected and prepared, training data must be made available to compute, checkpoints and model artefacts must be retained and information may need to move between infrastructure providers as workloads and commercial priorities change.