AI is changing the cybersecurity scenario at breakneck speed. In 2024 alone, 87% of organizations encountered AI-driven cyberattacks, and an overwhelming 91% expect these AI-enhanced threats to grow even more in the next few years. Yet defenders are struggling to keep up – only about a quarter of companies feel highly confident in their ability to detect these AI-powered attacks. 

This imbalance between offense and defense is alarming. It’s not just about statistics, either. At a recent RSA security conference, experts pulled back the curtain on a real “evil” AI chatbot that can find and exploit software vulnerabilities faster than traditional tools can react. The demonstration provided a sobering glimpse of how cybercriminals are weaponizing AI, and it underscored a critical point: we are entering an era where AI-enabled cyberattacks could outpace our best defenses.

In this post, we’ll explore the rise of these AI-driven threats, recount what happened at that RSA demo of a rogue AI called WormGPT, compare these malicious tools to the traditional security measures. 

Key Insights on Rogue AI Threats

Here are 4 key takeaways from this blog post on Rogue AI.

1. Rogue AI tools like WormGPT can find and exploit software flaws faster than traditional security tools, often bypassing ethical safeguards.
2. These tools are cheap, easily accessible, and require minimal skill to operate—lowering the barrier for cybercriminals to launch complex attacks.
3. Conventional scanners and ethical AI models often miss vulnerabilities that rogue AI detects and weaponizes with detailed attack instructions.
4. The cybersecurity gap is widening; defenders must rethink their strategies and consider using AI offensively to keep up with fast-evolving threats.

The Alarming Rise of AI-Driven Cyberattacks

Cybercriminals have begun using AI in ways that were unthinkable a few years ago. Generative AI can now automate hacking techniques, adapt malware, and craft convincing scam messages at scale. The result is a new breed of attacks that are more sophisticated, personalized, and harder to detect than ever before. In fact, 85% of cybersecurity professionals believe AI-driven attacks are more advanced and stealthy compared to traditional hacks. We’re seeing this play out in multiple domains of cybercrime. For example, AI chatbots can churn out highly persuasive phishing emails or fake business correspondence that even sharp-eyed employees might fall for. 

In one analysis, WormGPT (an AI model repurposed for crime) was asked to write a phony business email compromise scam, and “the results were unsettling” – the AI produced a remarkably strategic and convincing phishing lure. Attackers are also leveraging AI to conduct “multi-channel” schemes, mixing email, text messages, voice deepfakes, and more. In a real-world case, scammers combined an AI-generated voice call with messaging platforms to pose as a company CEO and successfully trick an employee into a fraudulent money transfer. These anecdotes highlight a troubling trend: AI is empowering criminals to broaden their attack surfaces and mimic legitimate behavior with chilling accuracy.

Behind the scenes, dark web forums have started offering custom DarkGPT AI tools built explicitly for malicious purposes. These go by names like WormGPT, FraudGPT, GhostGPT, and more – often borrowing the “GPT” moniker for familiarity. Unlike mainstream AI (such as ChatGPT, Copilot or Google Gemini) which have ethical guardrails, these rogue versions remove the safety limits. They are designed to answer any question or request – no matter how illegal or harmful. Ask them how to write ransomware or find a software bug to exploit, and they’ll comply without protest. 

This means that a would-be hacker with minimal skill can suddenly get step-by-step guidance from an AI “assistant” that has no conscience. It’s a paradigm shift in the threat landscape: sophisticated attacks no longer require sophisticated attackers. As one security researcher grimly observed, “AI is dramatically scaling the sophistication and personalization of cyberattacks”, even as businesses remain uncertain how to respond.

A Glimpse of ‘Evil AI’ at RSA 2025 – The WormGPT Demo

One of the clearest wake-up calls came at the RSA Conference in 2025, where security experts Sherri Davidoff and Matt Durrin of LMG Security demonstrated an “evil AI” in action. The atmosphere in the room was tense as they unveiled WormGPT, an AI chatbot purpose-built for hackers. WormGPT had been lurking in Telegram channels, dark web, and underground forums, advertised as a chatbot that “lets you do all sorts of illegal stuff … allowing anyone access to malicious activity without ever leaving the comfort of their home.” The LMG Security team managed to obtain a copy of WormGPT for as little as $50 via a hacker forum, and what they showed next made the audience’s blood run cold.

WormGPT’s official website bills it as an uncensored AI model designed to help with cybersecurity and hacking tasks. The RSA demo revealed just how potent this tool had become. Davidoff and Durrin walked the audience through a series of live tests, essentially a progression of WormGPT’s “evolution.”

First, they fed WormGPT the source code of an open-source project management app known to have a SQL injection flaw. This older 2024 version of the AI successfully identified the vulnerable code and even suggested a rudimentary exploit for it. The exploit attempt didn’t fully work – likely because that early WormGPT couldn’t ingest the entire codebase at once – but it proved the concept: the AI could spot a serious security weakness in real software.

Encouraged, the researchers ramped up the difficulty. They set up a test environment with the infamous Log4j vulnerability (a critical flaw that had made headlines) and asked a slightly newer version of WormGPT to find it. Again, the AI zeroed in on the remote code execution flaw in the server. This time it provided a bit more detail on how one might exploit it, though not quite enough for a novice to carry out easily. As Davidoff noted, “an intermediate hacker” could probably take WormGPT’s output and make it work – meaning the AI was getting closer to fully breaching a system on command. Still, there was a knowledge gap; the tool hadn’t yet handed over a complete, ready-to-use hack to an amateur.

That gap didn’t last long. The next iteration of WormGPT they tested gave detailed, explicit instructions on exploiting Log4j – even generating attack code that plugged in the target server’s actual IP address. In the live demo, those instructions worked flawlessly to compromise the test system. At this point, anyone in the room who had been skeptical fell silent. The AI had basically just pwned a server, providing the exploit code and steps on a silver platter. This was no longer theoretical – it was happening right in front of us, in seconds, at the push of a button.

For a final trial, the team presented WormGPT with a truly tough challenge: a vulnerable e-commerce application (Magento) with a hidden, multi-step security flaw. This was a known complex exploit that required chaining two separate vulnerabilities. The question: Could WormGPT find and explain the whole attack? Astonishingly, it did. The latest version of WormGPT rapidly pinpointed the weakness and outlined the two-part exploit needed to take over the Magento server.

Even more telling, the usual “good guy” tools failed to flag the issue. An open-source scanner (SonarQube) only reported an unrelated minor issue and ChatGPT didn’t catch the vulnerability at all. WormGPT not only found the critical flaw quickly, but it also went above and beyond – it volunteered a full breakdown of how to hack the server step by step, complete with explanations for each step. It even offered the final exploit code without being explicitly asked. In other words, this malicious AI acted like a knowledgeable mentor guiding a newcomer on exactly how to pull off a hack.

The audience reaction was a mix of awe and anxiety. As Davidoff candidly put it during the session, “I’m a little nervous to see where we’re going to be with hacker AI tools in another six months.” The progress within a year was stark – from partially helpful hints to fully working exploits on demand. Perhaps the most unsettling realization was voiced by the presenters: we are still “in the very early infant stages” of hacker AI.

If WormGPT could improve this dramatically in a matter of months, what happens as these tools continue to learn and evolve? It was a true “aha” moment for many in cybersecurity: a glimpse of a future where AIs built for offense may consistently outpace our ability to patch and secure systems. And chillingly, the bar for launching successful attacks is getting lower – now “as long as you have money for a subscription, you’re in the game.”

Rogue AI vs. Traditional Defenses: Why the Good Guys Are Trailing

This demonstration underscored a concerning asymmetry between rogue AI tools and traditional cybersecurity defenses. On one hand, we have AI systems like WormGPT that are purpose-built to find weaknesses and exploit them with ruthless efficiency. On the other hand, most organizations still rely on conventional tools – and constrained AI models – that simply can’t match that ferocity.

Consider the Magento test: an automated code scanner (SonarQube) and even an advanced general AI (ChatGPT) both missed a critical vulnerability that a specialized evil AI caught and weaponized. Traditional scanners often work by checking code against known patterns or signatures of vulnerabilities. They can flag common mistakes (and indeed SonarQube did flag something, just not the right issue), but they might fail to connect the dots on more complex or novel exploit chains.

ChatGPT, meanwhile, has knowledge limited to its training data and is bound by ethical guidelines – it won’t freely describe how to hack a system, and it may not have full context of a codebase if the project is large. In contrast, WormGPT’s newest version could ingest unlimited amounts of code and had no qualms about churning out an attack plan. Essentially, the bad actors have an AI with a laser focus on breaking things, whereas the good guys’ tools either play by the rules or lack the same depth of data access.

Ethics and mindset also play a role in this divide. Mainstream AI developers (and security teams) operate with a focus on safety, legality, and positive use-cases – as they should. But that mindset can be a handicap when imagining how to combat the worst-case scenarios. Davidoff pointed out that while the industry talks about using AI for the “betterment of society,” we also need to urgently think about using AI to guard against the worst of humanity. The reality is that attackers are not holding back. They are exploring every avenue that AI opens up, from automating password cracking to generating polymorphic malware that evades antivirus detection. Meanwhile, defenders often impose limits on AI (to avoid false positives, prevent misuse, etc.) or hesitate to deploy equally aggressive AI for defense due to ethical and legal concerns. This creates a dangerous imbalance: the dark side of AI innovation is sprinting ahead, while the defensive side moves more cautiously.

Another stark difference is accessibility. Tools like WormGPT are being hawked in semi-open forums and chat channels. They’re cheap and readily available to anyone with a few crypto coins or dollars to spare (earlier versions were sold on hacker forums for a few hundred euros, and later even via Telegram for around $50). This means a low-skill attacker doesn’t need to develop expertise – they can simply rent an evil AI. In effect, AI is lowering the entry barrier for cybercrime, doing much of the “heavy lifting” that used to require specialized knowledge. By contrast, high-end defensive AI solutions (like advanced threat detection systems) can be expensive, complex to deploy, and typically require skilled analysts to interpret their output. Small businesses and individuals don’t have easy access to cutting-edge defensive AI, but an aspiring hacker can cheaply get an AI mentor for offense. It’s a sobering “democratization” of cyberattack capabilities.

Is there any silver lining here? Possibly. The presenters at RSA noted that those operating in the shadows often don’t scrutinize who else might be lurking there with them. In other words, security researchers can infiltrate and study these rogue AI tools just as the LMG team did, helping the “good guys” learn how they work. By analyzing hacker AI, defenders might develop countermeasures or even use similar AI to probe their own systems before the bad actors do. In fact, AI can be a powerful ally for defense as well – many organizations are starting to use AI for threat detection, pattern recognition, and automated incident response. The key is that defenders must be just as innovative and aggressive in applying AI for protection, while still upholding ethical use. The window to act is now, because the longer we wait, the further behind we could fall in this AI-fueled cat-and-mouse game.

To Sum Up

The emergence of “evil” AI tools like WormGPT is a stark reminder that cybersecurity is a constantly moving target. Threat actors are enthusiastically embracing AI to supercharge their attacks – from finding hidden software bugs to crafting scams that sound eerily human. It is as scary as it sounds, but it’s not a hopeless situation. Awareness is the first line of defense: by understanding how these rogue AIs work and what they’re capable of, we can better prepare ourselves. The cybersecurity community is already taking action, pushing for AI-assisted code auditing and threat detection to fight fire with fire. As individuals, adopting strong security habits and tools will go a long way toward minimizing the “splash damage” when breaches occur.

Ultimately, the race between attackers and defenders is entering a new phase – one where algorithms duel in cyberspace. To keep pace, organizations and users alike will need to be proactive and agile. 

We must ask hard questions, like the one posed at RSA: “What if hackers use evil AI to find vulnerabilities before we can fix them?” – and then work urgently on the answers. By coupling human expertise with smart technology (and yes, even AI on the defense side), we can still tip the balance in our favor. The key is not to be complacent. The scenario may be daunting, but with vigilance, education, and the right safeguards, we can make it much harder for the “evil” AI out there to have its way. In the face of rapidly evolving cyber threats, staying informed and prepared is our best collective defense.

Sources: Recent cybersecurity reports and conference demonstrations have informed this discussion. Notably, statistics on the prevalence of AI-driven attacks and organizational preparedness come from SoSafe’s 2025 global survey. Insights into WormGPT and the RSA conference demo are drawn from PCWorld’s first-hand feature by Alaina Yee, as well as reporting by security researcher Brian Krebs on the origins of WormGPT. These sources underline the reality of AI’s double-edged sword in cybersecurity – and why taking protective action is so critical.