Malicious Underground AI Models Like WormGPT 4 Are Supercharging Cybercrime in 2024
Executive Summary
In early 2024, cybersecurity researchers uncovered an expanding underground marketplace for custom large language models (LLMs) such as WormGPT 4 and KawaiiGPT, designed to facilitate cybercrime. These jailbroken and open-source models, advertised and sold across dark web forums, lower the technical barrier for attackers by offering tools to scan for vulnerabilities, automate malware development, and accelerate tasks like phishing and lateral movement. Their accessibility—with minimal setup time, user-friendly interfaces, and affordable pricing—has enabled a broader range of cybercriminals to automate sophisticated attacks previously requiring advanced skills.
The emergence of malicious LLMs highlights a growing trend where generative AI is weaponized in cybercrime. Unlike earlier incidents, these tools are now commercialized and widely supported, signaling a shift from simple model jailbreaking to specialized AI-enabled attack platforms. This evolution increases the urgency for organizations to strengthen AI risk management, augment detection strategies, and adapt compliance controls to address the new threat landscape.
Why This Matters Now
Malicious LLMs like WormGPT 4 and KawaiiGPT are actively lowering barriers for cybercriminals, making sophisticated cyberattacks cheaper, faster, and more accessible to non-expert threat actors. Their commercial availability and growing underground support communities underscore an urgent need for organizations to evaluate their security posture against AI-driven threats and adapt quickly.
Attack Path Analysis
Attackers leveraged underground AI-powered tools like WormGPT and KawaiiGPT to assist in initial network reconnaissance and generating malicious scripts, enabling compromise of exposed or misconfigured cloud assets. Subsequently, automated LLM-driven scripts facilitated privilege escalation by exploiting weak IAM roles or misconfigurations. Once access was gained, adversaries moved laterally between cloud workloads or across Kubernetes clusters using AI-generated traversal techniques. Attackers established command & control via covert outbound channels, possibly hiding activity within encrypted traffic or leveraging common remote access methods. Exfiltration of sensitive data or credentials was assisted by AI toolkits that generated scripts for data collection, packaging, and exfil. Finally, impact was realized by deploying ransomware, deleting resources, or causing operational disruption via automated, AI-orchestrated malware or scripts.
Kill Chain Progression
Initial Compromise
Description
AI-assisted attackers used malicious LLMs to scan for vulnerable cloud endpoints or misconfigurations and generate tailored phishing or exploit scripts, enabling unauthorized access.
Related CVEs
CVE-2024-27564
CVSS 7.5A server-side request forgery (SSRF) vulnerability in ChatGPT's pictureproxy.php allows attackers to inject malicious URLs, leading the application to make unintended requests.
Affected Products:
OpenAI ChatGPT – commit f9f4bbc
Exploit Status:
exploited in the wild
MITRE ATT&CK® Techniques
Develop Capabilities
Compromise Accounts
Command and Scripting Interpreter
User Execution
Phishing
Valid Accounts
Automated Exfiltration
Obfuscated Files or Information
Potential Compliance Exposure
Mapping incident impact across multiple compliance frameworks.
PCI DSS 4.0 – Security Awareness Training
Control ID: 12.6.1
NYDFS 23 NYCRR 500 – Cybersecurity Policy
Control ID: 500.03
DORA (Digital Operational Resilience Act) – ICT Security Awareness and Training
Control ID: Article 9(2)(e)
NIS2 Directive – Incident Prevention, Detection and Response Measures
Control ID: Article 21(2)(d)
CISA ZTMM 2.0 – Monitoring of Threat Activities
Control ID: Security Operations – Detect & Respond
Sector Implications
Industry-specific impact of the vulnerabilities, including operational, regulatory, and cloud security risks.
Computer/Network Security
Underground AI hacking tools like WormGPT directly target cybersecurity infrastructure, lowering attack barriers and enabling automated vulnerability exploitation against security frameworks.
Financial Services
AI-powered cybercriminal tools threaten encrypted transactions and east-west traffic security, potentially compromising NIST and PCI compliance requirements for financial institutions.
Health Care / Life Sciences
Malicious LLMs pose significant risks to HIPAA-compliant data protection systems, threatening patient data through automated phishing and lateral movement capabilities.
Information Technology/IT
Commercial AI hacking subscriptions enable automated exploitation of cloud infrastructure, Kubernetes environments, and hybrid connectivity systems across IT service providers.
Sources
- Underground AI models promise to be hackers ‘cyber pentesting waifu’https://cyberscoop.com/malicious-llm-tools-cybercrime-wormgpt-kawaiigpt/Verified
- Malicious LLMs are letting even unskilled hackers to craft dangerous new malwarehttps://www.techradar.com/pro/security/malicious-llms-are-letting-even-unskilled-hackers-to-craft-dangerous-new-malwareVerified
- WormGPT 4 and KawaiiGPT: New Dark LLMs Boost Cybercrime Automationhttps://www.securityweek.com/wormgpt-4-and-kawaiigpt-new-dark-llms-boost-cybercrime-automation/Verified
- Hackers Offer Lifetime Access to WormGPT and KawaiiGPT for $220https://cyberpress.org/access-to-wormgpt/Verified
Frequently Asked Questions
Cloud Native Security Fabric Mitigations and ControlsCNSF
Applying Zero Trust segmentation, east-west traffic controls, egress policy enforcement, and real-time anomaly detection would have significantly limited or detected attacker movement across the kill chain. CNSF and Aviatrix fabric capabilities can disrupt automated attack scripts by enforcing least privilege, monitoring internal flows, and preventing unauthorized data exfiltration.
Control: Cloud Firewall (ACF)
Mitigation: Prevents unauthorized access to exposed endpoints.
Control: Zero Trust Segmentation
Mitigation: Limits blast radius by restricting cross-segment privilege upgrades.
Control: East-West Traffic Security
Mitigation: Detects and blocks unauthorized east-west traffic.
Control: Egress Security & Policy Enforcement
Mitigation: Stops or flags unauthorized outbound connections.
Control: Encrypted Traffic (HPE)
Mitigation: Detects and prevents unauthorized data in transit.
Detects and responds to malicious or destructive activity.
Impact at a Glance
Affected Business Functions
- Email Communications
- Data Security
- Network Operations
Estimated downtime: 3 days
Estimated loss: $500,000
Potential exposure of sensitive customer data due to phishing attacks and malware generated by malicious LLMs.
Recommended Actions
Key Takeaways & Next Steps
- • Enforce cloud-native segmentation and compartmentalization of workloads to minimize lateral attacker movement.
- • Apply centralized, fine-grained egress filtering to block unauthorized outbound and C2 traffic from workloads or Kubernetes clusters.
- • Monitor and baseline internal east-west traffic flows to rapidly detect anomalous or AI-driven lateral activity.
- • Implement strong, least-privilege IAM policies and automate identity enforcement to reduce privilege escalation risk.
- • Continuously inspect encrypted and unencrypted traffic for signs of exfiltration or ransomware, leveraging real-time threat detection and automated response.
