Executive Summary
In February 2026, Praetorian released Augustus, an open-source vulnerability scanner designed to test Large Language Models (LLMs) against a comprehensive suite of adversarial attacks. Augustus automates over 210 distinct attack vectors, including prompt injections and jailbreaks, across 28 LLM providers. This tool addresses the growing need for robust security testing as enterprises rapidly integrate generative AI into their products. By providing a portable, single-binary solution, Augustus facilitates seamless integration into continuous integration/continuous deployment (CI/CD) pipelines, enabling security teams to identify and mitigate vulnerabilities efficiently.
The release of Augustus underscores the escalating threats targeting LLMs, as adversaries increasingly exploit these models for malicious purposes. The tool's comprehensive testing capabilities highlight the necessity for organizations to proactively assess and fortify their AI systems against evolving attack methodologies.
Why This Matters Now
As enterprises accelerate the adoption of generative AI technologies, the release of Augustus highlights the urgent need for robust security measures to protect Large Language Models (LLMs) from sophisticated adversarial attacks. This tool provides organizations with the capability to proactively identify and mitigate vulnerabilities, ensuring the integrity and reliability of AI-driven applications in an increasingly threat-laden landscape.
Attack Path Analysis
The attacker initiated a multi-turn conversation with the LLM, gradually steering it towards generating prohibited content. By exploiting the model's conversational context, the attacker bypassed initial safeguards, leading to unauthorized information disclosure. This manipulation allowed the attacker to extract sensitive data, culminating in a significant security breach.
Kill Chain Progression
Initial Compromise
Description
The attacker engaged the LLM in a benign conversation, establishing trust and setting the stage for manipulation.
MITRE ATT&CK® Techniques
Data Manipulation: Stored Data Manipulation
Modify Authentication Process: Pluggable Authentication Modules
Application Layer Protocol: Web Protocols
Exploitation for Client Execution
Command and Scripting Interpreter: PowerShell
Valid Accounts
Obfuscated Files or Information
Input Capture: Keylogging
Potential Compliance Exposure
Mapping incident impact across multiple compliance frameworks.
PCI DSS 4.0 – Ensure that security policies and operational procedures for developing and maintaining secure systems and software are documented, in use, and known to all affected parties.
Control ID: 6.4.3
NYDFS 23 NYCRR 500 – Cybersecurity Policy
Control ID: 500.03
DORA – ICT Risk Management Framework
Control ID: Article 5
CISA ZTMM 2.0 – Identity and Access Management
Control ID: 3.1
NIS2 Directive – Cybersecurity Risk Management Measures
Control ID: Article 21
Sector Implications
Industry-specific impact of the vulnerabilities, including operational, regulatory, and cloud security risks.
Computer Software/Engineering
LLM multi-turn attack tools like Augustus v0.0.9 directly threaten AI/ML security infrastructure, requiring enhanced guardrails against conversational jailbreaks and prompt injection techniques.
Financial Services
Multi-turn LLM attacks bypass single-turn guardrails in customer service chatbots, risking unauthorized data extraction and compliance violations across banking applications.
Health Care / Life Sciences
Healthcare AI systems face elevated risks from conversational attack strategies that gradually extract protected health information through seemingly innocent multi-turn interactions.
Computer/Network Security
Security professionals must adapt detection capabilities for sophisticated multi-turn LLM attacks that use backtracking and persona-based approaches to evade traditional guardrails.
Sources
- Augustus v0.0.9: Multi-Turn Attacks for LLMs That Fight Backhttps://www.praetorian.com/blog/llm-multi-turn-attacks-augustus/Verified
- Augustus – Open-source LLM Vulnerability Scanner With 210+ Attacks Across 28 LLM Providershttps://www.cryptika.com/augustus-open-source-llm-vulnerability-scanner-with-210-attacks-across-28-llm-providers/Verified
- We Built an Open-Source Tool to Attack-Test LLMs. Here’s What We Found.https://medium.com/@praetorianguard/we-built-an-open-source-tool-to-attack-test-llms-heres-what-we-found-e47b8521cad9Verified
Frequently Asked Questions
Cloud Native Security Fabric Mitigations and ControlsCNSF
Aviatrix Zero Trust CNSF is pertinent to this incident as it could likely limit the attacker's ability to manipulate the LLM and extract sensitive information by enforcing strict segmentation and identity-aware policies.
Control: Cloud Native Security Fabric (CNSF)
Mitigation: The attacker's ability to establish initial trust with the LLM could likely be constrained, reducing the risk of successful manipulation.
Control: Zero Trust Segmentation
Mitigation: The attacker's ability to escalate privileges by manipulating the LLM's context could likely be limited, reducing the risk of bypassing content filters.
Control: East-West Traffic Security
Mitigation: The attacker's ability to move laterally and access sensitive information across different domains could likely be constrained, reducing the risk of unauthorized data access.
Control: Multicloud Visibility & Control
Mitigation: The attacker's ability to maintain control over the LLM's responses could likely be limited, reducing the risk of achieving unauthorized outputs.
Control: Egress Security & Policy Enforcement
Mitigation: The attacker's ability to exfiltrate sensitive information could likely be constrained, reducing the risk of data breaches.
The potential reputational damage and compliance violations resulting from unauthorized data disclosure could likely be mitigated, reducing the overall impact of the incident.
Impact at a Glance
Affected Business Functions
- AI Model Security
- Application Security
- Incident Response
Estimated downtime: N/A
Estimated loss: N/A
Potential exposure of sensitive AI model behaviors and vulnerabilities.
Recommended Actions
Key Takeaways & Next Steps
- • Implement Zero Trust Segmentation to restrict LLM interactions based on user identity and context.
- • Enhance Threat Detection & Anomaly Response mechanisms to identify and respond to unusual conversational patterns.
- • Apply Egress Security & Policy Enforcement to monitor and control the flow of sensitive information.
- • Utilize Multicloud Visibility & Control to gain comprehensive insights into LLM interactions across platforms.
- • Regularly update and enforce content moderation policies to prevent exploitation of LLMs through multi-turn conversations.
