Artificial Intelligence is transforming cybersecurity, introducing both exciting opportunities and new risks. As large language models (LLMs) like Llama 3 gain more powerful capabilities, their potential to either aid defenders or become tools for attackers has become a critical area of study. CYBERSECEVAL 3, developed by Meta AI, steps in as a robust benchmark to evaluate these risks. For cybersecurity officers, enthusiasts, and developers working with AI, understanding this benchmark provides essential insights into how LLMs influence the evolving security landscape.

 What is CYBERSECEVAL 3?  

CYBERSECEVAL 3 is the latest version of Meta’s security evaluation framework, building on CYBERSECEVAL 1 and 2. It assesses eight key cybersecurity risks across two broad categories:  

1. Third-party risks: Automated phishing, scaling offensive cyber operations, and autonomous hacking.  
2. Application risks: Prompt injection, insecure code generation, and misuse of LLMs for malicious purposes.

By testing LLMs like Llama 3 405B, 70B, and 8B, CYBERSECEVAL 3 helps contextualize their strengths and weaknesses, providing practical guidance for deploying these models responsibly.

 Key Findings from the Study  

1. AI-Driven Phishing Campaigns

Llama 3 405B proved capable of simulating multi-turn phishing attacks, using detailed victim profiles to increase the persuasiveness of its interactions. However, it performed on par with GPT-4 Turbo and Qwen 2-72B.

• Mitigation: The introduction of LlamaGuard ensures that cloud-deployed LLMs can monitor and block suspicious requests, reducing the misuse of AI for phishing attempts.

2. Scaling Manual Cyber Operations

In a capture-the-flag (CTF) simulation, Llama 3 405B was used to assist both expert and novice participants in solving hacking challenges.

• Results: The use of the LLM did not significantly enhance the completion rates or speed compared to traditional search engines.  
• Conclusion: While LLMs can assist in planning phases like network reconnaissance, they do not drastically improve offensive capabilities.

3. Autonomous Offensive Cyber Operations  

Llama 3 models were tested in a controlled environment, simulating ransomware attacks across phases like reconnaissance, exploitation, and persistence.

• Outcome: The models struggled with complex tasks like exploitation and maintaining network access, indicating limited autonomous hacking capabilities at this stage.

 Insecure Code Generation and Prompt Injection Risks  

    1. Insecure Code Generation  

When LLMs were evaluated for code completion tasks, 31% of the outputs contained vulnerabilities, such as SQL injection risks or buffer overflows.  

Solution: Meta released CodeShield, a tool that scans LLM-generated code to block insecure suggestions before they are implemented in production.

2. Prompt Injection Attacks

Prompt injections occur when attackers manipulate LLMs to bypass safety mechanisms and generate harmful responses. Llama 3 models exhibited some vulnerabilities in these scenarios.

Mitigation: PromptGuard helps detect and block these attacks, preventing harmful prompts from influencing the model’s behavior.

 What CYBERSECEVAL 3 Means for Cybersecurity Professionals  

The insights provided by CYBERSECEVAL 3 are invaluable for aspiring cybersecurity officers and professionals working at the intersection of AI and security. Here are the key takeaways:

1. 1. Master AI Tools for Cybersecurity: While LLMs can’t replace human expertise, they assist in areas like reconnaissance, coding, and planning. Professionals should learn to integrate these tools into their workflows.
2. Implement Guardrails: Tools like LlamaGuard and CodeShield are essential to secure AI systems. Understanding how to apply them will be crucial for those deploying LLMs.
3. Stay Updated on AI Regulations: Governments and organizations are increasingly concerned about AI misuse. Tracking policy developments will keep professionals aligned with industry best practices.

Key Areas to Explore Further

1. Detailed Phishing Simulation Insights:
   • The study mentions a manual and automated grading system to assess how persuasive LLM-generated phishing attempts were.
   • Llama 3 was tested against peers like GPT-4 Turbo and Qwen 2-72B Instruct, and the risk level was comparable across these models.
   • Human evaluations also validated the automated grading scores, showing a high correlation between human and machine judgments.
2. Limitations of LLMs in Offensive Operations:
   • Novices showed slight improvements with LLMs (faster completion by about 9 minutes per phase), but experts performed worse with AI assistance, potentially due to inefficiencies caused by the model’s suggestions.
   • Key insight: LLMs did not significantly impact outcomes compared to search engines, reinforcing the need for human expertise in real-world cyber operations.
3. Advanced Risks with Code Interpreters:
   • The study highlighted a growing trend of allowing LLMs to execute code in sandboxed environments (e.g., Python). This capability introduces risks of privilege escalation and container escape attacks.
   • LlamaGuard was found to effectively block malicious code execution requests, reducing attack success rates to near zero.
4. Prompt Injection Insights:
   • The success rate for prompt injection attacks was 20-40%, which aligns with other state-of-the-art models.
   • Multilingual prompt injection risks were observed, as non-English attacks had higher success rates, indicating a need for language-specific safeguards.
5. Real-World Scenarios:
   • LLMs were evaluated using “capture-the-flag” scenarios hosted on Hack The Box. Participants were given Linux machines to practice hacking tasks like network reconnaissance, vulnerability identification, and exploitation.
   • Interesting Feedback from Experts: Some found the LLM more distracting than helpful, indicating that AI can sometimes slow down processes by offering too much unnecessary information.
6. Public Tools and Transparency:
   • Meta released all non-manual elements of the CYBERSECEVAL 3 framework to encourage community collaboration and improvement. Tools like PromptGuard and CodeShield are now publicly available to support secure AI deployment.
   • Meta encourages continuous risk evaluation by releasing these benchmarks on GitHub, promoting ongoing monitoring of new AI models.

To Sum Up

CYBERSECEVAL 3 offers a comprehensive evaluation of how LLMs influence cybersecurity operations. While models like Llama 3 exhibit some offensive and defensive capabilities, the risks they introduce can be mitigated with the right tools and practices. For cybersecurity officers and developers, the future lies in leveraging AI responsibly, staying ahead of attackers, and continuously evolving defenses as these technologies develop.

References:

Wan, S., Nikolaidis, C., Song, D., Molnar, D., Crnkovich, J., Grace, J., Bhatt, M., Chennabasappa, S., Whitman, S., Ding, S., Ionescu, V., Li, Y., & Saxe, J. (2024). CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models. Meta AI. Published on July 23, 2024.