Attacks on Generative AI Data and Using Vector Encryption to Stop Them

At DEF CON, Patrick Walsh and Bob Wall of IronCore Labs discussed vulnerabilities in Retrieval Augmented Generation (RAG) workflows and proposed vector encryption to secure sensitive data.

(Join the AI Security group at https://www.linkedin.com/groups/14545517 or https://x.com/AISecHub for more similar content)

🤔 What is Retrieval-Augmented Generation (RAG)?

RAG is a technique for enhancing the accuracy, reliability, and timeliness of Large Language Models (#LLMs) that allows them to answer questions about data they weren’t trained on, including private data, by fetching relevant documents and adding those documents as context to the prompts submitted to a LLM.

As the adoption of GenAI tools has soared, security has done little to keep up. New classes of data, and especially vector data, is flooding into new and untested data stores. Vector databases are getting copies of health data, financial data, HR data, emails, and everything else, but they have no intrinsic security. What’s worse, the vectors themselves can be reversed in embedding inversion attacks that turn those vectors back into faces, sentences, and even pictures. We discuss these new attacks and a new branch of cryptography, vector encryption, which allows for privacy preserving searches to happen over the encrypted vectors. We’ll discuss the benefits, trade-offs, and current state of the field and the open source software we’ve built to meet the new need.

🚨 Key Security Issues

Vector Inversion Attacks: Numerical embeddings used in vector searches can be inverted to recover sensitive data such as proprietary information, personal details, or internal documents. These attacks are low-effort and highly effective.

Data Proliferation: RAG workflows create multiple copies of sensitive data across systems, including vector databases, logs, and related components, significantly increasing the attack surface.

Prompt Exploitation: Malicious actors can manipulate embedded documents or prompt workflows, enabling unauthorized access or exfiltration.

Vector Database Vulnerabilities: Vector databases often lack robust security measures, with common issues including inadequate authentication, reliance solely on TLS for encryption, and exposed access points.

🛠️ Proposed Mitigation

Partially Homomorphic Encryption (#PHE): Encrypts vectors while retaining their comparability for distance-based searches, ensuring secure functionality.

Ironcore Alloy: An open-source library for implementing vector encryption with minimal performance impact (https://lnkd.in/geZK9_Jh).

This approach secures vector-based RAG workflows by preventing unauthorized access to sensitive embeddings, mitigating risks without disrupting functionality. As vector databases grow rapidly, fueled by hundreds of millions in venture Venture Capital funding, many startups still lack robust security measures, such as proper authentication, encryption for data at rest, and secure configurations. Implementing solutions like vector encryption is essential to close these gaps and ensure the safe adoption of these technologies in modern AI workflows.

📚 More Info

🔗 Security Risks with RAG Architectures https://ironcorelabs.com/security-risks-rag/

📺 DEF CON 32 — Attacks on GenAI data & using vector encryption to stop them https://www.youtube.com/watch?v=Lxg9YyFJ8s0

#AISecurity #Cybersecurity #AITrust #AIRegulation #AIRisk #AISafety #LLMSecurity #ResponsibleAI #DataProtection #AIGovernance #AIGP #SecureAI #AIAttacks #AICompliance #AIAttackSurface #AICybersecurity #AIThreats #AIHacking #MaliciousAI #AIGuardrails #ISO42001 #GenAISecurity #GenerativeAI #DataSecurity #VectorDatabases #RAGWorkflows #Cybersecurity #AIEncryption #DataPrivacy #InversionAttacks #VentureCapital Angel Investor