AI security: prompt injection, data leaks and how to defend
The biggest AI security risks are prompt injection, where hidden instructions in content trick a model into ignoring its rules; sensitive data leaking through prompts, outputs or logs; agents and tools with too much access; and insecure model or plugin supply chains. No single fix removes prompt injection, so defend in layers: limit what the AI can access and do, separate trusted instructions from untrusted content, require approval for risky actions, filter outputs, log everything and test regularly.
What makes AI security different?
Traditional software follows code; AI models follow language. That means anyone who can get text in front of a model, through a web page, email, document or chat message, may be able to influence what it does. As assistants gain access to email, files and business systems through connectors and agents, the impact of a successful attack grows. The OWASP Top 10 for LLM Applications lists prompt injection as the leading risk, alongside sensitive information disclosure, supply chain issues and excessive agency.
What is prompt injection?
Prompt injection is when input changes a model’s behaviour against the developer’s intent. Direct injection comes from a user typing instructions such as “ignore your previous rules”. Indirect injection is more dangerous: instructions hidden in content the AI reads, such as white text on a web page, a comment in a document or an email, tell it to leak data or take an action. Because models cannot reliably tell instructions from data, there is no complete fix; the goal is to limit the damage.
What are the other main risks?
- Data leaks: Confidential data in prompts, retrieved documents, outputs or logs reaching the wrong people.
- Excessive agency: Agents with broad permissions that can send, delete or pay without checks.
- Insecure tools and connectors: Malicious or poorly secured plugins and MCP servers.
- Supply chain: Compromised models, datasets or packages.
- Insecure output handling: AI output used directly in code, queries or web pages without validation.
- Shadow AI: Staff using unapproved tools with company data.
Our guides to MCP security risks, API security and whether vibe-coded software is secure cover related threats.
How do you defend AI systems?
- Least privilege: Give each assistant or agent only the data and actions it needs; default to read-only.
- Separate trust levels: Keep system instructions apart from untrusted content, and tell the model which is which.
- Human approval for risky actions: Sending, paying, deleting and sharing need a person to confirm.
- Validate outputs: Treat AI output as untrusted input to other systems; sanitise and check it.
- Protect data: Mask secrets, restrict retrieval by user permissions and set log retention.
- Monitor and log: Record prompts, tool calls and actions; alert on unusual patterns.
- Red-team regularly: Test with injection attempts before and after launch.
How do you stop data leaking through AI?
- Use business plans with no training on your data
- Apply the same access rules to AI retrieval as to the source systems
- Remove secrets and unnecessary personal data before sending prompts
- Limit what connectors can see, folder by folder
- Set retention periods for prompts and logs
- Tell staff which data may go into which tools
A clear AI usage policy and a plan for shadow AI reduce leaks from everyday use.
Why do agents raise the stakes?
A chatbot that only answers questions can embarrass you; an agent that can act can cause real harm. When an agent reads email, browses the web and can send messages, a single malicious page can trigger an action. Design agents so that untrusted content cannot directly cause high-impact actions, and monitor them in production. Our guide to AgentOps covers monitoring, and human-in-the-loop AI covers approvals.
How do you test AI security?
Build a set of attack cases, including hidden instructions in documents, attempts to extract system prompts and requests for other users’ data, and run them whenever you change prompts, models or tools. Combine automated tests with manual red-teaming by people who think like attackers. Track results over time, because model updates can change behaviour.
What does this look like in practice?
A software company in Dubai launches an AI assistant that reads customer support tickets and drafts replies. In testing, a ticket containing hidden text tries to make the assistant include another customer’s details. The team limits retrieval to the ticket’s own account, marks ticket text as untrusted, requires agents to approve every reply and logs all tool calls. It adds the attack to its test suite and runs it on each release.
How UIUX Studio helps
We design and build secure AI assistants and agents with least-privilege access, approvals and monitoring. Talk to our Agentic AI & Automation or Product Engineering teams.
Frequently asked questions
What is prompt injection?
Input that tricks an AI model into ignoring its rules or taking unintended actions.
Can prompt injection be prevented?
Not fully; limit access, require approvals and monitor.
Is ChatGPT safe for company data?
Business plans with data commitments are safer than personal accounts.
What is excessive agency?
An AI agent having more permissions than its task needs.
How often should we test AI security?
On every significant change and regularly in production.