AI generated code security comes down to the same checks as any other code, with extra attention on five points: secrets pasted into the code, packages that do not exist or were never vetted, access control that only lives in the UI, cloud permissions that are too wide, and a pipeline that checks none of it. An app built in a few days with an assistant can pass every demo and still fail on one of these in production. This is the checklist I run before launch.
AI generated code security: the short answer
Before release I check, in this order:
- no keys or passwords in the code or in the repository history;
- every dependency exists, is the intended one and is pinned;
- every page and API checks who the user is and what they may see;
- cloud permissions are minimal, with nothing public by accident;
- a pipeline repeats these checks on every change, along with the tests.
The rest of this note explains why each point weighs more when a model writes the code.
What the research says
The first systematic study, Asleep at the Keyboard? by Pearce and colleagues (IEEE Symposium on Security and Privacy 2022), had GitHub Copilot complete 89 security-relevant scenarios, producing 1,689 programs. The authors found about 40% of them vulnerable. That was a 2021 model and today's are better, but the lesson holds: a model writes plausible code, and plausible is a different property from secure.
A second study, We Have a Package for You! by Spracklen and colleagues (USENIX Security 2025), measured how often models suggest packages that do not exist. The average was at least 5.2% for commercial models and 21.7% for open-source ones, with 205,474 unique hallucinated package names. The authors describe this as "a novel form of package confusion attack".
Secrets in the code and in the history
When you ask an assistant to wire up a service, the shortest path is to paste the key into the code. The OpenSSF Security-Focused Guide for AI Code Assistant Instructions puts it plainly: "Never include API keys, passwords, or secrets in code output."
What I check:
- automated secret scanning on the whole repository, history included;
- local config files excluded from version control;
- secrets in a secrets manager or in the runtime environment;
- separate keys for development and production.
If a key was ever committed, deleting the line is not enough. Revoke it and issue a new one.
Hallucinated packages and slopsquatting
Every package the assistant adds needs a quick check: it exists, the name is right, it is maintained, and it comes from the official registry. The OpenSSF guide describes slopsquatting as attacks "where attackers create malicious packages with names commonly hallucinated by AI models", and notes that, unlike typosquatting, it "doesn't rely on misspellings."
The OWASP Top 10:2025 now has a category for this risk, A03:2025 Software Supply Chain Failures. In practice:
- a lock file in the repository and pinned versions;
- automated scanning for known vulnerabilities in dependencies;
- no new package merged until a person has checked its name and source.
Access control and user data
This is where generated code fails most quietly. The demo shows the right data to the right user, but the API answers anyone who changes an ID in the URL. A01:2025 Broken Access Control sits at the top of the OWASP Top 10:2025.
What I verify:
- every endpoint checks the user on the server, in addition to the UI;
- every query filters data by owner or group;
- admin roles are separate and protected by a second factor;
- an automated test tries to read another user's data and must fail.
Cloud permissions and configuration
Apps built in a hurry often land on cloud resources with wide permissions: a public storage bucket, a database reachable from the internet, a service account with admin rights. Misconfiguration is second in the same OWASP list (A02:2025 Security Misconfiguration).
The rules are the ones I apply as a DevSecOps consultant on any project: least privilege for each service, infrastructure as code, backups tested with a real restore, access logs switched on.
A pipeline that checks every change
With AI the code changes more often and in bigger chunks, so the checks have to run on their own. The OWASP Top 10 CI/CD Security Risks ranks "Insufficient Flow Control Mechanisms" first: code that reaches production without review or approval.
The minimum pipeline I set up:
- a protected main branch that only accepts reviewed changes;
- tests, secret scanning, dependency scanning and static analysis on every change;
- an explicit approval for production deployments.
The wider method is in What is DevSecOps.
Rules for the assistant, and its limits
Many problems can be prevented before the code is written. Coding assistants read project instruction files, and the OpenSSF guide proposes security rules to put there. It also suggests asking the AI to "review and improve its own work", a technique it calls Recursive Criticism and Improvement. That helps, but the guide is clear on who is accountable: you should "critically evaluate and edit AI-generated code just as you would code written by a human colleague."
The Claude Code security documentation says the same from the tool's side: "You're responsible for reviewing proposed code and commands for safety before approval." I use it with limited permissions and treat the pipeline as the safety net. How I work with these tools day to day is in AI for software development.
Frequently asked questions
Is AI-generated code secure?
Not by default. Studies have found vulnerabilities and invented dependencies in a share of generated code. It becomes trustworthy through human review, tests, and a pipeline that checks secrets, dependencies and access control on every change.
What is slopsquatting?
An attack where someone registers a malicious package under a name that AI models tend to invent. When a developer installs the suggested package without checking it, the attacker's code runs in their project. Pinned dependencies and a human check on every new package stop it.
Can AI coding assistants check their own code for security?
Partly. Asking the assistant to review its own output catches some issues, and the OpenSSF guide recommends it. It does not replace independent checks: secret scanning, dependency scanning, access control tests and a human review.
What should you ask a developer who builds with AI?
Where the secrets live, how new dependencies are vetted, which tests cover access control, and who approves production releases. Also make sure the code and the access to your systems stay yours.
Sources
- Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, Pearce et al., IEEE S&P 2022
- We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs, Spracklen et al., USENIX Security 2025
- Security-Focused Guide for AI Code Assistant Instructions, OpenSSF
- OWASP Top 10:2025, OWASP Foundation
- OWASP Top 10 CI/CD Security Risks, OWASP Foundation
- Security, Claude Code Docs