Original research

We scanned 400 real AI-built apps. Here's what we found

Everyone has an opinion about whether AI-built apps are safe. Almost nobody has counted. So we built a frozen list of 400 public repositories genuinely built with Lovable, Bolt and v0, and started scanning them. The headline number: 18.5% had a .env file committed straight into the repository.

How we picked the 400

This is the part most "we analysed AI code" posts skip, and it's the part that decides whether the numbers mean anything.

Searching GitHub for repos that mention Lovable returns hundreds of thousands of results — tutorials, blog posts, starter templates, people who simply wrote the word in a README. Sampling those gives you a number about people who talk about AI builders, not people who use them.

So we selected on a build artefact instead: a .lovable file, lovable-tagger in package.json, or a .bolt/ directory. Those are written by the tool, not by a human describing it.

What the corpus looks like

35.2% of them use Supabase — which matters, because Supabase is where row-level security either protects your users' data or doesn't.

Finding 1: nearly one in five committed their .env

74 of 400 repositories had a .env file sitting in the repo root. That is the single most damaging thing you can publish by accident. A .env is where your database URL, your service-role key, your Stripe secret and your AI provider key all live in one text file.

We recorded only whether the file was present. We never opened one. Those are strangers' live credentials, and there is no version of reading them that's defensible — so the study treats them as a yes/no signal and nothing more.

What makes this a tooling failure rather than a user failure: nobody types git add .env on purpose. It happens because the generated project didn't ship a .gitignore rule covering it, and the founder ran git add . exactly as every tutorial tells them to. The AI wrote the app; nothing told them the app came with a live grenade in it.

Finding 2: the code itself is worse than the score suggests

We then scanned individual source files with the same engine that powers VibeSafe. This part of the study is still running — the numbers below cover 115 files across 41 repositories, not the full corpus. Treat them as a strong signal, not a final answer.

An average of 73/100 sounds survivable. It isn't, and the reason is the distribution: the average is dragged up by files that are mostly fine, while the worst files scored 5, 5, 5, 23 and 27. Security doesn't average. One file scoring 5 is enough.

What the critical issues actually were

Two things stand out.

Vulnerable dependencies lead by a distance. These aren't code the AI wrote badly — they're packages it pulled in that already had known holes. The generator picked a plausible library and never checked whether that version was safe.

Prompt injection appears at all. Five criticals in a 115-file sample. These are apps that pass user input into an LLM call without separating instructions from data. It's a brand-new bug class that didn't exist in the tutorials these tools learned from, and it's already shipping.

Finding 3: the most common problems aren't "security" problems

Across every severity, the two largest categories were missing state handling (53) and code quality (53) — ahead of every classic vulnerability class.

Missing state handling means a payment, upload or webhook flow that implements the happy path and nothing else. No branch for payment failed, payment pending, webhook delivered twice, or an empty result. AI writes code that makes the demo work, because that's what it was optimised to do. "The demo works" and "the demo works when Stripe calls back twice" are different programs.

That's the honest shape of the risk. Most AI-built apps don't fail because of an exotic exploit. They fail because a real user did something slightly unusual on a Tuesday.

What these numbers do not show

What to do with this if you built with AI

• Check your .env right now. Run git ls-files | grep .env in your project. If anything comes back, your keys are public — rotate them, then remove the file from history. Deleting it in a new commit is not enough; the old commit still has it.
• Check your dependencies. The single biggest critical category, and the easiest to fix — most are a version bump.
• If you use Supabase, confirm RLS is actually on. A public anon key with row-level security enabled is the correct, safe pattern. The same key without it exposes every row to anyone who opens DevTools.
• Walk one flow that can fail. Try paying with a declining card. Refresh the confirmation page. If your app can't cope, that's the 53-count category showing up in your product.

Related: