Your AI Picked the Packages. Did Anyone Check Them?

Quick answer: when an AI tool builds your app, it doesn't write everything from scratch. It pulls in ready-made code from other people, called packages or dependencies, and some of those can have known security holes. You won't see this anywhere on screen, because a vulnerable package works perfectly until someone abuses it. To check, you need a list of what your app actually uses and a comparison against public databases of known vulnerabilities, which GitHub's free alerts or a code scan like FlawPilot's can do for you. Expect to find a few. It's normal, and most are fixed by updating to a newer version.

Think of your app like a house. The AI is the builder, and packages are the doors, windows and locks it bought from suppliers instead of making by hand. That's sensible, it's how nearly all modern software is built. The problem is that some of those suppliers later announce "the lock we sold in that batch can be opened with a hairpin," and nobody goes back to swap the lock unless someone is paying attention.

With a vibe-coded app, usually nobody is. The AI picked the parts, the app works, and the list of parts sits in a file you've never opened.

This is part 2 of our series on what vibe-coded apps miss. Start with the overview here.

In short: your app is mostly made of other people's code. You didn't choose it, you can't see it, and parts of it may already be publicly known to be unsafe. Checking takes minutes, and fixing is usually an update, not a rebuild.

What a dependency actually is

A dependency is a piece of code someone else wrote and published so other developers can reuse it. Need a date picker, a chart, a way to handle payments or log users in? There's a package for that. Your app "depends" on it, hence the name.

Packages also depend on other packages. So when the AI adds one package to your project, it may quietly bring along ten or twenty more that package needs. A small app can easily end up with hundreds of these underneath, even if only a handful were named on purpose.

In a JavaScript app (which is what most AI builders produce), the packages your app asks for are listed in a file called package.json. A second file, usually package-lock.json, records the exact version of every package, including all the hidden ones.

What the problem looks like

That's the tricky part: it looks like nothing.

• The app loads fine.
• Every button works.
• Nothing shows up as an error.

A vulnerable package doesn't announce itself. The flaw only matters when someone finds your app and sends it input designed to exploit that specific weakness. Because known vulnerabilities are published openly (so developers can fix them), attackers can read the same lists and go looking for apps that haven't updated.

The symptoms, when they eventually appear, are the expensive kind: data that leaks, accounts that get taken over, or a site that starts behaving in ways you didn't build.

Why AI tools miss it

This isn't a case of the AI being careless. It's a gap in what the tools are designed to do.

They learned from the past

AI models are trained on code written before a certain date. When they suggest a package, they often suggest the version that was common in that code. A version that was the safe, popular choice a while ago may have had problems announced since.

"It works" is the finish line

AI builders optimize for the thing you asked for: a working feature. When the feature works, the job looks done. Nothing in that loop asks "is this package still maintained?" or "has anything been reported against this version?"

They add more than they need

When an AI tries a few approaches to solve a problem, it can install packages along the way and never remove the ones it ended up not using. Every unused package is still part of your app, and still something that could need patching.

Sometimes the package doesn't exist

AI tools occasionally invent package names that sound right but were never real. That's a known issue in AI-generated code, and attackers have started publishing packages under plausible made-up names, hoping an AI will suggest them and someone will install them without checking. A package with a convincing name is not proof it's the package you think it is.

Nobody comes back later

Even a perfectly chosen package can have a vulnerability reported next month. Developers on a team usually have automated alerts and a habit of updating. A founder who built the app in a weekend usually has neither.

Check it yourself

You don't need to read code for this. You need about fifteen minutes and access to your project's files.

• Get your code into GitHub. Most AI builders (Lovable, Bolt, Replit, v0) have an export or "connect to GitHub" option. If you use Cursor, your project is likely already in a Git repository. Having it on GitHub unlocks the free checks below.
• Open package.json and read the list. Look under dependencies and devDependencies. You're not judging the code, just getting a feel for how many there are and whether the names make sense for what your app does. A to-do app with a package for PDF editing and three different chart libraries is a sign of leftovers.
• Turn on GitHub's security alerts. In your repository, go to the Security tab and enable Dependabot alerts. GitHub compares your packages against its database of known vulnerabilities and lists anything it finds, with a severity level for each.
• Look up anything unfamiliar. Search the package name on npmjs.com. Check three things: is it downloaded widely, when was it last updated, and does it link to a real source code repository? A package with very few downloads, no repository and a name almost identical to a famous one deserves a closer look.
• Ask your AI tool to explain its choices. A prompt like "List every package in package.json, what it's used for in this app, and whether any are unused" is a good start. Remove what isn't used, then test that the app still works.
• Check that a lock file exists. If you see package-lock.json (or yarn.lock or pnpm-lock.yaml), your exact versions are recorded. Without one, each new install can pull in different versions than the ones you tested.

How FlawPilot spots it

FlawPilot looks at this from two directions: your code and your live site.

Code scan: what's inside your repository

When you connect your repository to FlawPilot's code scan, it checks for vulnerable dependencies: packages in your project that match known, publicly reported vulnerabilities. Each one shows up as a finding, so you can see which package is affected and what the issue is, rather than a vague "your app has problems" warning.

The same code scan also covers code security, code quality and exposed secrets (things like API keys accidentally committed to the code), so one connection gives you a wider picture than dependencies alone.

For each finding, FlawPilot can generate an AI fix suggestion that explains how to resolve it. You can hand it straight to your AI builder or a developer instead of trying to work it out yourself.

Quick scan: what the outside world can see

FlawPilot's quick scan doesn't need access to your code. You give it your site's address, and its Security pillar checks for known CVEs (the public ID system for reported vulnerabilities) along with security headers, TLS and exposed secrets. This is the view an attacker gets: not your files, just what your live site reveals about itself.

Running both is the fuller answer. The code scan finds problems in what you've built, and the quick scan shows which of them are visible from outside.

Run a FlawPilot scan on your app to see where you stand.

What "fixed" looks like

When a developer, or your AI tool, says the dependencies are sorted, here's what that should mean:

• No known high or critical vulnerabilities remain, or each remaining one has a clear reason (for example, the vulnerable part of the package is never used by your app, and someone actually checked).
• Unused packages are gone. The list in package.json matches what the app does.
• A lock file is committed, so versions don't drift silently.
• Alerts stay on. Dependabot or a scheduled scan keeps watching after today.
• The app was tested after updating. Updates sometimes change how a package behaves, so "updated" should always come with "and everything still works."

Urgent or can it wait?

Not every finding means drop everything. A rough way to sort them:

The goal isn't zero findings forever. Software doesn't work that way. The goal is knowing what's in your app and having a habit of keeping it current.

Final Thoughts

The packages in your app are the part you have the least visibility into and the least say over. You didn't pick them, you can't see them on screen, and the AI that chose them won't come back to check on them. That doesn't make them bad. It makes them unchecked.

The fix is mostly a habit, not a project. Get a list of what you're running, turn on alerts, update what's flagged, and remove what isn't used. A few minutes now saves you from finding out about a public vulnerability the way nobody wants to: from someone who used it.