Skill Files Are Shadow Code Now
Skill Files Are Shadow Code Now
A new dataset found millions of public agent skill files on GitHub. The lesson is not that developers like markdown. It is that enterprises are about to inherit executable policy from files their scanners still treat as documentation.
On August 11, researchers published GitSkills, a dataset of 3,797,117 SKILL.md files collected from 282,200 public GitHub repositories in July 2026. They grouped those into 1,877,981 distinct contents and preserved repository, path, hash, metadata, and selected history for research.
That number should land differently for security teams than it does for software engineering researchers.
An agent skill is not just a README. It is a natural-language operating procedure that a model may load, interpret, and execute at runtime. It can point to scripts, reference files, prompts, commands, APIs, browser steps, and trust assumptions. It can change how an agent decides what to do.
In other words: skill files are becoming shadow code.
The uncomfortable part is not the scale alone. It is the combination of scale, weak provenance, and probabilistic selection.
Why this is not documentation risk
Security programs know how to deal with code. Code has owners, repositories, reviews, CI checks, package manifests, dependency graphs, SBOM entries, vulnerability scanners, and release gates.
Skills do not fit neatly into that machine.
The GitSkills paper makes the core distinction clear: skills are mostly natural language, the model selects them at runtime, and there is no compiler or type checker validating the selection. They also spread by copying folders across repositories rather than through a single package manager or registry.
That breaks three assumptions behind normal software governance:
- Inventory is incomplete. A skill can live in a cloned repo, shared prompt folder, internal wiki export, marketplace listing, or copied archive.
- Review is ambiguous. The dangerous behavior may be in wording, examples, references, or task-routing criteria rather than executable code.
- Execution is contextual. The same skill can be harmless for one agent and dangerous for another if the second agent has shell access, production credentials, or write permissions.
The control failure
Treating SKILL.md as documentation is like treating a CI workflow as a comment. The file may be text, but the agent reads it as operational authority.
The new supply chain is chosen by the agent
Last month, Island researchers disclosed AgentBaiting, part of the broader FakeGit operation. They found roughly 7,600 malicious GitHub repositories, with 800+ posing as AI Skills or MCP servers, and more than 600 listings across public AI capability registries.
The attack was simple and ugly: fake repositories used convincing names, lookalike maintainers, polished READMEs, and malicious downloads. In tests, agents and assistants could discover those repositories without being handed a malicious link, then repeat attacker-written installation instructions to the user.
GitSkills shows why this pattern will scale. There are now millions of natural-language capability files in public repositories. Some are useful. Some are stale. Some are copied without context. Some will be attacker-controlled. Most will not have the signals your dependency scanner expects.
That turns “find me a skill for X” into a security-sensitive operation.
The security boundary is no longer only “which package did npm install”. It is also “which natural-language operating procedure did the agent decide to obey”.
Map it to OWASP Agentic Top 10 (2026)
This is not a single OWASP category. Skill sprawl touches several agentic risks at once:
| OWASP risk | How skill files trigger it | What to control |
|---|---|---|
| ASI04 Agentic supply chain | Skills are copied, mirrored, forked, and discovered through public repositories without a single trusted distribution path. | Allowlisted sources, content hashes, signed internal catalogs, and review status before activation. |
| ASI06 Memory and context poisoning | A skill can alter persistent agent behavior by redefining what counts as correct procedure, safe output, or trusted evidence. | Instruction isolation, context labels, source reputation, and separate treatment for durable policy. |
| ASI05 Unexpected code execution | Skill instructions often route the agent to scripts, shell commands, browser actions, installers, or local files. | Execution sandboxes, per-skill permissions, command previews, and reversible action logs. |
| ASI10 Rogue agents | Unreviewed skills can create long-lived behavior changes that survive beyond a single task or session. | Runtime monitoring, kill switches, drift detection, and rapid skill disablement. |
The key word is “agentic”. The risk comes from the combination of text, selection, tools, identity, memory, and action.
What to inspect before a skill runs
Most enterprises will eventually need an internal skill catalog. Until then, security teams can still impose a basic gate.
skill gate checks
source: repository owner, age, stars, recent commits, domain match
content: commands, downloads, secrets handling, network egress, hidden references
capability: requested tools, file paths, app scopes, browser permissions
context: production vs. dev, human approval level, persistence allowed
block: unsigned binaries, raw release ZIPs, “disable warnings”, credential collection, broad shell access
Do not make this a giant policy project before you get value. Start with the places where a bad skill becomes a real incident:
Developer agents
Block new skills from modifying dependency manifests, CI files, or shell startup files without human review.
Ops agents
Require explicit approval before a skill can touch cloud consoles, tickets, alert routing, or runbooks.
Data agents
Treat skill-provided query templates and export instructions as untrusted until source and scope are verified.
The missing artifact: a skill bill of materials
SBOMs made software dependencies visible. Agent systems need the same idea for natural-language capabilities.
A useful skill bill of materials should answer five questions:
- Which skills can each agent load?
- Who approved the skill and when?
- What hash and source did the approval cover?
- What tools, files, apps, and network paths can the skill reach?
- What changed since the last approved version?
Without that inventory, you cannot answer the incident response question that matters most: “Which agents may have followed this instruction?”
Practical stance
Skills should move through the same governance path as code that can affect production: source verification, review, constrained execution, telemetry, and revocation.
The CISO question
Ask a simple question this week:
Can any employee or agent introduce a new skill into an enterprise workflow without security review?
If the answer is yes, you have a shadow automation problem. It may not show up in your dependency scanner. It may not show up in your SaaS inventory. It may look like a markdown file in a repo nobody owns.
But when an agent reads it, loads it, and acts on it, that file becomes part of your control plane.
Sources
- GitSkills: A Dataset of Agent Skills on GitHub, submitted August 11, 2026.
- AgentBaiting: How 800+ Fake AI Skills and MCP Servers Delivered Malware, Island Security Research, July 20, 2026.
- OWASP Top 10 for Agentic Applications for 2026, OWASP GenAI Security Project.