Smokecheck by hotskills
See what a skill does before you run it.
Smokecheck is the scanner behind every security level on hotskills. It reads a skill’s instructions and files and reports what it runs, what it reaches over the network and what it asks for—without an LLM, offline, tied to the exact commit.
Who it is for
Teams
Know the level of every skill your team installs, and get told when one moves to a riskier level.
Agents
Ask an MCP-connected assistant to inspect a skill with check_skill and read the level before it installs anything.
Four ways to run it
Check a skill
Paste a public GitHub repository and read the report in your browser.
Command line
Run the open CLI against a local path or a repository before you publish.
GitHub Action
Add the Action to your workflow and let it check the skill on every change.
MCP check_skill
Your agent calls check_skill over the hotskills MCP and reads the level in its answer.
The CLI and the GitHub Action are free for authors.
What it reads
Smokecheck reads the skill’s instructions and the files in its folders. It looks at the code that would actually run and at imperative instructions in the SKILL.md—not prose, documentation, third-party libraries or tests. Every finding carries the file and line, so you can open it on GitHub yourself.
Four levels
Every skill gets one level, from the files that would run on your machine.
- Instructions only
- No executable files—the skill is text the agent reads.
- Local scripts
- Scripts that run on your machine, without reaching out.
- Network or broad shell
- The skill reaches the network or runs wide-ranging shell commands.
- Review
- A high-precision signal in the code that runs, or in the SKILL.md’s imperative instructions, is worth a look before you install.
Seven families of signals
Each level comes from families of signals. We publish the families and the method, not the individual checks behind them.
- Executable scripts—files the skill would run.
- Network—whether the skill reaches out over the network.
- Dangerous shell—wide-ranging or destructive shell commands.
- Secret access—reading keys, tokens, credentials or browser profiles.
- Broad permissions—a skill that asks for sweeping tool access.
- Obfuscation—hidden, encoded or invisible content in the code.
- Prompt injection—text that tells the agent to ignore instructions or hide actions.
How well it holds up
We measure Smokecheck on a blind sample, hand-labelled by someone who did not write the scanner and never used to tune it.
False “Review” flags: 2.2%—2 of the 90 skills that did not warrant Review, on a blind sample of 100 (0.3%–7.8%, 95% confidence, Clopper–Pearson), as of Oct 31, 2026.
Deterministic, offline, no LLM, tied to the commit.
What makes it different
Change alert over time
Smokecheck is tied to commits and history, so when a skill moves to a riskier level we show the date it changed and say so in the newsletter.
Install pinned to a commit
The install command points at the checked commit, not a moving branch. If the skill changes, you see the diff and the new level before you update.
Offensive security tools
A skill that is itself a security tool carries an “Offensive security tool” label, so its level reads in context.
Free for authors
The CLI and the GitHub Action cost nothing and link straight to the badge.
A report, in short
An example report on a sample skill—levels and findings with file and line.
Limits
Smokecheck reads what the files say, not what a skill does when it runs. It can miss things and it can flag something harmless.
An author can dispute a signal from Claim your skill, and the dispute goes to a human review queue.
Automated signals, not a guarantee.
