Menu

Individual skill

agent-ready-robots-txt

0upvotes

Sub-skill de agent-ready-cloudflare: Implement robots.txt

skills/agent-ready-cloudflare/robots-txt/SKILL.md

Read pinned source

Source commit ec4099f92fdeadcf1771c2a3f64efa4970733c89

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
Yes
Observed behavior
Instructions only
Last checked
Nov 11, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
133
Repository last updated
Nov 1, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt
# Implement robots.txt

Publish a valid robots.txt at your site root per
[RFC 9309](https://www.rfc-editor.org/rfc/rfc9309).

## Requirements

- Serve `/robots.txt` as `text/plain` with HTTP 200
- Include `User-agent` directives with `Allow`/`Disallow` rules
- Reference your sitemap if one exists: `Sitemap: https://example.com/sitemap.xml`

## Cloudflare

[AI Crawl Control](https://developers.cloudflare.com/ai-crawl-control/)
can manage your robots.txt from the dashboard, including AI-specific bot rules.
Back to top