Individual skill

robots-meta-conflict

Use when applies to sites that use both robots.txt disallow rules and meta robots noindex tags. Use when investigating why deindexed pages still appear in search results.

skills/robots-meta-conflict/SKILL.md

Read pinned source

Source commit e8d14d049010e2ce25afcd5e8609dfcbbe63e47e

What was checked

Matches the observed repository HEAD: Yes
Scanner rules are current: Yes
Repository scan complete: No
Skill scan complete: Yes
Observed behavior: Network or broad shell
Last checked: Nov 9, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings): 74,709
Repository last updated: Oct 31, 2026
License: MIT

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

frontend Repository topic
front-end-development Repository topic
front-end-developer-tool Repository topic
guidelines Repository topic
checklist Repository topic
reference Repository topic
web-development Repository topic
html Repository topic
css Repository topic
javascript Repository topic
engineering Derived category

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files

SKILL.mdcore · 2,221 bytes
references/rule.mddoc · 3,660 bytes

Read excerpt


# Robots Meta Conflict

Pages that appear in search results despite your intention to deindex them are usually blocked in robots.txt — Google cannot read the noindex directive it cannot fetch.

## Quick Reference

- If a page is blocked in robots.txt, crawlers never fetch it and therefore never read its `noindex` meta tag
- To properly deindex a page, use `<meta name="robots" content="noindex">` without blocking it in robots.txt
- If you block a URL in robots.txt, Google may still show it in results using anchor text from backlinks
- Audit pages that need to be deindexed by confirming they are crawlable but carry `noindex`

## Check

Cross-reference the site's robots.txt disallow paths against pages carrying `<meta name="robots" content="noindex">`. Flag any page where the URL matches a Disallow rule AND has a noindex tag — the noindex will never be seen by crawlers.