indexability-conflicts

• SKILL.mdcore · 3,512 bytes
• references/rule.mddoc · 4,931 bytes

# Avoid conflicting indexability signals

Conflicting indexability directives create unpredictable crawling and indexing behaviour. The most dangerous combination is robots.txt blocking a page that also has a `noindex` tag — the `noindex` is never read, but the URL is still known to Google, leaving it in a limbo state that wastes crawl budget.

## Quick Reference

- robots.txt blocks crawling; `noindex` blocks indexing — they are different mechanisms and should not be applied together
- A page blocked in robots.txt cannot receive a `noindex` directive because crawlers never read the page
- Canonical tags pointing to a `noindex` page create an unresolvable conflict — canonicalise to an indexable URL instead

## Check

For each page, collect four signals: (1) Is the URL path blocked by robots.txt? (2) Does the page HTML contain `<meta name='robots' content='noindex'>`? (3) Does the HTTP response include an `X-Robots-Tag: noindex` header? (4) Does the page's `<link rel='canonical'>` point to a different URL? Flag: pages blocked in robots.txt that also have noindex directives, pages with canonical pointing to a noindex URL, and pages with conflicting index/noindex signals from meta and HTTP header.