zubair-trabzada/geo-seo-claude

Read, Grep, Glob, Bash, WebFetch, Write

**Phase 1: Discovery (Sequential)** 1. Fetch homepage HTML (curl or WebFetch)

1. Fetch homepage HTML (curl or WebFetch)

# Full GEO audit of a website /geo audit https://example.com

/geo audit https://example.com

# Check if AI bots can see your site /geo crawlers https://example.com

/geo crawlers https://example.com

# Score a specific page for AI citability /geo citability https://example.com/blog/best-article

/geo citability https://example.com/blog/best-article

# Generate an llms.txt file for your site /geo llmstxt https://example.com

/geo llmstxt https://example.com

# Get a 60-second visibility snapshot /geo quick https://example.com

/geo quick https://example.com

# Generate a client-ready report /geo report https://example.com

/geo report https://example.com

#!/usr/bin/env python3

AI_CRAWLERS = {     "GPTBot": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)",

"GPTBot": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)",

"ClaudeBot": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +https://www.anthropic.com/claude-bot)",

"PerplexityBot": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)",

"GoogleBot": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",

"BingBot": "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)",

try:         response = requests.get(

response = requests.get(

try:         response = requests.get(robots_url, headers=DEFAULT_HEADERS, timeout=timeout)

response = requests.get(robots_url, headers=DEFAULT_HEADERS, timeout=timeout)

for key, check_url in [("llms_txt", llms_url), ("llms_full_txt", llms_full_url)]:         try:             response = requests.get(

try:             response = requests.get(                 check_url, headers=DEFAULT_HEADERS, timeout=timeout

response = requests.get(

for sitemap_url in sitemap_urls:         try:             response = requests.get(

try:             response = requests.get(                 sitemap_url, headers=DEFAULT_HEADERS, timeout=timeout

response = requests.get(

try:                             child_resp = requests.get(

child_resp = requests.get(

data = {"pages": pages, "count": len(pages)}     elif mode == "blocks":         response = requests.get(target_url, headers=DEFAULT_HEADERS, timeout=30)

elif mode == "blocks":         response = requests.get(target_url, headers=DEFAULT_HEADERS, timeout=30)         data = extract_content_blocks(response.text)

response = requests.get(target_url, headers=DEFAULT_HEADERS, timeout=30)

Bash

Bash

brand = '[Brand_Name]' # Check Wikipedia API directly api_url = f'https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch={quote_plus(brand)}&format=json'

# Check Wikipedia API directly api_url = f'https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch={quote_plus(brand)}&format=json' r = requests.get(api_url, headers={'User-Agent': 'GEO-Audit/1.0'}, timeout=15)

api_url = f'https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch={quote_plus(brand)}&format=json' r = requests.get(api_url, headers={'User-Agent': 'GEO-Audit/1.0'}, timeout=15) data = r.json()

r = requests.get(api_url, headers={'User-Agent': 'GEO-Audit/1.0'}, timeout=15)

if results and brand.lower() in results[0].get('title', '').lower():     print(f'WIKIPEDIA PAGE EXISTS: {results[0][\"title\"]}')     print(f'URL: https://en.wikipedia.org/wiki/{results[0][\"title\"].replace(\" \", \"_\")}')

print(f'WIKIPEDIA PAGE EXISTS: {results[0][\"title\"]}')     print(f'URL: https://en.wikipedia.org/wiki/{results[0][\"title\"].replace(\" \", \"_\")}') else:

print(f'URL: https://en.wikipedia.org/wiki/{results[0][\"title\"].replace(\" \", \"_\")}')

print('No direct Wikipedia page found') # Check Wikidata wd_url = f'https://www.wikidata.org/w/api.php?action=wbsearchentities&search={quote_plus(brand)}&language=en&format=json'

# Check Wikidata wd_url = f'https://www.wikidata.org/w/api.php?action=wbsearchentities&search={quote_plus(brand)}&language=en&format=json' r2 = requests.get(wd_url, headers={'User-Agent': 'GEO-Audit/1.0'}, timeout=15)

wd_url = f'https://www.wikidata.org/w/api.php?action=wbsearchentities&search={quote_plus(brand)}&language=en&format=json' r2 = requests.get(wd_url, headers={'User-Agent': 'GEO-Audit/1.0'}, timeout=15) wd = r2.json()

r2 = requests.get(wd_url, headers={'User-Agent': 'GEO-Audit/1.0'}, timeout=15)

**Method 2 — Direct URL check (backup verification):** 1. WebFetch: `https://en.wikipedia.org/wiki/[Brand_Name]` — check if the page loads (not a redirect to search)

1. WebFetch: `https://en.wikipedia.org/wiki/[Brand_Name]` — check if the page loads (not a redirect to search)

2. WebFetch: `https://en.wikipedia.org/wiki/[Founder_Name]` for founder article

Bash

Read, Write, Bash, Glob

Read, Grep, Glob, Bash, WebFetch, Write

Bash

- **Operator:** OpenAI - **User-Agent:** `GPTBot` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)`

- **User-Agent:** `GPTBot` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)` - **Purpose:** Fetches content for ChatGPT's web browsing, plugins, and search

- **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)`

- **Operator:** OpenAI - **User-Agent:** `OAI-SearchBot` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://docs.openai.com/bots/overview)`

- **User-Agent:** `OAI-SearchBot` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://docs.openai.com/bots/overview)` - **Purpose:** Specifically powers ChatGPT's search 

- **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://docs.openai.com/bots/overview)`

- **Operator:** OpenAI - **User-Agent:** `ChatGPT-User` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ChatGPT-User/1.0; +https://openai.com/bot)`

- **User-Agent:** `ChatGPT-User` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ChatGPT-User/1.0; +https://openai.com/bot)` - **Purpose:** Used when a ChatGPT user explicitly asks the model to 

- **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ChatGPT-User/1.0; +https://openai.com/bot)`

- **Operator:** Anthropic - **User-Agent:** `ClaudeBot` - **Full User-Agent String:** `ClaudeBot/1.0; +https://www.anthropic.com/claude-bot`

- **User-Agent:** `ClaudeBot` - **Full User-Agent String:** `ClaudeBot/1.0; +https://www.anthropic.com/claude-bot` - **Purpose:** Fetches web content for Claude's features including web search, citations, and analysis tools.

- **Full User-Agent String:** `ClaudeBot/1.0; +https://www.anthropic.com/claude-bot`

- **Operator:** Perplexity AI - **User-Agent:** `PerplexityBot` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)`

- **User-Agent:** `PerplexityBot` - **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)` - **Purpose:** Powers Perplexity's AI search engine, w

- **Full User-Agent String:** `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)`

- **Operator:** Amazon - **User-Agent:** `Amazonbot` - **Full User-Agent String:** `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (compatible; Amazonbot/0.1; +https://de

- **User-Agent:** `Amazonbot` - **Full User-Agent String:** `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (compatible; Amazonbot/0.1; +https://developer.amazon.com/supp

- **Full User-Agent String:** `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)`

- **Operator:** Common Crawl (nonprofit) - **User-Agent:** `CCBot` - **Full User-Agent String:** `CCBot/2.0 (https://commoncrawl.org/faq/)`

- **User-Agent:** `CCBot` - **Full User-Agent String:** `CCBot/2.0 (https://commoncrawl.org/faq/)` - **Purpose:** Builds the Common Crawl dataset, which is used as training data by many AI companies (Google, Meta, Stability AI, and others).

- **Full User-Agent String:** `CCBot/2.0 (https://commoncrawl.org/faq/)`

See https://contentsignals.org/ for the full specification.

Bash

https://example.com/llms.txt

- [Page Title](https://example.com/page-url): Concise description of what this page covers and why it matters.

- [Another Page](https://example.com/another-page): Description of content.

- [Less Critical Page](https://example.com/optional-page): Description.

## Contact - Website: https://example.com

- Website: https://example.com

- [Most Important Page](https://example.com/page): Description covering the key content on this page.

- [Second Page](https://example.com/page-2): Description of this page's content and value.

- [Third Page](https://example.com/page-3): What users and AI systems will find here.

- [Product A](https://example.com/product-a): Core features, target users, and pricing model for Product A.

- [Product B](https://example.com/product-b): What Product B does and how it differs from Product A.

- [Guide Title](https://example.com/guide): Comprehensive guide covering [topic] with [X] sections and practical examples.

- [Blog Post](https://example.com/blog/post): Analysis of [topic] with original data from [source].

- Website: https://example.com

4. **Include your differentiators.** If your site has unique data, original research, or exclusive features, highlight these in descriptions and Key Facts. 5. **Keep it concise.** The llms.txt should be scannable in under 60 seconds. Save d

5. **Keep it concise.** The llms.txt should be scannable in under 60 seconds. Save detail for llms-full.txt. 6. **Use absolute URLs.** Always include the full `https://` URL, never relative paths. 7. **Test after deployment.** After uploadi

6. **Use absolute URLs.** Always include the full `https://` URL, never relative paths.

7. **Test after deployment.** After uploading, verify the file is accessible at `https://yourdomain.com/llms.txt` with no redirects.

Read, Grep, Glob, Bash, WebFetch, Write

Read, Write, Bash, Glob, WebFetch

Read, Write, Bash, Glob

Read, Grep, Glob, Bash, Write

Read, Grep, Glob, Bash, WebFetch, Write

Read, Grep, Glob, Bash, WebFetch, Write

1. **Valid JSON**: Is the JSON-LD syntactically valid? Check for trailing commas, unquoted keys, malformed strings. 2. **Valid @type**: Does the `@type` match a recognized Schema.org type? Check against https://schema.org/docs/full.html.

2. **Valid @type**: Does the `@type` match a recognized Schema.org type? Check against https://schema.org/docs/full.html.

"@type": "WebSite",   "name": "Site Name",   "url": "https://example.com",

"name": "Site Name",   "url": "https://example.com",   "potentialAction": {

"url": "https://example.com",

"target": {       "@type": "EntryPoint",       "urlTemplate": "https://example.com/search?q={search_term_string}"

"@type": "EntryPoint",       "urlTemplate": "https://example.com/search?q={search_term_string}"     },

"urlTemplate": "https://example.com/search?q={search_term_string}"

1. **Wikipedia article** — highest authority entity link 2. **Wikidata item** — machine-readable entity identifier (e.g., `https://www.wikidata.org/wiki/Q12345`)

2. **Wikidata item** — machine-readable entity identifier (e.g., `https://www.wikidata.org/wiki/Q12345`)

{   "@context": "https://schema.org",

"@context": "https://schema.org",

"@type": "Organization",   "@id": "https://example.com/#organization",   "name": "Company Name",

"@id": "https://example.com/#organization",

"name": "Company Name",   "url": "https://example.com",   "logo": {

"url": "https://example.com",

"logo": {     "@type": "ImageObject",     "url": "https://example.com/logo.png",

"@type": "ImageObject",     "url": "https://example.com/logo.png",     "width": 600,

"url": "https://example.com/logo.png",

"@type": "Person",     "name": "Founder Name",     "sameAs": "https://www.linkedin.com/in/founder"

"name": "Founder Name",     "sameAs": "https://www.linkedin.com/in/founder"   },

"sameAs": "https://www.linkedin.com/in/founder"

},   "sameAs": [     "https://en.wikipedia.org/wiki/Company_Name",

"sameAs": [     "https://en.wikipedia.org/wiki/Company_Name",     "https://www.wikidata.org/wiki/Q12345",

"https://en.wikipedia.org/wiki/Company_Name",

"https://www.wikidata.org/wiki/Q12345",

"https://www.linkedin.com/company/company-name",

"https://www.youtube.com/@companyname",

"https://twitter.com/companyname",

"https://github.com/companyname",

"https://www.crunchbase.com/organization/company-name"

Read, Grep, Glob, Bash, WebFetch, Write

### 1.1 robots.txt Validity - Fetch `https://[domain]/robots.txt`

- Fetch `https://[domain]/robots.txt`

- Check for syntactic validity: proper `User-agent`, `Allow`, `Disallow` directives - Check for common errors: missing User-agent, wildcards blocking important paths, Disallow: / blocking entire site - Verify XML sitemap is referenced: `Sit

- Verify XML sitemap is referenced: `Sitemap: https://[domain]/sitemap.xml`

### Detection Method 1. Fetch the page with curl (no JavaScript execution): `curl -s [URL]`

1. Fetch the page with curl (no JavaScript execution): `curl -s [URL]`

3. If key content (headings, paragraphs, product info, article text) is MISSING from the curl output, the site relies on client-side rendering

### 8.1 Time to First Byte (TTFB) - Target: **< 800ms** (ideally < 200ms) - Measure with curl: `curl -o /dev/null -s -w 'TTFB: %{time_starttransfer}s\n' [URL]`

- Target: **< 800ms** (ideally < 200ms) - Measure with curl: `curl -o /dev/null -s -w 'TTFB: %{time_starttransfer}s\n' [URL]` - If TTFB > 800ms: check server location, caching, database queries, CDN usage

- Measure with curl: `curl -o /dev/null -s -w 'TTFB: %{time_starttransfer}s\n' [URL]`

**How to check:** Capture all `Link:` response headers from the standard homepage fetch (no extra request).

### Implementation Check 1. Check for IndexNow key file: `https://[domain]/.well-known/indexnow-key.txt` or similar

1. Check for IndexNow key file: `https://[domain]/.well-known/indexnow-key.txt` or similar

Bash

TEMP_DIR=$(mktemp -d) git clone --depth 1 https://github.com/zubair-trabzada/geo-seo-claude.git "$TEMP_DIR/repo"

git clone --depth 1 https://github.com/zubair-trabzada/geo-seo-claude.git "$TEMP_DIR/repo"

rm -rf "$TEMP_DIR"