bpawnzZ/lemonwrench

# failover still works: pin a dead host, expect a real page back (~45-90 s) rm -f ~/.cache/lemon/hosts.json LEMON_BASE=https://lemon-manuals.invalid LEMON_NOCACHE=1 python3 scripts/lemon.py get \

rm -f ~/.cache/lemon/hosts.json

rm -f ~/.cache/lemon/hosts.json LEMON_BASE=https://lemon-manuals.invalid LEMON_NOCACHE=1 python3 scripts/lemon.py get \   "/Pontiac/2007/G6%20V6-3.5L/Repair%20and%20Diagnosis/Starting%20and%20Charging/Charging%20System/Alternator/Service%20

LEMON_BASE=https://lemon-manuals.invalid LEMON_NOCACHE=1 python3 scripts/lemon.py get \

Named for [lemon-manuals.la](https://lemon-manuals.la), the source it scrapes: ~10,000 US/Canada vehicles, 1960–2025, free, no login. Two databases per year — a 2025 capture (`LEMON`) for modern cars and 2013 DVD dumps (`CHARM`) for 90s/200

$ python3 scripts/lemon.py torque "Pontiac/2007/G6 V6-3.9L" valve cover -p 1 ## Valve Rocker Arm Cover Replacement - Left Side https://lemon-manuals.la/Pontiac/2007/G6%20V6-3.9L/.../Valve%20Rocker%20Arm%20Cover%20Replacement%20-%20Left%20Si

## Valve Rocker Arm Cover Replacement - Left Side https://lemon-manuals.la/Pontiac/2007/G6%20V6-3.9L/.../Valve%20Rocker%20Arm%20Cover%20Replacement%20-%20Left%20Side/ Tighten the bolts to 10 N.m (89 lb in).

https://lemon-manuals.la/Pontiac/2007/G6%20V6-3.9L/.../Valve%20Rocker%20Arm%20Cover%20Replacement%20-%20Left%20Side/

git clone https://github.com/bpawnzZ/lemonwrench.git ~/.myagent/skills/lemonwrench

python3 scripts/lemon.py find "Chevrolet/2007/Impala V6-3.9L" generator bracket   # local grep python3 scripts/lemon.py sub  "https://lemon-manuals.la/.../Alternator/"          # reveal children

python3 scripts/lemon.py sub  "https://lemon-manuals.la/.../Alternator/"          # reveal children

python3 scripts/lemon.py get  "https://lemon-manuals.la/.../Service%20and%20Repair/"

python3 scripts/lemon.py pic  https://lemon-manuals.la/images25/GM1557668/

python3 scripts/lemon.py note https://lemon-manuals.la/images25/GM1557668/ "right rocker cover: 6-bolt ring, coil bracket above, MAP connector"

Primary source: **https://lemon-manuals.la** — ~10,000 US/Canada vehicles across **69 makes, 1960–2025**, free, no login. Domestic, import, orphan brands (Pontiac, Oldsmobile, Saturn, Saab, AMC, Geo, Mercury, Suzuki), Euro exotics, trucks/c

`read`/`get` emit `[FIGURE https://lemon-manuals.la/images25/XXXX/]` lines, with a

python3 $S pic https://lemon-manuals.la/images25/GM1557668/   # -> ~/.cache/lemon/img/GM1557668.png

python3 $S note https://lemon-manuals.la/images25/GM1557668/ "right rocker cover, bolt ring, coil bracket above"

- URLs are case-sensitive and the make spelling is the site's (`Dodge and Ram`,   `Mercedes Benz`); `canon_make` handles it, manual curl does not.

`Mercedes Benz`); `canon_make` handles it, manual curl does not.

- Mirrors: `charm.li` (Operation CHARM — separate host *and* corpus, tried second),   then `lemon-manuals.org.ua`, `lemon-manuals.gy`, `lemon.dogeware.me`. The script   fails over automatically on a timeout, 404 or 429/5xx; `LEMON_BASE=http

then `lemon-manuals.org.ua`, `lemon-manuals.gy`, `lemon.dogeware.me`. The script   fails over automatically on a timeout, 404 or 429/5xx; `LEMON_BASE=https://…` puts   one host first without disabling the rest.

fails over automatically on a timeout, 404 or 429/5xx; `LEMON_BASE=https://…` puts

- **A reply that isn't a page is a failed host, not an answer.** 404 bodies and   Cloudflare challenge pages become `NotFound`/`Blocked` inside `_get()`, are skipped   per host, and are never written to the page cache. `fetch()` only return

Cloudflare challenge pages become `NotFound`/`Blocked` inside `_get()`, are skipped   per host, and are never written to the page cache. `fetch()` only returns the   `Page Not Found` sentinel when *every* host 404s — before 2.2 the first ho

per host, and are never written to the page cache. `fetch()` only returns the

before a host loses its turn, so a slow host that keeps sending bytes is never cut   off. - If every host errors, `fetch()` tries an Internet Archive snapshot

off. - If every host errors, `fetch()` tries an Internet Archive snapshot   (`archive.org/wayback/available`, fetched with the `if_` modifier). archive.org

- If every host errors, `fetch()` tries an Internet Archive snapshot

path and no 2010+ manuals — a 404 from it is normal and costs one round trip.   When `index` is crawling at a few KB/s, don't wait it out: `get` the charm.li tree   (`http://charm.li/<Make>/<Year>/<Vehicle>/Repair and Diagnosis/`, ~2.9 MB i

When `index` is crawling at a few KB/s, don't wait it out: `get` the charm.li tree   (`http://charm.li/<Make>/<Year>/<Vehicle>/Repair and Diagnosis/`, ~2.9 MB in   seconds) and grep it locally, then fetch only the winning pages by their abs

(`http://charm.li/<Make>/<Year>/<Vehicle>/Repair and Diagnosis/`, ~2.9 MB in   seconds) and grep it locally, then fetch only the winning pages by their absolute   URLs.

#!/usr/bin/env bash

| sed 's|figures/||' | sort -u | while read -r file; do     id="${file%.*}"     path=$(python3 "$LEMON" pic "https://lemon-manuals.la/images25/$id/" | tail -1)

id="${file%.*}"     path=$(python3 "$LEMON" pic "https://lemon-manuals.la/images25/$id/" | tail -1)     cp "$path" "figures/$file"

path=$(python3 "$LEMON" pic "https://lemon-manuals.la/images25/$id/" | tail -1)

Every request in the script already goes through `fetch()`, and every URL is

cached for 7 days. Both now raise inside `_get()`, are skipped per host, and are never written to the page cache. Callers that already test for `Page Not Found` in the body keep working: `fetch()` returns that sentinel only

never written to the page cache. Callers that already test for `Page Not Found` in the body keep working: `fetch()` returns that sentinel only when every host 404s.

`Page Not Found` in the body keep working: `fetch()` returns that sentinel only

`fetch()` walks a fixed mirror list and treats the *first* host that answers

- Add `http://charm.li` (Operation CHARM, the upstream of LEMON's CHARM set) to

- [x] 1.1 Add `http://charm.li` to the mirror list.

- [x] 1.3 Raise on 404 and on Cloudflare challenge bodies inside `_get()`; treat       both as "this host is no use for this page". - [x] 1.4 `fetch()`: only return the `Page Not Found` sentinel when **every**

both as "this host is no use for this page". - [x] 1.4 `fetch()`: only return the `Page Not Found` sentinel when **every**       host 404s; never cache a 404 or a challenge page.

- [x] 1.4 `fetch()`: only return the `Page Not Found` sentinel when **every**

#!/usr/bin/env python3

[Credential-related source omitted]

[Credential-related source omitted]

MIRRORS = ["https://lemon-manuals.la",

"http://charm.li",              # Operation CHARM: separate host + corpus

"https://lemon-manuals.org.ua", "https://lemon-manuals.gy",

"https://lemon.dogeware.me"]

[Credential-related source omitted]

[Credential-related source omitted]

HOSTS = os.path.join(CACHE, "hosts.json")   # sticky host + cooldowns DOWN_FOR = 900                              # a failed host sorts last for 15 min WAYBACK = "https://archive.org/wayback/available?url="

DOWN_FOR = 900                              # a failed host sorts last for 15 min WAYBACK = "https://archive.org/wayback/available?url=" CHALLENGE = "cdn-cgi/challenge-platform"    # Cloudflare JS wall, not a page

WAYBACK = "https://archive.org/wayback/available?url="

[Credential-related source omitted]

[Credential-related source omitted]

def _get(url, timeout):     req = urllib.request.Request(url, headers={"User-Agent": UA})

req = urllib.request.Request(url, headers={"User-Agent": UA})

try:         with urllib.request.urlopen(req, timeout=timeout, context=_ctx()) as r:             body = r.read().decode("utf-8", "replace")

with urllib.request.urlopen(req, timeout=timeout, context=_ctx()) as r:

try:         q = urllib.request.Request(WAYBACK + urllib.parse.quote(url, safe=""),

q = urllib.request.Request(WAYBACK + urllib.parse.quote(url, safe=""),

headers={"User-Agent": UA})         with urllib.request.urlopen(q, timeout=30, context=_ctx()) as r:             snap = json.load(r).get("archived_snapshots", {}).get("closest") or {}

with urllib.request.urlopen(q, timeout=30, context=_ctx()) as r:

if snap.get("status") != "200" or not snap.get("url"):             return None         with urllib.request.urlopen(

return None         with urllib.request.urlopen(                 urllib.request.Request(_if_raw(snap["url"]), headers={"User-Agent": UA}),

with urllib.request.urlopen(

urllib.request.Request(_if_raw(snap["url"]), headers={"User-Agent": UA}),

def fetch(url, timeout=None, nocache=False):

names = open(path, encoding="utf-8").read().split("\n")     else:         body = fetch(BASE + "/")

else:         body = fetch(BASE + "/")         names = [urllib.parse.unquote(u[len(BASE) + 1:].strip("/"))

body = fetch(BASE + "/")

v + "Repair%20and%20Diagnosis/"):         try:             body = fetch(BASE + urllib.parse.quote(cand, safe="/%"), nocache=refresh)

try:             body = fetch(BASE + urllib.parse.quote(cand, safe="/%"), nocache=refresh)         except Exception:

body = fetch(BASE + urllib.parse.quote(cand, safe="/%"), nocache=refresh)

try:         lt = BASE + v + "Labor%20Times/"         body = fetch(lt, nocache=refresh)

lt = BASE + v + "Labor%20Times/"         body = fetch(lt, nocache=refresh)         if "Page Not Found" not in body[:600] and "Labor times are only available" not in body:

body = fetch(lt, nocache=refresh)

def render(url, body=None):     body = body if body is not None else fetch(url)

body = body if body is not None else fetch(url)

body = fetch(url)

continue         seen.add(label)         body = fetch(url)

seen.add(label)         body = fetch(url)         lines, target = torque_lines(text_of(body)), (url, label)

body = fetch(url)

for sub_url, sub_label in ([] if lines else sublinks(url, body)[:4]):             if not lines:                 lines, target = (torque_lines(text_of(fetch(sub_url))),

if not lines:                 lines, target = (torque_lines(text_of(fetch(sub_url))),                                  (sub_url, sub_label))

lines, target = (torque_lines(text_of(fetch(sub_url))),

'<li><a href="X/">X</a></li><img alt="a" src="/images25/A/">'          '<img src="/icons/wrench.svg">')     assert imgs_of(h) == ["https://lemon-manuals.la/images25/A/"]

'<img src="/icons/wrench.svg">')     assert imgs_of(h) == ["https://lemon-manuals.la/images25/A/"]     assert "LEMON Manuals" not in text_of(h) and "X" in text_of(h)

assert imgs_of(h) == ["https://lemon-manuals.la/images25/A/"]

assert score("Torque Specifications", "/v/x/Torque/", ["torque"], 1) is not None     assert score("Torque", "/v/x/Y/", ["brake"], 1) is None     assert _swap_base("https://lemon-manuals.la/v/x/", "https://lemon.dogeware.me") \

assert score("Torque", "/v/x/Y/", ["brake"], 1) is None     assert _swap_base("https://lemon-manuals.la/v/x/", "https://lemon.dogeware.me") \         == "https://lemon.dogeware.me/v/x/"

assert _swap_base("https://lemon-manuals.la/v/x/", "https://lemon.dogeware.me") \

== "https://lemon.dogeware.me/v/x/"

assert "http://charm.li" in _bases()

assert _swap_base("http://charm.li/v/x/", "https://lemon-manuals.la") \

== "https://lemon-manuals.la/v/x/"

assert _ours("http://charm.li/v/x/")

global _ST     _ST = {"prefer": "http://charm.li", "down": {"https://lemon-manuals.la": time.time() + 60}}

_ST = {"prefer": "http://charm.li", "down": {"https://lemon-manuals.la": time.time() + 60}}

[Credential-related source omitted]

[Credential-related source omitted]

b = _bases()         assert b[0] == "http://charm.li" and b[-1] == "https://lemon-manuals.la", b     _ST = {}

assert b[0] == "http://charm.li" and b[-1] == "https://lemon-manuals.la", b

_ST = {}     assert "http://charm.li" in imgs_of('<img src="/images/X/1/">', "http://charm.li/p/")[0]     assert _if_raw("http://web.archive.org/web/20190101/https://x/y/") \

assert "http://charm.li" in imgs_of('<img src="/images/X/1/">', "http://charm.li/p/")[0]

assert _if_raw("http://web.archive.org/web/20190101/https://x/y/") \

== "http://web.archive.org/web/20190101/if_/https://x/y/"

assert _if_raw("http://web.archive.org/web/20190101/http://x/y/") \

== "http://web.archive.org/web/20190101/if_/http://x/y/"

assert len(torque_lines(t)) == 2, torque_lines(t)     assert torque_lines("no numbers here at all") == []     b = "https://lemon-manuals.la/v/x/"

assert torque_lines("no numbers here at all") == []     b = "https://lemon-manuals.la/v/x/"     assert [u for u, _ in sublinks(b, '<a href="y/">Y</a><a href="/v/x/">self</a>')] == [b + "y/"]

b = "https://lemon-manuals.la/v/x/"

elif cmd == "years":         make = urllib.parse.quote(canon_make(args[0]), safe="/%")         for u, label in links(BASE + f"/{make}/", fetch(BASE + f"/{make}/")):

make = urllib.parse.quote(canon_make(args[0]), safe="/%")         for u, label in links(BASE + f"/{make}/", fetch(BASE + f"/{make}/")):             if re.fullmatch(r"\d{4}", label):

for u, label in links(BASE + f"/{make}/", fetch(BASE + f"/{make}/")):

pat = re.compile(args[2], re.I) if len(args) > 2 else None         url = BASE + f"/{make}/{year}/"         body = fetch(url)

body = fetch(url)

print(render(args[0]))     elif cmd == "sub":         body = fetch(args[0])

elif cmd == "sub":         body = fetch(args[0])         for u, l in sublinks(args[0], body):

body = fetch(args[0])

print(f"{u}\t{l}")     elif cmd == "imgs":         print("\n".join(imgs_of(fetch(args[0]), args[0])))

elif cmd == "imgs":         print("\n".join(imgs_of(fetch(args[0]), args[0])))     elif cmd == "pic":

print("\n".join(imgs_of(fetch(args[0]), args[0])))

u = args[0]         os.makedirs(IMG_DIR, exist_ok=True)         req = urllib.request.Request(u, headers={"User-Agent": UA})

os.makedirs(IMG_DIR, exist_ok=True)         req = urllib.request.Request(u, headers={"User-Agent": UA})         with urllib.request.urlopen(req, timeout=120, context=_ctx()) as r:

req = urllib.request.Request(u, headers={"User-Agent": UA})

with urllib.request.urlopen(req, timeout=120, context=_ctx()) as r: