Individual skill
light-crawl
Build, debug, and operate reproducible web/app data acquisition, including research dataset handoffs. Use for repeated extraction, pagination, JS-rendered or blocked sites, signed APIs, and verifiable or resumable crawlers. Route among HTTP/API, Scrapling, runtime inspection and packaged-app analysis as evidence requires; ordinary single-page lookups do not need a crawler.
light-crawl/SKILL.md
Source commit 6597430ad09d8f7c3518ec4cb08829be2b417e18
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 10, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 907
- Repository last updated
- Nov 8, 2026
- License
- MIT
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- academic-writing Repository topic
- ai-agent Repository topic
- claude-code Repository topic
- codex Repository topic
- llm-skills Repository topic
- paper-writing Repository topic
- research-tools Repository topic
- scientific-workflow Repository topic
- agent-skills Repository topic
- ai-skills Repository topic
- data Derived category
- research Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
.gitignoreasset · 505 bytesARCHITECTURE.mddoc · 12,324 bytesCHANGELOG.mddoc · 10,815 bytesLICENSEasset · 1,081 bytesREADME.mddoc · 14,944 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesSKILL.mdcore · 12,939 bytesTHIRD_PARTY_NOTICES.mddoc · 2,224 bytesbenchmarks/README.mddoc · 3,778 bytesbenchmarks/__init__.pyruns · 0 bytesbenchmarks/fixtures/__init__.pyruns · 0 bytesbenchmarks/fixtures/cloudflare_app/__init__.pyruns · 0 bytesbenchmarks/fixtures/cloudflare_app/server.pyruns · 3,144 bytesbenchmarks/fixtures/electron-demo/README.mddoc · 369 bytesbenchmarks/fixtures/electron-demo/index.htmlasset · 384 bytesbenchmarks/fixtures/electron-demo/main.jsruns · 613 bytesbenchmarks/fixtures/electron-demo/package.jsonruns · 175 bytesbenchmarks/fixtures/electron-demo/preload.jsruns · 165 bytesbenchmarks/fixtures/login_app/__init__.pyruns · 0 bytesbenchmarks/fixtures/login_app/server.pyruns · 4,383 bytesbenchmarks/fixtures/signed_app/__init__.pyruns · 0 bytesbenchmarks/fixtures/signed_app/server.pyruns · 5,152 bytesbenchmarks/l3_capture_check.pyruns · 2,704 bytesbenchmarks/live_check.pyruns · 2,613 bytesbenchmarks/signed_repro.pyruns · 2,675 bytesdocs/CLI.mddoc · 7,244 bytesdocs/INDEX.mddoc · 1,828 bytesdocs/INSTALL.mddoc · 4,198 bytesdocs/PACKAGING.mddoc · 2,514 bytesdocs/SCHEDULING.mddoc · 2,257 bytesdocs/STABILITY.mddoc · 2,389 bytesevaluations/2026-11-06-upgrade.mddoc · 6,887 bytesexamples/quotes/README.mddoc · 430 bytesexamples/quotes/site.jsonasset · 504 bytesexamples/site-packs/README.mddoc · 1,268 bytesexamples/site-packs/books/README.mddoc · 783 bytesexamples/site-packs/books/fixtures/request.jsonasset · 204 bytesexamples/site-packs/books/fixtures/response.htmlasset · 51,294 bytesexamples/site-packs/books/fixtures/response.meta.jsonasset · 441 bytesexamples/site-packs/books/repro/replay.pyruns · 4,835 bytesexamples/site-packs/books/site.jsonasset · 747 bytesexamples/site-packs/books/spider/site.jsonasset · 637 bytesexamples/site-packs/books/spider/spider.pyruns · 5,236 bytesexamples/site-packs/books/verify.pyruns · 1,384 bytesexamples/site-packs/github-api/README.mddoc · 711 bytesexamples/site-packs/github-api/fixtures/request.jsonasset · 227 bytesexamples/site-packs/github-api/fixtures/response.jsonasset · 6,608 bytesexamples/site-packs/github-api/fixtures/response.meta.jsonasset · 486 bytesexamples/site-packs/github-api/site.jsonasset · 717 bytesexamples/site-packs/github-api/spider/site.jsonasset · 264 bytesexamples/site-packs/github-api/spider/spider.pyruns · 1,766 bytesexamples/site-packs/github-api/verify.pyruns · 1,243 bytesexamples/site-packs/login-local/README.mddoc · 941 bytesexamples/site-packs/login-local/fixtures/private.jsonasset · 161 bytesexamples/site-packs/login-local/repro/__init__.pyruns · 0 bytesexamples/site-packs/login-local/repro/login.pyruns · 1,105 bytesexamples/site-packs/login-local/repro/session.pyruns · 4,836 bytesexamples/site-packs/login-local/site.jsonasset · 663 bytesexamples/site-packs/login-local/verify.pyruns · 3,120 bytesexamples/site-packs/quotes/README.mddoc · 1,285 bytesexamples/site-packs/quotes/fixtures/request.jsonasset · 263 bytesexamples/site-packs/quotes/fixtures/response.htmlasset · 11,013 bytesexamples/site-packs/quotes/fixtures/response.meta.jsonasset · 1,068 bytesexamples/site-packs/quotes/repro/replay.pyruns · 4,835 bytesexamples/site-packs/quotes/site.jsonasset · 905 bytesexamples/site-packs/quotes/spider/site.jsonasset · 504 bytesexamples/site-packs/quotes/spider/spider.pyruns · 5,236 bytesexamples/site-packs/quotes/verify.pyruns · 1,334 bytesexamples/site-packs/signed-local/README.mddoc · 830 bytesexamples/site-packs/signed-local/fixtures/response.jsonasset · 142 bytesexamples/site-packs/signed-local/repro/__init__.pyruns · 0 bytesexamples/site-packs/signed-local/repro/sign.pyruns · 1,338 bytesexamples/site-packs/signed-local/site.jsonasset · 706 bytesexamples/site-packs/signed-local/verify.pyruns · 1,606 bytesinstall.ps1runs · 7,335 bytesinstall.shruns · 5,922 bytespyproject.tomlasset · 1,102 bytesreferences/acquisition.mddoc · 5,719 bytesreferences/deep-targets.mddoc · 4,099 bytesreferences/evidence.mddoc · 5,045 bytesreferences/learning.mddoc · 4,063 bytesreferences/project-layout.mddoc · 6,417 bytesreferences/research-data.mddoc · 7,231 bytesreferences/routing.mddoc · 4,648 bytesreferences/runtime-reverse.mddoc · 5,635 bytesreferences/safety.mddoc · 4,569 bytesrun_light_crawl.pyruns · 399 bytesscripts/bootstrap.ps1runs · 1,492 bytesscripts/bootstrap.shruns · 536 bytesscripts/ci.ps1runs · 1,864 bytesscripts/ci.shruns · 1,439 bytesscripts/drift.ps1runs · 3,044 bytesscripts/light-crawl.ps1runs · 776 bytesscripts/light-crawl.shruns · 909 bytesscripts/package.pyruns · 8,843 bytesscripts/rea.ps1runs · 389 bytesscripts/rea.shruns · 210 bytesscripts/register-mcp.ps1runs · 4,151 bytesscripts/register_mcp.pyruns · 5,136 bytesscripts/release.ps1runs · 797 bytesscripts/schedule-drift.ps1runs · 1,510 bytesscripts/smoke.ps1runs · 639 bytessrc/light_crawl/__init__.pyruns · 159 bytessrc/light_crawl/__main__.pyruns · 195 bytessrc/light_crawl/adapters/__init__.pyruns · 369 bytessrc/light_crawl/adapters/js_reverse.pyruns · 1,983 bytessrc/light_crawl/adapters/rea.pyruns · 1,932 bytessrc/light_crawl/adapters/scrapling.pyruns · 1,537 bytessrc/light_crawl/benchmark.pyruns · 9,653 bytessrc/light_crawl/capture.pyruns · 12,618 bytessrc/light_crawl/cli.pyruns · 41,971 bytessrc/light_crawl/dataset.pyruns · 11,550 bytessrc/light_crawl/doctor.pyruns · 8,462 bytessrc/light_crawl/integrity.pyruns · 1,775 bytessrc/light_crawl/learn.pyruns · 4,906 bytessrc/light_crawl/metrics.pyruns · 2,384 bytessrc/light_crawl/migrate.pyruns · 2,384 bytessrc/light_crawl/models.pyruns · 10,163 bytessrc/light_crawl/probe.pyruns · 14,209 bytessrc/light_crawl/py.typedasset · 0 bytessrc/light_crawl/report.pyruns · 7,749 bytessrc/light_crawl/router.pyruns · 8,410 bytessrc/light_crawl/scaffold.pyruns · 3,159 bytessrc/light_crawl/sessions.pyruns · 5,208 bytessrc/light_crawl/site.pyruns · 10,833 bytessrc/light_crawl/state.pyruns · 10,084 bytessrc/light_crawl/utils.pyruns · 6,021 bytessrc/light_crawl/validation.pyruns · 4,888 bytessrc/light_crawl/verify.pyruns · 11,514 bytestemplates/case/scope.mddoc · 1,185 bytestemplates/case/target.jsonasset · 243 bytestemplates/repro/README.mddoc · 647 bytestemplates/repro/replay.pyruns · 4,835 bytestemplates/repro/request.example.jsonasset · 201 bytestemplates/repro/sign_lab.pyruns · 4,251 bytestemplates/site/site.jsonasset · 551 bytestemplates/spider/README.mddoc · 1,115 bytestemplates/spider/generic_spider.pyruns · 5,236 bytestemplates/spider/stealth_spider.pyruns · 7,537 bytestemplates/verify/verify_case.pyruns · 566 bytestests/test_benchmark.pyruns · 3,434 bytestests/test_capture.pyruns · 4,072 bytestests/test_cli.pyruns · 6,375 bytestests/test_dataset.pyruns · 8,252 bytestests/test_integrity.pyruns · 7,138 bytestests/test_l3_capture.pyruns · 810 bytestests/test_learn.pyruns · 2,880 bytestests/test_live_fixtures.pyruns · 822 bytestests/test_login_encryption.pyruns · 1,827 bytestests/test_metrics.pyruns · 2,436 bytestests/test_migrate.pyruns · 2,508 bytestests/test_probe.pyruns · 5,353 bytestests/test_register_mcp.pyruns · 2,786 bytestests/test_research_flow.pyruns · 6,047 bytestests/test_router.pyruns · 4,047 bytestests/test_sessions.pyruns · 2,778 bytestests/test_signed_repro.pyruns · 956 bytestests/test_site_check.pyruns · 2,399 bytestests/test_site_output.pyruns · 2,033 bytestests/test_site_pack.pyruns · 2,271 bytestests/test_site_run.pyruns · 2,372 bytestests/test_site_runtime.pyruns · 1,281 bytestests/test_state.pyruns · 3,058 bytestests/test_validation.pyruns · 4,498 bytestests/test_verify.pyruns · 3,210 bytes
Read excerpt
# light-crawl
## 项目目录与续作
需要落盘或继续已有项目时,先读 [项目目录约定](references/project-layout.md),按其中的本阶段位置、版本与关联规则保存;用户指定路径和旧项目优先。包内资源相对本 skill,研究产物相对真实项目,二者不要混用。只做局部讨论时无需初始化目录。
A routing-first crawler skill. The agent's job is to **compile a target into a
deterministic scraper**, not to click through every page in a chat loop.
For research, first identify the caller's actual data gap and prefer an existing
dataset, lawful download or documented API when sufficient. Read
[research-data.md](references/research-data.md) when collecting research inputs
or handing data to Literature, Design, Experiment or Analysis. Acquisition is
an on-demand service in the research network, not a mandatory stage or evidence