Individual skill

web-scraper

Web scraping expertise covering HTTP client selection, HTML parsing with BeautifulSoup and Cheerio, browser automation with Playwright and Puppeteer, rate limiting, robots.txt compliance, session and cookie handling, anti-detection techniques, data extraction patterns, pagination handling, and legal considerations. Use when the user asks about web scraper, web scraper best practices, or needs guidance on web scraper implementation. Do NOT use when the user needs a different specialized skill or is asking about an unrelated technology domain.

src/process/resources/skills-library/bodies/skills/software-engineering/web-scraper/SKILL.md

Source commit 4c030c7b01c241df7b6e3535635a970f10d781af

What was checked

Matches the observed repository HEAD

Yes

Scanner rules are current

No

Repository scan complete

No

Skill scan complete

No

Observed behavior

Not checked yet

Last checked

Nov 4, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.