Individual skill
vs-crawler
Crawl websites (news, blogs, papers, GitHub, product docs, RSS feeds) into a fixed-schema JSONL file, then create a dataset and a searchable application in Viking AI Search. Supports one-time crawl and scheduled recurring crawl with automatic incremental sync.
skills/vs-crawler/SKILL.md
Source commit cba429a719e37ee744f7afa1e42c6538babd1d65
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 3, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 1,191
- Repository last updated
- Oct 16, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent-tools Repository topic
- ai-agents Repository topic
- ai-search Repository topic
- ai-search-engine Repository topic
- automation Repository topic
- cli Repository topic
- command-line-tool Repository topic
- conversational-search Repository topic
- information-retrieval Repository topic
- rag Repository topic
- docs Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
Read excerpt
# Viking Content Crawler
## When to Use
Use this skill when the user wants to crawl content from websites and import it into Viking AI Search to build a searchable knowledge base. This covers news sites, blogs, academic papers, GitHub repositories, product documentation, RSS feeds, and similar web content sources.
The agent writes crawler code tailored to the target sites, outputs data in a fixed JSONL schema, and then hands off to the `vs-item-onboarding` skill for dataset creation and import.
Do not use this skill when:
- The user already has a local file ready to import (use `vs-item-onboarding` directly).
- The user wants to import from a database (use `vs-item-onboarding` directly with MySQL).
## Version Check