Individual skill
evo-parallel-index-building
Provides utilities for parallelizing TF-IDF inverted index construction using a MapReduce approach - chunking documents, computing local TF and DF in worker processes, merging partial results, and finalizing TF-IDF scores with global DF.
tasks-evolved/parallel-tfidf-search/environment/skills/evo-parallel-index-building/SKILL.md
Source commit 84ca182da055fafb6453dcec4a6defdbbfd987c1
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Local scripts
- Last checked
- Nov 6, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 97
- Repository last updated
- Sep 10, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent-skills Repository topic
- ai-agents Repository topic
- llm-agents Repository topic
- open-world Repository topic
- self-evolving-agents Repository topic
- engineering Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 1,720 bytesSKILL.mdcore · 1,720 bytesscripts/utils.pyruns · 8,964 bytes
Read excerpt
# Parallel TF-IDF Index Building
## Overview
Parallelizes TF-IDF index construction using MapReduce:
1. **Chunk**: Split documents into chunks
2. **Map**: Workers compute local TF and DF per chunk
3. **Reduce**: Merge partial results (Counter.update for DF, dict.update for TF)
4. **Finalize**: Compute IDF and TF-IDF scores, build inverted index
## Key Formulas (matching sequential.py)
- TF(t,d) = count(t in d) / total_tokens(d)
- IDF(t) = log(N / df(t)) + 1
- TF-IDF = TF * IDF
- Posting lists sorted by score descending