Not in the top 1,000 by GitHub stars
FrontisAI/NatureBench
NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?
History
Review weekly changes to SKILL.md before updating. Dates mark the weeks in which we observed each version; checks apply to the recorded commits.
View weekly changes
naturegym/.agents/skills/data-check/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/data-verify/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/dockerfile-fix/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/filter-verify/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Instructions only
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/paper-filter/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/paper-preprocess/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/task-build/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/task-fix/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/task-verify/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.agents/skills/verify-apply/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/data-check/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/data-verify/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/dockerfile-fix/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/filter-verify/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Instructions only
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/paper-filter/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/paper-preprocess/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/task-build/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/task-fix/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/task-verify/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
naturegym/.claude/skills/verify-apply/SKILL.md
Week of Oct 26, 2026 · Line counts unavailable
Before Not checked yet → After Network or broad shell
First observation → f92b849bc8d6
No earlier version was observed for comparison.
Heat
1.2- Star growth, last 7 daysNot available
- Star growth, last 24 hoursNot available
- Rank movement, last 7 daysNot available
- Commits, last 30 daysNot available
- Accounts asking about it through MCP, last 7 daysNot available
- Hot votes this weekNot available
Heat never changes the order by stars, and sponsorship never touches it. How heat works
Stars over time
30-day change not yet available: not enough history.
Individual skills in this repository
- data-check
naturegym/.agents/skills/data-check - data-verify
naturegym/.agents/skills/data-verify - dockerfile-fix
naturegym/.agents/skills/dockerfile-fix - filter-verify
naturegym/.agents/skills/filter-verify - paper-filter
naturegym/.agents/skills/paper-filter - paper-preprocess
naturegym/.agents/skills/paper-preprocess - task-build
naturegym/.agents/skills/task-build - task-fix
naturegym/.agents/skills/task-fix - task-verify
naturegym/.agents/skills/task-verify - verify-apply
naturegym/.agents/skills/verify-apply - data-check
naturegym/.claude/skills/data-check - data-verify
naturegym/.claude/skills/data-verify - dockerfile-fix
naturegym/.claude/skills/dockerfile-fix - filter-verify
naturegym/.claude/skills/filter-verify - paper-filter
naturegym/.claude/skills/paper-filter - paper-preprocess
naturegym/.claude/skills/paper-preprocess - task-build
naturegym/.claude/skills/task-build - task-fix
naturegym/.claude/skills/task-fix - task-verify
naturegym/.claude/skills/task-verify - verify-apply
naturegym/.claude/skills/verify-apply
How it works
data-check
When your agent uses it
Acquire and verify data for CNS papers that passed paper-filter. Clones repositories, downloads and organizes datasets per evaluation setting, performs Level 4 deep verification (acquisition integrity, initial state, separability, consistency, per-setting completeness), creates an organized data environment. Updates filter_result.json with verification results, acquisition details, and task_info corrections.
What it loads into context
Frontmatter
103 tokensAlways available to the agent
SKILL.md
3,482 tokensWhen the skill is activated
Reference documents
15,689 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Context
- Input Requirements
- Output
- Workflow
- Judgment Principles
- Using inspect_data.py
Scripts executed separately
scripts/inspect_data.py
Files
SKILL.mdcore15,870 bytesSKILL.mdcore15,870 bytesreferences/acquisition_guide.mddoc16,978 bytesreferences/check_rules.mddoc29,108 bytesreferences/core_definitions.mddoc7,055 bytesreferences/inspect_data.mddoc4,594 bytesreferences/output_schema.mddoc12,806 bytesscripts/inspect_data.pyruns27,611 bytes
Read SKILL.md
# Data-Check Skill
Downloads repositories and datasets referenced by papers that passed paper-filter, then performs Level 4 deep verification — checking acquisition integrity, initial state, separability, consistency, per-setting completeness. Results are written back into `filter_result.json`, providing a verified data foundation for downstream task package construction.
## Context
paper-filter's Level 3 (Data Completeness) is **lightweight** — it checks link accessibility, views directory structures, and reads READMEs, but does **not actually download data**. This skill performs the actual download and deep verification, then writes results back into `filter_result.json` as **Level 4**.
### Terminology Reference
This skill uses the same core definitions as paper-filter. See [references/core_definitions.md](references/core_definitions.md) for formal definitions of:
- **T = (A, Data, M, S, B)**: The ML task tuple
- **D_dev**: Development data space (all prior information available to the original authors for solving the problem)
- **D_eval = (X_test, Y_ref)**: Evaluation data space with formal input/reference decompositionView the complete source on GitHubdata-verify
When your agent uses it
Verify data components in filter_result.json after data-check. Checks Algorithm A boundary, evaluation setting validity, separability correctness, data directory integrity, and description consistency. Outputs a verification report for human review — does not modify filter_result.json or data directories.
What it loads into context
Frontmatter
106 tokensAlways available to the agent
SKILL.md
1,785 tokensWhen the skill is activated
Reference documents
8,591 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Context
- Input Requirements
- Workflow
- Judgment Principles
- Output Files
Files
SKILL.mdcore8,471 bytesSKILL.mdcore8,471 bytesSKILL.mdcore8,471 bytesreferences/check_rules.mddoc31,919 bytesreferences/output_schema.mddoc7,608 bytes
Read SKILL.md
# Data-Verify Skill
Verify data components produced by paper-filter and data-check. Outputs a verification report (`data_verify_result.json`) for human review — does **not** modify `filter_result.json` or data directories.
## Context
paper-filter (Level 1-3) and data-check (Level 4) run automatically and may produce systematic errors in data component classification, evaluation setting identification, and separability descriptions. This skill performs independent verification and outputs an actionable report.
### Core Principle
**Provide Algorithm A with the same initial conditions it had, but do not include any of A's own operations or outputs.**
Why: The goal is not to reproduce the paper's algorithm — it is to test whether a Solver can independently solve the same ML problem using any method, potentially surpassing the original. The benchmark gives the Solver the same starting point as the paper's authors, then scores its output against ground truth using the paper's metrics, with the paper's reported scores as baselines for comparison. Including A's outputs would leak solutions; excluding legitimate initial data would make the task unfairly harder.
View the complete source on GitHubdockerfile-fix
When your agent uses it
Fix Dockerfile issues identified by batch_verify.sh. Reads verify_result.txt to diagnose import failures, version conflicts, and missing dependencies, then repairs the Dockerfile and updates packages.json. May also fix scripts (Script-First Rule) when packages are unavailable.
What it loads into context
Frontmatter
87 tokensAlways available to the agent
SKILL.md
3,477 tokensWhen the skill is activated
Reference documents
0 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input
- Prerequisites
- Reference Materials
- Procedure
- Critical Rules
Files
SKILL.mdcore14,762 bytes
Read SKILL.md
# Dockerfile Fix Skill
Fix Dockerfile issues discovered by runtime verification (batch_verify.sh). This skill diagnoses actual import failures, version conflicts, and missing dependencies from `verify_result.txt`, then repairs the Dockerfile, updates `packages.json`, and optionally fixes scripts when packages are unavailable.
## Input
User provides:
1. **task_package_path**: Path to the task package directory (containing `problem/`, `evaluation/`, `environment/`)
2. **dockerfile_name** (optional): Name of the Dockerfile to fix. Default: `Dockerfile.v3`. Can also be `Dockerfile` or any other name.
## Prerequisites
The following files must exist in `{task_package_path}/environment/`:
- `verify_result.txt` — output from batch_verify.sh, with status "has failures" or "build failed"View the complete source on GitHubfilter-verify
When your agent uses it
Verify paper-filter results by checking pass/reject judgment correctness, validating task_info field accuracy, and detecting internal inconsistencies. Outputs actionable corrections to filter_result.json.
What it loads into context
Frontmatter
60 tokensAlways available to the agent
SKILL.md
1,139 tokensWhen the skill is activated
Reference documents
6,920 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Context
- Workflow
- Judgment Principles
Files
SKILL.mdcore5,473 bytesreferences/check_rules.mddoc28,593 bytesreferences/output_schema.mddoc3,422 bytes
Read SKILL.md
# Filter-Verify Skill
Verify filter_result.json produced by paper-filter. Three core responsibilities:
1. Check whether the pass/reject judgment is correct
2. Validate that task_info fields are accurate and complete
3. Produce an actionable corrections list to drive filter_result.json updates
## Input Requirements
Paper directory must contain:
- `preprocessed/text.md`: Full paper text
- `preprocessed/links.json`: Extracted links
- `preprocessed/figures/`: Extracted figures and tables
- `filter_result.json`: paper-filter outputView the complete source on GitHubpaper-filter
When your agent uses it
Filter CNS papers for suitability in extracting machine learning tasks. Uses a three-level cascade funnel (Task Nature → Evaluation System → Data Completeness) with progressive filtering; stops immediately upon rejection. Input is preprocessed paper data; output is the filtering decision result.
What it loads into context
Frontmatter
90 tokensAlways available to the agent
SKILL.md
1,304 tokensWhen the skill is activated
Reference documents
13,285 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Supplementary Materials
- Workflow
- Judgment Principles
- Output Files
Scripts executed separately
scripts/estimate_size.pyscripts/validate_links.py
Files
SKILL.mdcore5,930 bytesSKILL.mdcore5,930 bytesSKILL.mdcore5,930 bytesSKILL.mdcore5,930 bytesreferences/core_definitions.mddoc7,055 bytesreferences/estimate_size_usage.mddoc4,618 bytesreferences/level1_rules.mddoc7,567 bytesreferences/level2_rules.mddoc6,304 bytesreferences/level3_rules.mddoc19,010 bytesreferences/output_schema.mddoc12,330 bytesreferences/validate_links_usage.mddoc4,211 bytesscripts/estimate_size.pyruns20,864 bytesscripts/validate_links.pyruns22,259 bytes
Read SKILL.md
# Paper Filter Skill
Filter CNS papers for machine learning task extraction suitability.
## Input Requirements
Before invoking this skill, provide:
1. **Paper Folder Path**: Directory containing original paper files and preprocessed data
2. **Output Directory**: Directory to store filtering results
Paper folder structure:
- `{paper_id}.pdf`: Original paper PDF file
- `{paper_id}.html`: HTML version of the paper
- `preprocessed/`: Preprocessed data subdirectory, containing:View the complete source on GitHubpaper-preprocess
When your agent uses it
Preprocessing CNS papers (PDF + HTML) to extract text, figures, and links. Outputs structured data for use by the paper-filter skill.
What it loads into context
Frontmatter
55 tokensAlways available to the agent
SKILL.md
714 tokensWhen the skill is activated
Reference documents
0 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Workflow
- Usage
- Output Structure
- Dependencies
Scripts executed separately
scripts/extract_figures.pyscripts/extract_links.pyscripts/extract_text.pyscripts/preprocess.py
Files
SKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesscripts/extract_figures.pyruns13,508 bytesscripts/extract_links.pyruns7,883 bytesscripts/extract_text.pyruns9,636 bytesscripts/preprocess.pyruns6,134 bytes
Read SKILL.md
# Paper Preprocess Skill
Preprocess CNS papers using HTML (text/links) and PDF (figures) to extract structured data.
## Input Requirements
Before invoking this skill, provide:
1. **Input Directory**: A directory containing both `*.pdf` and `*.html` files for the same paper (Nature/Springer structured HTML). The folder name is used as `paper_id`.
2. **Output Directory**: The base directory to store preprocessing results
## Workflow
### Step 1: Extract Text (from HTML)
View the complete source on GitHubtask-build
When your agent uses it
Build a structured task package from a paper that has passed paper-preprocess, paper-filter, and data-check. Reorganizes data, generates problem description, evaluator, metadata, and environment Dockerfile.v3.
What it loads into context
Frontmatter
76 tokensAlways available to the agent
SKILL.md
3,337 tokensWhen the skill is activated
Reference documents
23,439 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Output Structure
- Core Principles
- Workflow
Files
SKILL.mdcore14,773 bytesreferences/Dockerfile.base.v3asset4,923 bytesreferences/base_packages.jsonasset2,777 bytesreferences/data_description_guide.mddoc4,121 bytesreferences/data_organization_guide.mddoc14,792 bytesreferences/environment_guide.mddoc45,105 bytesreferences/evaluator_guide.mddoc13,784 bytesreferences/metadata_guide.mddoc10,312 bytesreferences/readme_guide.mddoc8,462 bytes
Read SKILL.md
# Task Build Skill
Build a structured ML benchmark task package from a CNS paper that has completed the three preprocessing stages (paper-preprocess → paper-filter → data-check).
## Input Requirements
Before invoking this skill, provide:
1. **Paper Folder Path**: Directory containing the paper and all prior processing results
2. **Output Directory**: Where to build the final task package (can be same as paper folder or a new location)
Paper folder must contain:
- `{paper_id}.pdf` and `{paper_id}.html`: Original paper files
- `preprocessed/`: paper-preprocess output (`text.md`, `links.json`, `figures/`, `tables/`)
- `filter_result.json`: Combined output from paper-filter + data-check (must have `final_result.data_check_passed == true`)View the complete source on GitHubtask-fix
When your agent uses it
Fix issues identified by task-verify in a task package. Reads task_verify_result.json, prioritizes failed checks over warnings, applies targeted fixes following task-build rules, re-runs dynamic testing, and outputs a fix log.
What it loads into context
Frontmatter
80 tokensAlways available to the agent
SKILL.md
2,641 tokensWhen the skill is activated
Reference documents
8,431 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Output
- Core Principles
- Workflow
- Escalation: Issues That Cannot Be Auto-Fixed
Files
SKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesreferences/fix_rules.mddoc37,184 bytes
Read SKILL.md
# Task-Fix Skill
Fix issues in a task package identified by task-verify, following task-build rules and conventions.
## Input Requirements
Before invoking this skill, provide:
1. **Task Package Path**: Directory containing the task package (`problem/`, `evaluation/`, `environment/`, `metadata.json`)
2. **Paper Folder Path**: Directory containing the original paper and prior processing results (needed for context)
The task package directory must contain:
- `task_verify_result.json`: Output from task-verify (the issues to fix)
The paper folder must contain:View the complete source on GitHubtask-verify
When your agent uses it
Verify task packages built by task-build. Checks document consistency, benchmark design principles, and performs dynamic testing with a baseline solver to ensure the task package is valid and usable.
What it loads into context
Frontmatter
89 tokensAlways available to the agent
SKILL.md
1,273 tokensWhen the skill is activated
Reference documents
12,602 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Verification Workflow
- Output
- Critical Rules
- Execution Notes
Scripts executed separately
scripts/run_baseline_test.py
Files
SKILL.mdcore5,661 bytesSKILL.mdcore5,661 bytesreferences/baseline_solver_guide.mddoc7,517 bytesreferences/output_schema.mddoc4,588 bytesreferences/verification_rules.mddoc42,141 bytesscripts/run_baseline_test.pyruns4,313 bytes
Read SKILL.md
# Task-Verify Skill
Verify that a task package built by task-build is valid, consistent, and usable as a benchmark.
## Input Requirements
User provides:
- **task_package_path**: Path to task package directory containing problem/, evaluation/, metadata.json
Optional:
- **filter_result.json**: If present in task package, used for algorithm name verification (C2.3, C2.4)
## Verification Workflow
View the complete source on GitHubverify-apply
When your agent uses it
Apply corrections from filter-verify (verification_result.json) or data-verify (data_verify_result.json) to filter_result.json. User specifies source type explicitly. Applies field-level and structural corrections, performs inline verification for new/modified settings, writes change log.
What it loads into context
Frontmatter
172 tokensAlways available to the agent
SKILL.md
795 tokensWhen the skill is activated
Reference documents
2,409 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Core Principle: Pipeline Continuity
- Input Requirements
- Workflow
Files
SKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesreferences/apply_rules.mddoc10,436 bytes
Read SKILL.md
# Verify-Apply Skill
Apply verification corrections to `filter_result.json`. Reads a verification result JSON, applies every correction/finding, and writes a change log.
## Core Principle: Pipeline Continuity
**After apply, `filter_result.json` must be directly consumable by the next pipeline stage — no re-runs of previous skills required.**
- filter-verify apply → output must match paper-filter output structure (ready for data-check)
- data-verify apply → output must match data-check output structure (ready for task-build)
This means any new or structurally modified evaluation setting must have the same completeness as what the corresponding upstream skill would produce.
> **Note:** If any concepts, operations, requirements, or explanations are unclear during implementation or adjustment, please refer to the documentation in the `paper-filter` and `data-check` skills for detailed guidance.View the complete source on GitHubdata-check
When your agent uses it
Acquire and verify data for CNS papers that passed paper-filter. Clones repositories, downloads and organizes datasets per evaluation setting, performs Level 4 deep verification (acquisition integrity, initial state, separability, consistency, per-setting completeness), creates an organized data environment. Updates filter_result.json with verification results, acquisition details, and task_info corrections.
What it loads into context
Frontmatter
103 tokensAlways available to the agent
SKILL.md
3,482 tokensWhen the skill is activated
Reference documents
15,689 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Context
- Input Requirements
- Output
- Workflow
- Judgment Principles
- Using inspect_data.py
Scripts executed separately
scripts/inspect_data.py
Files
SKILL.mdcore15,870 bytesSKILL.mdcore15,870 bytesreferences/acquisition_guide.mddoc16,978 bytesreferences/check_rules.mddoc29,108 bytesreferences/core_definitions.mddoc7,055 bytesreferences/inspect_data.mddoc4,594 bytesreferences/output_schema.mddoc12,806 bytesscripts/inspect_data.pyruns27,611 bytes
Read SKILL.md
# Data-Check Skill
Downloads repositories and datasets referenced by papers that passed paper-filter, then performs Level 4 deep verification — checking acquisition integrity, initial state, separability, consistency, per-setting completeness. Results are written back into `filter_result.json`, providing a verified data foundation for downstream task package construction.
## Context
paper-filter's Level 3 (Data Completeness) is **lightweight** — it checks link accessibility, views directory structures, and reads READMEs, but does **not actually download data**. This skill performs the actual download and deep verification, then writes results back into `filter_result.json` as **Level 4**.
### Terminology Reference
This skill uses the same core definitions as paper-filter. See [references/core_definitions.md](references/core_definitions.md) for formal definitions of:
- **T = (A, Data, M, S, B)**: The ML task tuple
- **D_dev**: Development data space (all prior information available to the original authors for solving the problem)
- **D_eval = (X_test, Y_ref)**: Evaluation data space with formal input/reference decompositionView the complete source on GitHubdata-verify
When your agent uses it
Verify data components in filter_result.json after data-check. Checks Algorithm A boundary, evaluation setting validity, separability correctness, data directory integrity, and description consistency. Outputs a verification report for human review — does not modify filter_result.json or data directories.
What it loads into context
Frontmatter
106 tokensAlways available to the agent
SKILL.md
1,785 tokensWhen the skill is activated
Reference documents
8,591 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Context
- Input Requirements
- Workflow
- Judgment Principles
- Output Files
Files
SKILL.mdcore8,471 bytesSKILL.mdcore8,471 bytesSKILL.mdcore8,471 bytesreferences/check_rules.mddoc31,919 bytesreferences/output_schema.mddoc7,608 bytes
Read SKILL.md
# Data-Verify Skill
Verify data components produced by paper-filter and data-check. Outputs a verification report (`data_verify_result.json`) for human review — does **not** modify `filter_result.json` or data directories.
## Context
paper-filter (Level 1-3) and data-check (Level 4) run automatically and may produce systematic errors in data component classification, evaluation setting identification, and separability descriptions. This skill performs independent verification and outputs an actionable report.
### Core Principle
**Provide Algorithm A with the same initial conditions it had, but do not include any of A's own operations or outputs.**
Why: The goal is not to reproduce the paper's algorithm — it is to test whether a Solver can independently solve the same ML problem using any method, potentially surpassing the original. The benchmark gives the Solver the same starting point as the paper's authors, then scores its output against ground truth using the paper's metrics, with the paper's reported scores as baselines for comparison. Including A's outputs would leak solutions; excluding legitimate initial data would make the task unfairly harder.
View the complete source on GitHubdockerfile-fix
When your agent uses it
Fix Dockerfile issues identified by batch_verify.sh. Reads verify_result.txt to diagnose import failures, version conflicts, and missing dependencies, then repairs the Dockerfile and updates packages.json. May also fix scripts (Script-First Rule) when packages are unavailable.
What it loads into context
Frontmatter
87 tokensAlways available to the agent
SKILL.md
3,477 tokensWhen the skill is activated
Reference documents
0 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input
- Prerequisites
- Reference Materials
- Procedure
- Critical Rules
Files
SKILL.mdcore14,762 bytes
Read SKILL.md
# Dockerfile Fix Skill
Fix Dockerfile issues discovered by runtime verification (batch_verify.sh). This skill diagnoses actual import failures, version conflicts, and missing dependencies from `verify_result.txt`, then repairs the Dockerfile, updates `packages.json`, and optionally fixes scripts when packages are unavailable.
## Input
User provides:
1. **task_package_path**: Path to the task package directory (containing `problem/`, `evaluation/`, `environment/`)
2. **dockerfile_name** (optional): Name of the Dockerfile to fix. Default: `Dockerfile.v3`. Can also be `Dockerfile` or any other name.
## Prerequisites
The following files must exist in `{task_package_path}/environment/`:
- `verify_result.txt` — output from batch_verify.sh, with status "has failures" or "build failed"View the complete source on GitHubfilter-verify
When your agent uses it
Verify paper-filter results by checking pass/reject judgment correctness, validating task_info field accuracy, and detecting internal inconsistencies. Outputs actionable corrections to filter_result.json.
What it loads into context
Frontmatter
60 tokensAlways available to the agent
SKILL.md
1,139 tokensWhen the skill is activated
Reference documents
6,920 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Context
- Workflow
- Judgment Principles
Files
SKILL.mdcore5,473 bytesreferences/check_rules.mddoc28,593 bytesreferences/output_schema.mddoc3,422 bytes
Read SKILL.md
# Filter-Verify Skill
Verify filter_result.json produced by paper-filter. Three core responsibilities:
1. Check whether the pass/reject judgment is correct
2. Validate that task_info fields are accurate and complete
3. Produce an actionable corrections list to drive filter_result.json updates
## Input Requirements
Paper directory must contain:
- `preprocessed/text.md`: Full paper text
- `preprocessed/links.json`: Extracted links
- `preprocessed/figures/`: Extracted figures and tables
- `filter_result.json`: paper-filter outputView the complete source on GitHubpaper-filter
When your agent uses it
Filter CNS papers for suitability in extracting machine learning tasks. Uses a three-level cascade funnel (Task Nature → Evaluation System → Data Completeness) with progressive filtering; stops immediately upon rejection. Input is preprocessed paper data; output is the filtering decision result.
What it loads into context
Frontmatter
90 tokensAlways available to the agent
SKILL.md
1,304 tokensWhen the skill is activated
Reference documents
13,285 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Supplementary Materials
- Workflow
- Judgment Principles
- Output Files
Scripts executed separately
scripts/estimate_size.pyscripts/validate_links.py
Files
SKILL.mdcore5,930 bytesSKILL.mdcore5,930 bytesSKILL.mdcore5,930 bytesSKILL.mdcore5,930 bytesreferences/core_definitions.mddoc7,055 bytesreferences/estimate_size_usage.mddoc4,618 bytesreferences/level1_rules.mddoc7,567 bytesreferences/level2_rules.mddoc6,304 bytesreferences/level3_rules.mddoc19,010 bytesreferences/output_schema.mddoc12,330 bytesreferences/validate_links_usage.mddoc4,211 bytesscripts/estimate_size.pyruns20,864 bytesscripts/validate_links.pyruns22,259 bytes
Read SKILL.md
# Paper Filter Skill
Filter CNS papers for machine learning task extraction suitability.
## Input Requirements
Before invoking this skill, provide:
1. **Paper Folder Path**: Directory containing original paper files and preprocessed data
2. **Output Directory**: Directory to store filtering results
Paper folder structure:
- `{paper_id}.pdf`: Original paper PDF file
- `{paper_id}.html`: HTML version of the paper
- `preprocessed/`: Preprocessed data subdirectory, containing:View the complete source on GitHubpaper-preprocess
When your agent uses it
Preprocessing CNS papers (PDF + HTML) to extract text, figures, and links. Outputs structured data for use by the paper-filter skill.
What it loads into context
Frontmatter
55 tokensAlways available to the agent
SKILL.md
714 tokensWhen the skill is activated
Reference documents
0 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Workflow
- Usage
- Output Structure
- Dependencies
Scripts executed separately
scripts/extract_figures.pyscripts/extract_links.pyscripts/extract_text.pyscripts/preprocess.py
Files
SKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesSKILL.mdcore3,041 bytesscripts/extract_figures.pyruns13,508 bytesscripts/extract_links.pyruns7,883 bytesscripts/extract_text.pyruns9,636 bytesscripts/preprocess.pyruns6,134 bytes
Read SKILL.md
# Paper Preprocess Skill
Preprocess CNS papers using HTML (text/links) and PDF (figures) to extract structured data.
## Input Requirements
Before invoking this skill, provide:
1. **Input Directory**: A directory containing both `*.pdf` and `*.html` files for the same paper (Nature/Springer structured HTML). The folder name is used as `paper_id`.
2. **Output Directory**: The base directory to store preprocessing results
## Workflow
### Step 1: Extract Text (from HTML)
View the complete source on GitHubtask-build
When your agent uses it
Build a structured task package from a paper that has passed paper-preprocess, paper-filter, and data-check. Reorganizes data, generates problem description, evaluator, metadata, and environment Dockerfile.v3.
What it loads into context
Frontmatter
76 tokensAlways available to the agent
SKILL.md
3,337 tokensWhen the skill is activated
Reference documents
23,439 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Output Structure
- Core Principles
- Workflow
Files
SKILL.mdcore14,773 bytesreferences/Dockerfile.base.v3asset4,923 bytesreferences/base_packages.jsonasset2,777 bytesreferences/data_description_guide.mddoc4,121 bytesreferences/data_organization_guide.mddoc14,792 bytesreferences/environment_guide.mddoc45,105 bytesreferences/evaluator_guide.mddoc13,784 bytesreferences/metadata_guide.mddoc10,312 bytesreferences/readme_guide.mddoc8,462 bytes
Read SKILL.md
# Task Build Skill
Build a structured ML benchmark task package from a CNS paper that has completed the three preprocessing stages (paper-preprocess → paper-filter → data-check).
## Input Requirements
Before invoking this skill, provide:
1. **Paper Folder Path**: Directory containing the paper and all prior processing results
2. **Output Directory**: Where to build the final task package (can be same as paper folder or a new location)
Paper folder must contain:
- `{paper_id}.pdf` and `{paper_id}.html`: Original paper files
- `preprocessed/`: paper-preprocess output (`text.md`, `links.json`, `figures/`, `tables/`)
- `filter_result.json`: Combined output from paper-filter + data-check (must have `final_result.data_check_passed == true`)View the complete source on GitHubtask-fix
When your agent uses it
Fix issues identified by task-verify in a task package. Reads task_verify_result.json, prioritizes failed checks over warnings, applies targeted fixes following task-build rules, re-runs dynamic testing, and outputs a fix log.
What it loads into context
Frontmatter
80 tokensAlways available to the agent
SKILL.md
2,641 tokensWhen the skill is activated
Reference documents
8,431 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Output
- Core Principles
- Workflow
- Escalation: Issues That Cannot Be Auto-Fixed
Files
SKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesSKILL.mdcore11,326 bytesreferences/fix_rules.mddoc37,184 bytes
Read SKILL.md
# Task-Fix Skill
Fix issues in a task package identified by task-verify, following task-build rules and conventions.
## Input Requirements
Before invoking this skill, provide:
1. **Task Package Path**: Directory containing the task package (`problem/`, `evaluation/`, `environment/`, `metadata.json`)
2. **Paper Folder Path**: Directory containing the original paper and prior processing results (needed for context)
The task package directory must contain:
- `task_verify_result.json`: Output from task-verify (the issues to fix)
The paper folder must contain:View the complete source on GitHubtask-verify
When your agent uses it
Verify task packages built by task-build. Checks document consistency, benchmark design principles, and performs dynamic testing with a baseline solver to ensure the task package is valid and usable.
What it loads into context
Frontmatter
89 tokensAlways available to the agent
SKILL.md
1,273 tokensWhen the skill is activated
Reference documents
12,602 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Input Requirements
- Verification Workflow
- Output
- Critical Rules
- Execution Notes
Scripts executed separately
scripts/run_baseline_test.py
Files
SKILL.mdcore5,661 bytesSKILL.mdcore5,661 bytesreferences/baseline_solver_guide.mddoc7,517 bytesreferences/output_schema.mddoc4,588 bytesreferences/verification_rules.mddoc42,141 bytesscripts/run_baseline_test.pyruns4,313 bytes
Read SKILL.md
# Task-Verify Skill
Verify that a task package built by task-build is valid, consistent, and usable as a benchmark.
## Input Requirements
User provides:
- **task_package_path**: Path to task package directory containing problem/, evaluation/, metadata.json
Optional:
- **filter_result.json**: If present in task package, used for algorithm name verification (C2.3, C2.4)
## Verification Workflow
View the complete source on GitHubverify-apply
When your agent uses it
Apply corrections from filter-verify (verification_result.json) or data-verify (data_verify_result.json) to filter_result.json. User specifies source type explicitly. Applies field-level and structural corrections, performs inline verification for new/modified settings, writes change log.
What it loads into context
Frontmatter
172 tokensAlways available to the agent
SKILL.md
795 tokensWhen the skill is activated
Reference documents
2,409 tokensLoaded on request
Token counts are estimates. Scripts run separately; they are not counted as reference context.
The playbook
- Core Principle: Pipeline Continuity
- Input Requirements
- Workflow
Files
SKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesSKILL.mdcore3,387 bytesreferences/apply_rules.mddoc10,436 bytes
Read SKILL.md
# Verify-Apply Skill
Apply verification corrections to `filter_result.json`. Reads a verification result JSON, applies every correction/finding, and writes a change log.
## Core Principle: Pipeline Continuity
**After apply, `filter_result.json` must be directly consumable by the next pipeline stage — no re-runs of previous skills required.**
- filter-verify apply → output must match paper-filter output structure (ready for data-check)
- data-verify apply → output must match data-check output structure (ready for task-build)
This means any new or structurally modified evaluation setting must have the same completeness as what the corresponding upstream skill would produce.
> **Note:** If any concepts, operations, requirements, or explanations are unclear during implementation or adjustment, please refer to the documentation in the `paper-filter` and `data-check` skills for detailed guidance.View the complete source on GitHubSkills included
- data-check
- data-verify
- dockerfile-fix
- filter-verify
- paper-filter
- paper-preprocess
- task-build
- task-fix
- task-verify
- verify-apply
- data-check
- data-verify
- dockerfile-fix
- filter-verify
- paper-filter
- paper-preprocess
- task-build
- task-fix
- task-verify
- verify-apply
Repository facts
- License
- MIT
- Created
- Jun 22, 2026
- Last pushed
- Oct 20, 2026
- Forks
- 11
- Open issues
- 0
- Project activity
- Active
Before you install
Network or broad shellAutomated signals, not a guarantee
Checked Nov 4, 2026
Not fully analyzed: some files were too long, unreadable or missing. This never lowers the level.
Show 52 notes
naturegym/.agents/skills/data-check/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/data-verify/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/data-verify/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-filter/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-filter/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-filter/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-preprocess/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-preprocess/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-preprocess/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-preprocess/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/paper-preprocess/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/task-build/references/environment_guide.md:48Analysis incomplete: physical or joined line exceeds character limitnaturegym/.agents/skills/task-fix/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/task-fix/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/task-fix/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/task-fix/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/task-fix/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/task-verify/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/verify-apply/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzednaturegym/.agents/skills/verify-apply/SKILL.md:1Analysis incomplete: Referenced executable target unavailable; not analyzed
- 18 broad permissionsin 18 files: data-check/SKILL.md:1 data-verify/SKILL.md:1 dockerfile-fix/SKILL.md:1 and 15 more
- 28 dangerous shell commandsin 8 files: task-build/references/environment_guide.md:134 task-build/references/environment_guide.md:134 task-build/references/Dockerfile.base.v3:57 and 5 more
- 216 network referencesin 22 files: task-build/references/environment_guide.md:46 task-build/references/environment_guide.md:46 paper-filter/scripts/validate_links.py:132 and 19 more
- No hidden or obfuscated text
- No instruction overrides
- 16 executable scriptsin 16 files: data-check/scripts/inspect_data.py:1 paper-filter/scripts/estimate_size.py:1 paper-filter/scripts/validate_links.py:1 and 13 more
- 14 references to secretsin 10 files: task-build/references/readme_guide.md:73 task-verify/scripts/run_baseline_test.py:27 task-build/references/readme_guide.md:73 and 7 more
- No requests to vote in Hot
Show all 292 signals by file
naturegym/.agents/skills/data-check/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit - Network accessline 107
- Run all download/script Bash calls in the **foreground** — never use `run_in_background: true`. - Set `timeout: 86400000` (24 hours) on every download, clone, and acquisition Bash call. Without an explicit timeout the Bash tool defaults t - Network accessline 109
- Acquisition is **idempotent**: before downloading/copying/generating, check whether the target already exists. Use `wget -c` to resume partial downloads. If a session ends mid-download (e.g., a multi-day dataset exceeds 24h), the next inv
naturegym/.agents/skills/data-check/references/acquisition_guide.md
- Network accessline 111
- **Repositories**: If `<output_dir>/repositories/<owner>_<repo_name>/.git/` exists **and** `git -C <path> rev-parse HEAD` returns a valid commit hash, skip cloning. Otherwise delete the incomplete directory and re-clone (git clone is not r - Network accessline 112
- **Downloads**: Always use `wget -c` (resume) so that partially downloaded files continue from where they left off rather than restarting. If a file already exists and its size matches the expected size (from HTTP `Content-Length`), skip t - Network accessline 145
Download from URL to `data/<setting_dir>/` using `wget` or `curl`. Key requirements: - **Resume**: Always use `wget -c` (or `curl -C -`) to support interrupted/partial downloads - Network accessline 146
- **Resume**: Always use `wget -c` (or `curl -C -`) to support interrupted/partial downloads - Network accessline 150
wget -c \ - Network accessline 164
curl --noproxy '*' -C - -L -o <output> <url> - Network accessline 165
# or: env -u http_proxy -u https_proxy -u HTTP_PROXY -u HTTPS_PROXY \ wget -c --no-check-certificate <url> - Network accessline 166
env -u http_proxy -u https_proxy -u HTTP_PROXY -u HTTPS_PROXY wget -c --no-check-certificate <url> - Network accessline 167
wget -c --no-check-certificate <url>
naturegym/.agents/skills/data-check/references/inspect_data.md
naturegym/.agents/skills/data-check/scripts/inspect_data.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.agents/skills/data-verify/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash(python *), Bash(python3 *), Bash(find *), Bash(wc *), Bash(head *), Bash(diff *), Write
naturegym/.agents/skills/dockerfile-fix/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit
naturegym/.agents/skills/paper-filter/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, WebFetch, Bash(python *), Bash(python3 *)
naturegym/.agents/skills/paper-filter/references/estimate_size_usage.md
naturegym/.agents/skills/paper-filter/references/validate_links_usage.md
- Network accessline 60
"results": [ { "url": "https://zenodo.org/record/xxx", - Network accessline 61
{ "url": "https://zenodo.org/record/xxx", "status": "accessible", - Network accessline 62
"url": "https://zenodo.org/record/xxx", - Network accessline 64
"http_code": 200, "notes": null, "final_url": "https://zenodo.org/record/xxx", - Network accessline 65
"notes": null, "final_url": "https://zenodo.org/record/xxx", "cloud_type": null - Network accessline 66
"final_url": "https://zenodo.org/record/xxx", - Network accessline 68
}, { "url": "https://figshare.com/articles/xxx", - Network accessline 69
{ "url": "https://figshare.com/articles/xxx", "status": "accessible", - Network accessline 70
"url": "https://figshare.com/articles/xxx", - Network accessline 72
"http_code": 202, "notes": null, "final_url": "https://figshare.com/articles/xxx", - Network accessline 73
"notes": null, "final_url": "https://figshare.com/articles/xxx", "cloud_type": null - Network accessline 74
"final_url": "https://figshare.com/articles/xxx", - Network accessline 76
}, { "url": "https://physionet.org/content/mimiciii/", - Network accessline 77
{ "url": "https://physionet.org/content/mimiciii/", "status": "requires_auth", - Network accessline 78
"url": "https://physionet.org/content/mimiciii/",
naturegym/.agents/skills/paper-filter/scripts/estimate_size.py
- Executable scriptline 1
#!/usr/bin/env python3 - Access to secretsline 27
[Credential-related source omitted] - Network accessline 150
try: resp = requests.get( - Network accessline 151
resp = requests.get( - Network accessline 152
f"https://api.github.com/repos/{owner}/{repo}", - Network accessline 175
try: resp = requests.get( - Network accessline 176
resp = requests.get( - Network accessline 177
f"https://zenodo.org/api/records/{record_id}", - Network accessline 192
try: resp = requests.get( - Network accessline 193
resp = requests.get( - Network accessline 194
f"https://api.figshare.com/v2/articles/{article_id}", - Network accessline 209
try: resp = requests.get( - Network accessline 210
resp = requests.get( - Network accessline 211
f"https://huggingface.co/api/{repo_type}/{repo_id}",
naturegym/.agents/skills/paper-filter/scripts/validate_links.py
- Executable scriptline 1
#!/usr/bin/env python3 - Access to secretsline 65
[Credential-related source omitted] - Network accessline 132
direct_url = f"https://drive.google.com/uc?export=download&id={file_id}" - Network accessline 134
try: headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 135
headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) result["http_code"] = response.status_c - Network accessline 136
response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 155
get_response = requests.get(direct_url, timeout=timeout, headers=headers, stream=True, verify=SSL_VERIFY) - Network accessline 193
try: headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"} response = requests.get(url, timeout=timeout, headers=headers, allow_redirects=True, verify=SSL_VERIFY) - Network accessline 194
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"} response = requests.get(url, timeout=timeout, headers=headers, allow_redirects=True, verify=SSL_VERIFY) result["http_code"] = response. - Network accessline 195
response = requests.get(url, timeout=timeout, headers=headers, allow_redirects=True, verify=SSL_VERIFY) - Network accessline 243
try: headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 244
headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) result["http_code"] = response.status_c - Network accessline 245
response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 290
get_response = requests.get(url, timeout=timeout, allow_redirects=True, - Network accessline 384
response = requests.head(url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 520
description="Validate data link accessibility for zero-interaction acquisition check.", epilog="Examples:\n" " python validate_links.py https://example.com/data.zip https://zenodo.org/record/xxx\n" - Network accessline 521
epilog="Examples:\n" " python validate_links.py https://example.com/data.zip https://zenodo.org/record/xxx\n" " python validate_links.py --file links.json\n" - Network accessline 522
" python validate_links.py https://example.com/data.zip https://zenodo.org/record/xxx\n"
naturegym/.agents/skills/paper-preprocess/SKILL.md
- Broad permissionsline 1
Read, Bash(python *) - Network accessline 75
"links": [ { "url": "https://github.com/example/repo", - Network accessline 76
{ "url": "https://github.com/example/repo", "type": "github | zenodo | codeocean | huggingface | google_drive | dropbox | other", - Network accessline 77
"url": "https://github.com/example/repo", - Network accessline 81
} ], "data_availability": ["https://..."], - Network accessline 82
], "data_availability": ["https://..."], "code_availability": ["https://github.com/..."], - Network accessline 83
"data_availability": ["https://..."], - Network accessline 84
"code_availability": ["https://github.com/..."],
naturegym/.agents/skills/paper-preprocess/scripts/extract_figures.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.agents/skills/paper-preprocess/scripts/extract_links.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.agents/skills/paper-preprocess/scripts/extract_text.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.agents/skills/paper-preprocess/scripts/preprocess.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.agents/skills/task-build/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit
naturegym/.agents/skills/task-build/references/Dockerfile.base.v3
- Network accessline 34
gfortran \ git \ wget \ - Network accessline 35
git \ wget \ curl \ - Network accessline 36
wget \ - Network accessline 37
curl \ - Dangerous shell operationline 57
&& rm -rf /var/lib/apt/lists/* - Network accessline 91
torchvision==0.21.0 \ torchaudio==2.6.0 \ --index-url https://download.pytorch.org/whl/cu118 - Network accessline 93
--index-url https://download.pytorch.org/whl/cu118 - Network accessline 125
# ── Node.js + Agent CLIs (Claude Code, Codex, Gemini) ────────────────────── RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ - Dangerous shell operationline 126
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ - Network accessline 126
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ - Dangerous shell operationline 128
&& rm -rf /var/lib/apt/lists/* \
naturegym/.agents/skills/task-build/references/environment_guide.md
- Network accessline 46
- `v3` standardizes on **CUDA 11.8 + cuDNN 8** with `torch==2.6.0`, `torchvision==0.21.0`, and `torchaudio==2.6.0` installed from `https://download.pytorch.org/whl/cu118`. - Network accessline 48
- Torch ecosystem packages with compiled extensions must match **both** the torch minor and the CUDA variant. This includes packages such as `dgl`, `torch-geometric`, `pyg-lib`, `torch-scatter`, `torch-sparse`, `torch-cluster`, `torch-harmo - Network accessline 50
- **JAX ecosystem lock (v3 base, CUDA 11.8)**: JAX dropped `cuda11` jaxlib wheels after `0.4.25` — `jaxlib==0.4.26` and later only publish `+cuda12.cudnn89` / `+cuda12.cudnn91` variants (verify at `https://storage.googleapis.com/jax-release - Network accessline 115
**Known compatibility issues** (discovered from verified Dockerfile failures): - `catboost==1.2.7`: declares `numpy>=1.16.0,<2.0`. Fully compatible with base numpy==1.26.4. No ABI issues. However, if the task's Dockerfile installs packages - Network accessline 116
- `catboost==1.2.7`: declares `numpy>=1.16.0,<2.0`. Fully compatible with base numpy==1.26.4. No ABI issues. However, if the task's Dockerfile installs packages that trigger numpy upgrade to 2.x, catboost will break (`numpy.dtype size chang - Network accessline 117
- DGL from wrong torch wheel index: `dgl` installed from a mismatched wheel index will replace the base image's `torch==2.6.0+cu118` with a different version (e.g., CPU or newer minor version). This underlying change breaks the ABI for the - Dangerous shell operationline 134
- `vina==1.2.5`: builds from source and requires Boost C++ headers. The base image does not include Boost. Add `RUN apt-get update && apt-get install -y --no-install-recommends libboost-all-dev && rm -rf /var/lib/apt/lists/*` before `pip in - Network accessline 139
- `scikits.odes`: version 2.6.3 fails to build in the v3 base image environment — it relies on deprecated `numpy.distutils` which crashes with current setuptools (`TypeError: Compiler.__init__() takes from 1 to 3 positional arguments but 4 - Network accessline 140
- `neuraloperator==0.3.0`: the import name is `neuralop` (not `neuraloperator`). `torch-harmonics` is not declared as a hard dependency in its metadata, but `neuralop` imports it at runtime — import verification will fail with `No module na - Dangerous shell operationline 141
- `pytfa==0.9.4` has malformed metadata that `pip>=24.1` strictly rejects. The specific bad line lives in `setup.cfg` under `[options.extras_require]` — a bare `python_version>="3.6"` entry inside the `equilibrator` extras list (it is being - Network accessline 141
- `pytfa==0.9.4` has malformed metadata that `pip>=24.1` strictly rejects. The specific bad line lives in `setup.cfg` under `[options.extras_require]` — a bare `python_version>="3.6"` entry inside the `equilibrator` extras list (it is being - Network accessline 146
2. **Before adding `apt-get install`**: check (a) whether the base image already includes the system package; (b) whether the target Python package has a pre-built wheel on PyPI — if so, the `-dev` header packages needed for source compilat - Network accessline 147
3. **When download is unavoidable**: apply the robust patterns below (retry + fallback + validation) - **General: `git clone` / `pip install git+` in Dockerfiles**: Direct GitHub cloning from Docker build environments frequently suffers fro - Network accessline 148
- **General: `git clone` / `pip install git+` in Dockerfiles**: Direct GitHub cloning from Docker build environments frequently suffers from absolute timeouts (even with 3+ retries, it often remains completely blocked). **Preferred approach - Network accessline 151
RUN set -eux; \ for url in \ "https://mirror.ghproxy.com/https://github.com/<org>/<repo>/archive/<commit>.tar.gz" \ - Network accessline 152
for url in \ "https://mirror.ghproxy.com/https://github.com/<org>/<repo>/archive/<commit>.tar.gz" \ "https://codeload.github.com/<org>/<repo>/tar.gz/<commit>" \ - Network accessline 153
"https://mirror.ghproxy.com/https://github.com/<org>/<repo>/archive/<commit>.tar.gz" \ - Network accessline 154
"https://codeload.github.com/<org>/<repo>/tar.gz/<commit>" \ "https://github.com/<org>/<repo>/archive/<commit>.tar.gz"; do \ if wget -q --tries=3 --timeout=30 "$url" -O /tmp/repo.tar.gz; then break; fi; \ - Network accessline 155
"https://github.com/<org>/<repo>/archive/<commit>.tar.gz"; do \ if wget -q --tries=3 --timeout=30 "$url" -O /tmp/repo.tar.gz; then break; fi; \ done; \ - Network accessline 156
if wget -q --tries=3 --timeout=30 "$url" -O /tmp/repo.tar.gz; then break; fi; \ - Network accessline 160
python -m pip install --no-cache-dir /tmp/<repo>-<commit> ``` - *Git+ Pip Install (only when archive+sed patching is not viable)*: `RUN python -m pip install git+https://mirror.ghproxy.com/https://github.com/<org>/<repo>.git@<commit>` - Network accessline 161
``` - *Git+ Pip Install (only when archive+sed patching is not viable)*: `RUN python -m pip install git+https://mirror.ghproxy.com/https://github.com/<org>/<repo>.git@<commit>` — note that `pip install git+` cannot do the multi-URL fallba - Network accessline 162
- *Git+ Pip Install (only when archive+sed patching is not viable)*: `RUN python -m pip install git+https://mirror.ghproxy.com/https://github.com/<org>/<repo>.git@<commit>` — note that `pip install git+` cannot do the multi-URL fallback loo - Network accessline 166
RUN wget -q https://primary-source.com/binary -O /usr/local/bin/binary \ - Network accessline 167
|| wget -q https://backup-source.com/binary -O /usr/local/bin/binary \ - Dangerous shell operationline 173
RUN set -eux; export DEBIAN_FRONTEND=noninteractive; sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc - Network accessline 173
RUN set -eux; \ export DEBIAN_FRONTEND=noninteractive; \ sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /e - Network accessline 174
export DEBIAN_FRONTEND=noninteractive; \ sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list; - Network accessline 175
sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list; \ - Dangerous shell operationline 178
rm -rf /var/lib/apt/lists/*; \ - Dangerous shell operationline 179
if apt-get update && apt-get install -y --no-install-recommends <packages> && rm -rf /var/lib/apt/lists/*; then \ - Dangerous shell operationline 220
RUN apt-get update && apt-get install -y --no-install-recommends libxxx-dev && rm -rf /var/lib/apt/lists/* - Dangerous shell operationline 222
&& rm -rf /var/lib/apt/lists/* - Dangerous shell operationline 245
RUN apt-get update && apt-get install -y --no-install-recommends software-properties-common build-essential cmake git wget curl && add-apt-repository ppa:deadsnakes/ppa -y && apt-get update && apt-get install -y --no-ins - Network accessline 245
RUN apt-get update && apt-get install -y --no-install-recommends \ software-properties-common build-essential cmake git wget curl \ - Network accessline 246
software-properties-common build-essential cmake git wget curl \ - Dangerous shell operationline 252
&& rm -rf /var/lib/apt/lists/* - Network accessline 257
# Install all required packages from scratch RUN python -m pip install --no-cache-dir \ torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu118 - Network accessline 258
RUN python -m pip install --no-cache-dir torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu118 - Network accessline 259
torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu118
naturegym/.agents/skills/task-build/references/evaluator_guide.md
- Access to secretsline 39
[Credential-related source omitted]
naturegym/.agents/skills/task-build/references/readme_guide.md
naturegym/.agents/skills/task-fix/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit
naturegym/.agents/skills/task-verify/SKILL.md
naturegym/.agents/skills/task-verify/scripts/run_baseline_test.py
naturegym/.agents/skills/verify-apply/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzip *), Bash(cp *), Bash(mv *), Bash(du - Network accessline 4
context: fork agent: general-purpose allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip - Network accessline 5
agent: general-purpose allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzi - Dangerous shell operationline 6
allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzip *), Bash(cp *), Bash( - Network accessline 6
allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzip *), Bash(cp *), Bash(
naturegym/.claude/skills/data-check/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit - Network accessline 107
- Run all download/script Bash calls in the **foreground** — never use `run_in_background: true`. - Set `timeout: 86400000` (24 hours) on every download, clone, and acquisition Bash call. Without an explicit timeout the Bash tool defaults t - Network accessline 109
- Acquisition is **idempotent**: before downloading/copying/generating, check whether the target already exists. Use `wget -c` to resume partial downloads. If a session ends mid-download (e.g., a multi-day dataset exceeds 24h), the next inv
naturegym/.claude/skills/data-check/references/acquisition_guide.md
- Network accessline 111
- **Repositories**: If `<output_dir>/repositories/<owner>_<repo_name>/.git/` exists **and** `git -C <path> rev-parse HEAD` returns a valid commit hash, skip cloning. Otherwise delete the incomplete directory and re-clone (git clone is not r - Network accessline 112
- **Downloads**: Always use `wget -c` (resume) so that partially downloaded files continue from where they left off rather than restarting. If a file already exists and its size matches the expected size (from HTTP `Content-Length`), skip t - Network accessline 145
Download from URL to `data/<setting_dir>/` using `wget` or `curl`. Key requirements: - **Resume**: Always use `wget -c` (or `curl -C -`) to support interrupted/partial downloads - Network accessline 146
- **Resume**: Always use `wget -c` (or `curl -C -`) to support interrupted/partial downloads - Network accessline 150
wget -c \ - Network accessline 164
curl --noproxy '*' -C - -L -o <output> <url> - Network accessline 165
# or: env -u http_proxy -u https_proxy -u HTTP_PROXY -u HTTPS_PROXY \ wget -c --no-check-certificate <url> - Network accessline 166
env -u http_proxy -u https_proxy -u HTTP_PROXY -u HTTPS_PROXY wget -c --no-check-certificate <url> - Network accessline 167
wget -c --no-check-certificate <url>
naturegym/.claude/skills/data-check/references/inspect_data.md
naturegym/.claude/skills/data-check/scripts/inspect_data.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.claude/skills/data-verify/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash(python *), Bash(python3 *), Bash(find *), Bash(wc *), Bash(head *), Bash(diff *), Write
naturegym/.claude/skills/dockerfile-fix/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit
naturegym/.claude/skills/paper-filter/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, WebFetch, Bash(python *), Bash(python3 *)
naturegym/.claude/skills/paper-filter/references/estimate_size_usage.md
naturegym/.claude/skills/paper-filter/references/validate_links_usage.md
- Network accessline 60
"results": [ { "url": "https://zenodo.org/record/xxx", - Network accessline 61
{ "url": "https://zenodo.org/record/xxx", "status": "accessible", - Network accessline 62
"url": "https://zenodo.org/record/xxx", - Network accessline 64
"http_code": 200, "notes": null, "final_url": "https://zenodo.org/record/xxx", - Network accessline 65
"notes": null, "final_url": "https://zenodo.org/record/xxx", "cloud_type": null - Network accessline 66
"final_url": "https://zenodo.org/record/xxx", - Network accessline 68
}, { "url": "https://figshare.com/articles/xxx", - Network accessline 69
{ "url": "https://figshare.com/articles/xxx", "status": "accessible", - Network accessline 70
"url": "https://figshare.com/articles/xxx", - Network accessline 72
"http_code": 202, "notes": null, "final_url": "https://figshare.com/articles/xxx", - Network accessline 73
"notes": null, "final_url": "https://figshare.com/articles/xxx", "cloud_type": null - Network accessline 74
"final_url": "https://figshare.com/articles/xxx", - Network accessline 76
}, { "url": "https://physionet.org/content/mimiciii/", - Network accessline 77
{ "url": "https://physionet.org/content/mimiciii/", "status": "requires_auth", - Network accessline 78
"url": "https://physionet.org/content/mimiciii/",
naturegym/.claude/skills/paper-filter/scripts/estimate_size.py
- Executable scriptline 1
#!/usr/bin/env python3 - Access to secretsline 27
[Credential-related source omitted] - Network accessline 150
try: resp = requests.get( - Network accessline 151
resp = requests.get( - Network accessline 152
f"https://api.github.com/repos/{owner}/{repo}", - Network accessline 175
try: resp = requests.get( - Network accessline 176
resp = requests.get( - Network accessline 177
f"https://zenodo.org/api/records/{record_id}", - Network accessline 192
try: resp = requests.get( - Network accessline 193
resp = requests.get( - Network accessline 194
f"https://api.figshare.com/v2/articles/{article_id}", - Network accessline 209
try: resp = requests.get( - Network accessline 210
resp = requests.get( - Network accessline 211
f"https://huggingface.co/api/{repo_type}/{repo_id}",
naturegym/.claude/skills/paper-filter/scripts/validate_links.py
- Executable scriptline 1
#!/usr/bin/env python3 - Access to secretsline 65
[Credential-related source omitted] - Network accessline 132
direct_url = f"https://drive.google.com/uc?export=download&id={file_id}" - Network accessline 134
try: headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 135
headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) result["http_code"] = response.status_c - Network accessline 136
response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 155
get_response = requests.get(direct_url, timeout=timeout, headers=headers, stream=True, verify=SSL_VERIFY) - Network accessline 193
try: headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"} response = requests.get(url, timeout=timeout, headers=headers, allow_redirects=True, verify=SSL_VERIFY) - Network accessline 194
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"} response = requests.get(url, timeout=timeout, headers=headers, allow_redirects=True, verify=SSL_VERIFY) result["http_code"] = response. - Network accessline 195
response = requests.get(url, timeout=timeout, headers=headers, allow_redirects=True, verify=SSL_VERIFY) - Network accessline 243
try: headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 244
headers = {"User-Agent": "Mozilla/5.0 (compatible; PaperFilterBot/1.0)"} response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) result["http_code"] = response.status_c - Network accessline 245
response = requests.head(direct_url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 290
get_response = requests.get(url, timeout=timeout, allow_redirects=True, - Network accessline 384
response = requests.head(url, timeout=timeout, allow_redirects=True, headers=headers, verify=SSL_VERIFY) - Network accessline 520
description="Validate data link accessibility for zero-interaction acquisition check.", epilog="Examples:\n" " python validate_links.py https://example.com/data.zip https://zenodo.org/record/xxx\n" - Network accessline 521
epilog="Examples:\n" " python validate_links.py https://example.com/data.zip https://zenodo.org/record/xxx\n" " python validate_links.py --file links.json\n" - Network accessline 522
" python validate_links.py https://example.com/data.zip https://zenodo.org/record/xxx\n"
naturegym/.claude/skills/paper-preprocess/SKILL.md
- Broad permissionsline 1
Read, Bash(python *) - Network accessline 75
"links": [ { "url": "https://github.com/example/repo", - Network accessline 76
{ "url": "https://github.com/example/repo", "type": "github | zenodo | codeocean | huggingface | google_drive | dropbox | other", - Network accessline 77
"url": "https://github.com/example/repo", - Network accessline 81
} ], "data_availability": ["https://..."], - Network accessline 82
], "data_availability": ["https://..."], "code_availability": ["https://github.com/..."], - Network accessline 83
"data_availability": ["https://..."], - Network accessline 84
"code_availability": ["https://github.com/..."],
naturegym/.claude/skills/paper-preprocess/scripts/extract_figures.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.claude/skills/paper-preprocess/scripts/extract_links.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.claude/skills/paper-preprocess/scripts/extract_text.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.claude/skills/paper-preprocess/scripts/preprocess.py
- Executable scriptline 1
#!/usr/bin/env python3
naturegym/.claude/skills/task-build/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit
naturegym/.claude/skills/task-build/references/Dockerfile.base.v3
- Network accessline 34
gfortran \ git \ wget \ - Network accessline 35
git \ wget \ curl \ - Network accessline 36
wget \ - Network accessline 37
curl \ - Dangerous shell operationline 57
&& rm -rf /var/lib/apt/lists/* - Network accessline 91
torchvision==0.21.0 \ torchaudio==2.6.0 \ --index-url https://download.pytorch.org/whl/cu118 - Network accessline 93
--index-url https://download.pytorch.org/whl/cu118 - Network accessline 125
# ── Node.js + Agent CLIs (Claude Code, Codex, Gemini) ────────────────────── RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ - Dangerous shell operationline 126
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ - Network accessline 126
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ - Dangerous shell operationline 128
&& rm -rf /var/lib/apt/lists/* \
naturegym/.claude/skills/task-build/references/environment_guide.md
- Network accessline 46
- `v3` standardizes on **CUDA 11.8 + cuDNN 8** with `torch==2.6.0`, `torchvision==0.21.0`, and `torchaudio==2.6.0` installed from `https://download.pytorch.org/whl/cu118`. - Network accessline 48
- Torch ecosystem packages with compiled extensions must match **both** the torch minor and the CUDA variant. This includes packages such as `dgl`, `torch-geometric`, `pyg-lib`, `torch-scatter`, `torch-sparse`, `torch-cluster`, `torch-harmo - Network accessline 50
- **JAX ecosystem lock (v3 base, CUDA 11.8)**: JAX dropped `cuda11` jaxlib wheels after `0.4.25` — `jaxlib==0.4.26` and later only publish `+cuda12.cudnn89` / `+cuda12.cudnn91` variants (verify at `https://storage.googleapis.com/jax-release - Network accessline 115
**Known compatibility issues** (discovered from verified Dockerfile failures): - `catboost==1.2.7`: declares `numpy>=1.16.0,<2.0`. Fully compatible with base numpy==1.26.4. No ABI issues. However, if the task's Dockerfile installs packages - Network accessline 116
- `catboost==1.2.7`: declares `numpy>=1.16.0,<2.0`. Fully compatible with base numpy==1.26.4. No ABI issues. However, if the task's Dockerfile installs packages that trigger numpy upgrade to 2.x, catboost will break (`numpy.dtype size chang - Network accessline 117
- DGL from wrong torch wheel index: `dgl` installed from a mismatched wheel index will replace the base image's `torch==2.6.0+cu118` with a different version (e.g., CPU or newer minor version). This underlying change breaks the ABI for the - Dangerous shell operationline 134
- `vina==1.2.5`: builds from source and requires Boost C++ headers. The base image does not include Boost. Add `RUN apt-get update && apt-get install -y --no-install-recommends libboost-all-dev && rm -rf /var/lib/apt/lists/*` before `pip in - Network accessline 139
- `scikits.odes`: version 2.6.3 fails to build in the v3 base image environment — it relies on deprecated `numpy.distutils` which crashes with current setuptools (`TypeError: Compiler.__init__() takes from 1 to 3 positional arguments but 4 - Network accessline 140
- `neuraloperator==0.3.0`: the import name is `neuralop` (not `neuraloperator`). `torch-harmonics` is not declared as a hard dependency in its metadata, but `neuralop` imports it at runtime — import verification will fail with `No module na - Dangerous shell operationline 141
- `pytfa==0.9.4` has malformed metadata that `pip>=24.1` strictly rejects. The specific bad line lives in `setup.cfg` under `[options.extras_require]` — a bare `python_version>="3.6"` entry inside the `equilibrator` extras list (it is being - Network accessline 141
- `pytfa==0.9.4` has malformed metadata that `pip>=24.1` strictly rejects. The specific bad line lives in `setup.cfg` under `[options.extras_require]` — a bare `python_version>="3.6"` entry inside the `equilibrator` extras list (it is being - Network accessline 146
2. **Before adding `apt-get install`**: check (a) whether the base image already includes the system package; (b) whether the target Python package has a pre-built wheel on PyPI — if so, the `-dev` header packages needed for source compilat - Network accessline 147
3. **When download is unavoidable**: apply the robust patterns below (retry + fallback + validation) - **General: `git clone` / `pip install git+` in Dockerfiles**: Direct GitHub cloning from Docker build environments frequently suffers fro - Network accessline 148
- **General: `git clone` / `pip install git+` in Dockerfiles**: Direct GitHub cloning from Docker build environments frequently suffers from absolute timeouts (even with 3+ retries, it often remains completely blocked). **Preferred approach - Network accessline 151
RUN set -eux; \ for url in \ "https://mirror.ghproxy.com/https://github.com/<org>/<repo>/archive/<commit>.tar.gz" \ - Network accessline 152
for url in \ "https://mirror.ghproxy.com/https://github.com/<org>/<repo>/archive/<commit>.tar.gz" \ "https://codeload.github.com/<org>/<repo>/tar.gz/<commit>" \ - Network accessline 153
"https://mirror.ghproxy.com/https://github.com/<org>/<repo>/archive/<commit>.tar.gz" \ - Network accessline 154
"https://codeload.github.com/<org>/<repo>/tar.gz/<commit>" \ "https://github.com/<org>/<repo>/archive/<commit>.tar.gz"; do \ if wget -q --tries=3 --timeout=30 "$url" -O /tmp/repo.tar.gz; then break; fi; \ - Network accessline 155
"https://github.com/<org>/<repo>/archive/<commit>.tar.gz"; do \ if wget -q --tries=3 --timeout=30 "$url" -O /tmp/repo.tar.gz; then break; fi; \ done; \ - Network accessline 156
if wget -q --tries=3 --timeout=30 "$url" -O /tmp/repo.tar.gz; then break; fi; \ - Network accessline 160
python -m pip install --no-cache-dir /tmp/<repo>-<commit> ``` - *Git+ Pip Install (only when archive+sed patching is not viable)*: `RUN python -m pip install git+https://mirror.ghproxy.com/https://github.com/<org>/<repo>.git@<commit>` - Network accessline 161
``` - *Git+ Pip Install (only when archive+sed patching is not viable)*: `RUN python -m pip install git+https://mirror.ghproxy.com/https://github.com/<org>/<repo>.git@<commit>` — note that `pip install git+` cannot do the multi-URL fallba - Network accessline 162
- *Git+ Pip Install (only when archive+sed patching is not viable)*: `RUN python -m pip install git+https://mirror.ghproxy.com/https://github.com/<org>/<repo>.git@<commit>` — note that `pip install git+` cannot do the multi-URL fallback loo - Network accessline 166
RUN wget -q https://primary-source.com/binary -O /usr/local/bin/binary \ - Network accessline 167
|| wget -q https://backup-source.com/binary -O /usr/local/bin/binary \ - Dangerous shell operationline 173
RUN set -eux; export DEBIAN_FRONTEND=noninteractive; sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc - Network accessline 173
RUN set -eux; \ export DEBIAN_FRONTEND=noninteractive; \ sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /e - Network accessline 174
export DEBIAN_FRONTEND=noninteractive; \ sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list; - Network accessline 175
sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g; s|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list; \ - Dangerous shell operationline 178
rm -rf /var/lib/apt/lists/*; \ - Dangerous shell operationline 179
if apt-get update && apt-get install -y --no-install-recommends <packages> && rm -rf /var/lib/apt/lists/*; then \ - Dangerous shell operationline 220
RUN apt-get update && apt-get install -y --no-install-recommends libxxx-dev && rm -rf /var/lib/apt/lists/* - Dangerous shell operationline 222
&& rm -rf /var/lib/apt/lists/* - Dangerous shell operationline 245
RUN apt-get update && apt-get install -y --no-install-recommends software-properties-common build-essential cmake git wget curl && add-apt-repository ppa:deadsnakes/ppa -y && apt-get update && apt-get install -y --no-ins - Network accessline 245
RUN apt-get update && apt-get install -y --no-install-recommends \ software-properties-common build-essential cmake git wget curl \ - Network accessline 246
software-properties-common build-essential cmake git wget curl \ - Dangerous shell operationline 252
&& rm -rf /var/lib/apt/lists/* - Network accessline 257
# Install all required packages from scratch RUN python -m pip install --no-cache-dir \ torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu118 - Network accessline 258
RUN python -m pip install --no-cache-dir torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu118 - Network accessline 259
torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu118
naturegym/.claude/skills/task-build/references/evaluator_guide.md
- Access to secretsline 39
[Credential-related source omitted]
naturegym/.claude/skills/task-build/references/readme_guide.md
naturegym/.claude/skills/task-fix/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Bash, WebFetch, Write, Edit
naturegym/.claude/skills/task-verify/SKILL.md
naturegym/.claude/skills/task-verify/scripts/run_baseline_test.py
naturegym/.claude/skills/verify-apply/SKILL.md
- Broad permissionsline 1
Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzip *), Bash(cp *), Bash(mv *), Bash(du - Network accessline 4
context: fork agent: general-purpose allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip - Network accessline 5
agent: general-purpose allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzi - Dangerous shell operationline 6
allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzip *), Bash(cp *), Bash( - Network accessline 6
allowed-tools: Read, Grep, Glob, Write, Bash(rm *), Bash(mkdir *), Bash(find *), Bash(ls *), Bash(git clone *), Bash(GIT_SSL_NO_VERIFY=1 git clone *), Bash(wget *), Bash(curl *), Bash(tar *), Bash(unzip *), Bash(gunzip *), Bash(cp *), Bash(
How the signals fit together
Facts from a static read of the files at the pinned commit. They describe what the code does, not why.
Some files could not be read; the facts below still hold.
Declared and executed: 16 facts
- Allows Bash without restrictions, though it has no scripts or shell blocksdockerfile-fix/SKILL.md:6High confidence
- Contacts zenodo.org, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/estimate_size.py:177High confidence
- Contacts api.figshare.com, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/estimate_size.py:194High confidence
- Contacts huggingface.co, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/estimate_size.py:211High confidence
- Contacts drive.google.com, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/validate_links.py:132High confidence
- Contacts zenodo.org, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/validate_links.py:522High confidence
- Allows Bash without restrictions, though it has no scripts or shell blockstask-build/SKILL.md:6High confidence
- Allows Bash without restrictions, though it has no scripts or shell blockstask-fix/SKILL.md:6High confidence
- Allows Bash without restrictions, though it has no scripts or shell blocksdockerfile-fix/SKILL.md:6High confidence
- Contacts zenodo.org, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/estimate_size.py:177High confidence
- Contacts api.figshare.com, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/estimate_size.py:194High confidence
- Contacts huggingface.co, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/estimate_size.py:211High confidence
- Contacts drive.google.com, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/validate_links.py:132High confidence
- Contacts zenodo.org, which SKILL.md, the README and the plugin manifest do not mentionpaper-filter/scripts/validate_links.py:522High confidence
- Allows Bash without restrictions, though it has no scripts or shell blockstask-build/SKILL.md:6High confidence
- Allows Bash without restrictions, though it has no scripts or shell blockstask-fix/SKILL.md:6High confidence
Signal confidence: 270 facts
- Broad permissions in the frontmatter of the SKILL.mddata-check/SKILL.md:1High confidence
- Network access in prose of the SKILL.mddata-check/SKILL.md:107Low confidence
- Network access in prose of the SKILL.mddata-check/SKILL.md:109Low confidence
- Network access in prose of documentationdata-check/references/acquisition_guide.md:111Low confidence
- Network access in prose of documentationdata-check/references/acquisition_guide.md:112Low confidence
- Network access in prose of documentationdata-check/references/acquisition_guide.md:145Low confidence
- Network access in prose of documentationdata-check/references/acquisition_guide.md:146Low confidence
- Network access in code of documentationdata-check/references/acquisition_guide.md:150Low confidence
- Network access in code of documentationdata-check/references/acquisition_guide.md:164Low confidence
- Network access in code of documentationdata-check/references/acquisition_guide.md:165Low confidence
- Network access in code of documentationdata-check/references/acquisition_guide.md:166Low confidence
- Network access in code of documentationdata-check/references/acquisition_guide.md:167Low confidence
- Network access in code of documentationdata-check/references/inspect_data.md:113Low confidence
- Network access in code of documentationdata-check/references/inspect_data.md:114Low confidence
- Executable script in code of a file that runsdata-check/scripts/inspect_data.py:1High confidence
- Broad permissions in the frontmatter of the SKILL.mddata-verify/SKILL.md:1High confidence
- Broad permissions in the frontmatter of the SKILL.mddockerfile-fix/SKILL.md:1High confidence
- Broad permissions in the frontmatter of the SKILL.mdpaper-filter/SKILL.md:1High confidence
- Network access in code of documentationpaper-filter/references/estimate_size_usage.md:35Low confidence
- Network access in code of documentationpaper-filter/references/estimate_size_usage.md:36Low confidence
- Network access in code of documentationpaper-filter/references/estimate_size_usage.md:37Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:60Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:61Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:62Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:64Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:65Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:66Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:68Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:69Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:70Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:72Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:73Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:74Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:76Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:77Low confidence
- Network access in code of documentationpaper-filter/references/validate_links_usage.md:78Low confidence
- Executable script in code of a file that runspaper-filter/scripts/estimate_size.py:1High confidence
- Access to secrets in code of a file that runs: a variable, placeholder or call, not a stored valuepaper-filter/scripts/estimate_size.py:27Low confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:150High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:151High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:152High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:175High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:176High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:177High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:192High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:193High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:194High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:209High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:210High confidence
- Network access in code of a file that runspaper-filter/scripts/estimate_size.py:211High confidence
and 220 more
Works with
| Agent | Status | Evidence | Install to |
|---|---|---|---|
| Claude Code | Compatible | Standard SKILL.md naturegym/.agents/skills/data-check/SKILL.md | .claude/skills/<skill-name>/ |
| Codex | Compatible | Standard SKILL.md naturegym/.agents/skills/data-check/SKILL.md | .agents/skills/<skill-name>/ |
| Cursor | Compatible | Standard SKILL.md naturegym/.agents/skills/data-check/SKILL.md | .cursor/skills/<skill-name>/ |
| Gemini CLI | Compatible | Standard SKILL.md naturegym/.agents/skills/data-check/SKILL.md | .gemini/skills/<skill-name>/ |
| GitHub Copilot | Compatible | Standard SKILL.md naturegym/.agents/skills/data-check/SKILL.md | .github/skills/<skill-name>/ |
| OpenCode | Compatible | Standard SKILL.md naturegym/.agents/skills/data-check/SKILL.md | .opencode/skills/<skill-name>/ |
- Native The repository ships a manifest, folder or frontmatter field for this agent.
- Compatible A standard SKILL.md that this agent can read.
- Unknown No evidence either way.
Derived automatically from paths, manifests and frontmatter. Not tested on each agent.