Not in the top 1,000 by GitHub stars

alexpiar/be-ambitious-harness

Make AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and an independent verifier that never sees the worker's reasoning. Plain markdown, EN + RU.

History

Review weekly changes to SKILL.md before updating. Dates mark the weeks in which we observed each version; checks apply to the recorded commits.

View weekly changes

SKILL.md

Week of Nov 2, 2026 · Line counts unavailable

Before Not checked yet → After Network or broad shell

First observation → bc374f4bafcf

No earlier version was observed for comparison.

Compare on GitHub

ru/SKILL.md

Week of Nov 2, 2026 · Line counts unavailable

Before Not checked yet → After Local scripts

First observation → bc374f4bafcf

No earlier version was observed for comparison.

Compare on GitHub

View on GitHubAdd to my skills

Verified commit bc374f4bafcf738e48d18ec1b57ada17ad374ddd

23GitHub stars
0stars in 7 days
2skills in this repository
23commits in 30 days

Heat

1.6
  • Star growth, last 7 daysNot available
  • Star growth, last 24 hoursNot available
  • Rank movement, last 7 daysNot available
  • Commits, last 30 daysNot available
  • Accounts asking about it through MCP, last 7 daysNot available
  • Hot votes this weekNot available

Heat never changes the order by stars, and sponsorship never touches it. How heat works

Stars over time

Oct 25, 2026Oct 27, 202623

30-day change not yet available: not enough history.

Individual skills in this repository

How it works

be-ambitious-harness

When your agent uses it

Harness for ambitious AI task statements. Use when the user writes /be-ambitious, "make this task deeper", "improve the task statement", "raise the bar", "strengthen the prompt" (or in Russian «сделай задачу глубже», «улучши постановку», «поставь амбициознее», «усили промпт») — or before launching any substantial task worth raising the bar on. Routes the task by scenario (design, data, code, writing, decision, generic), runs the analysis, builds a prompt with a gate check, hands it to a worker, runs an independent verifier. Not needed for five-minute tasks.

What it loads into context

Frontmatter

151 tokens

Always available to the agent

SKILL.md

824 tokens

When the skill is activated

Reference documents

214,146 tokens

Loaded on request

Token counts are estimates. Scripts run separately; they are not counted as reference context.

The playbook

  1. Flow
  2. Harness rules
  3. Files

Scripts executed separately

  • evals/fixtures/calibrate_t2pass.py
  • evals/fixtures/gen_events.py
  • evals/reference/L2/ref_lib.py
  • evals/reference/L2/run.py
  • evals/reference/L2/score.py
  • evals/reference/T1/gate.py
  • evals/reference/T2-pass/ref_lib.py
  • evals/reference/T2-pass/run.py
  • evals/reference/T2-pass/score.py
  • evals/reference/T2/ref_lib.py
  • evals/reference/T2/run.py
  • evals/reference/T2/score.py
  • evals/reference/T3/check.sh
  • evals/reference/T3/test/settings-locale-without-region.test.js
  • evals/reference/T5/score.py
  • evals/reference/oracle_churn.py
  • evals/runs/2026-10-21-gate/L2-A/churn.py
  • evals/runs/2026-10-21-gate/L2-B/checks_extra.py
  • evals/runs/2026-10-21-gate/L2-B/churn_pipeline.py
  • evals/runs/2026-10-21-gate/L2-B/features.py
  • evals/runs/2026-10-21-gate/L2-B/robustness.py
  • evals/runs/2026-10-21-gate/L2-B/score.py
  • evals/runs/2026-10-21-gate/L2-B/survival.py
  • evals/runs/2026-10-21-gate/fixtures/gen_events.py
  • evals/runs/2026-10-21-gate/fixtures/hidden/t3-hidden.test.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/bin/open-screen.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/package.json
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/app.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/currency.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/locale.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/storage.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/screens/home.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/screens/settings.js
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/test/settings.test.js
  • tools/gate-check.py
  • tools/lint-harness.py
  • tools/tests/test_gate_check.py

Files

  • .github/workflows/lint.ymlasset574 bytes
  • .gitignoreasset381 bytes
  • CITATION.cffasset855 bytes
  • LICENSEasset1,083 bytes
  • README.es.mddoc30,324 bytes
  • README.mddoc26,597 bytes
  • README.ru.mddoc43,244 bytes
  • README.zh-CN.mddoc27,487 bytes
  • SKILL.mdcore3,491 bytes
  • agents/planner.mddoc4,048 bytes
  • agents/verifier.mddoc1,894 bytes
  • agents/worker.mddoc1,715 bytes
  • checklists/post-run.mddoc2,090 bytes
  • checklists/pre-flight.mddoc2,029 bytes
  • core/gate.mddoc2,974 bytes
  • core/prompt-template.mddoc2,447 bytes
  • core/report-format.mddoc1,896 bytes
  • evals/fixtures/README.mddoc17,234 bytes
  • evals/fixtures/calibrate_t2pass.pyruns28,084 bytes
  • evals/fixtures/gen_events.pyruns19,980 bytes
  • evals/reference/L1/journal.mddoc2,428 bytes
  • evals/reference/L1/reference.mddoc6,971 bytes
  • evals/reference/L2/holdout_ids.txtdoc10,800 bytes
  • evals/reference/L2/journal.mddoc11,319 bytes
  • evals/reference/L2/model.jsonasset3,891 bytes
  • evals/reference/L2/oracle.jsonasset4,786 bytes
  • evals/reference/L2/oracle.txtdoc1,088 bytes
  • evals/reference/L2/prompt.mddoc11,514 bytes
  • evals/reference/L2/ref_lib.pyruns32,378 bytes
  • evals/reference/L2/reference.mddoc12,923 bytes
  • evals/reference/L2/results.jsonasset15,823 bytes
  • evals/reference/L2/run.pyruns4,591 bytes
  • evals/reference/L2/score.pyruns10,600 bytes
  • evals/reference/README.mddoc9,537 bytes
  • evals/reference/T1/gate.pyruns5,214 bytes
  • evals/reference/T1/journal.mddoc4,962 bytes
  • evals/reference/T1/materials/README.mddoc1,257 bytes
  • evals/reference/T1/materials/funnel.csvasset1,890 bytes
  • evals/reference/T1/materials/sessions.csvasset1,181 bytes
  • evals/reference/T1/prompt.mddoc10,157 bytes
  • evals/reference/T1/reference.mddoc11,566 bytes
  • evals/reference/T2-pass/holdout_ids.txtdoc10,800 bytes
  • evals/reference/T2-pass/journal.mddoc12,475 bytes
  • evals/reference/T2-pass/model.jsonasset4,336 bytes
  • evals/reference/T2-pass/model_base.jsonasset4,000 bytes
  • evals/reference/T2-pass/oracle.jsonasset6,341 bytes
  • evals/reference/T2-pass/oracle.txtdoc1,334 bytes
  • evals/reference/T2-pass/prompt.mddoc12,958 bytes
  • evals/reference/T2-pass/ref_lib.pyruns32,378 bytes
  • evals/reference/T2-pass/reference.mddoc14,262 bytes
  • evals/reference/T2-pass/results.jsonasset29,715 bytes
  • evals/reference/T2-pass/run.pyruns6,443 bytes
  • evals/reference/T2-pass/score.pyruns10,600 bytes
  • evals/reference/T2/holdout_ids.txtdoc10,800 bytes
  • evals/reference/T2/journal.mddoc10,950 bytes
  • evals/reference/T2/model.jsonasset3,934 bytes
  • evals/reference/T2/model_with_proxies.jsonasset4,371 bytes
  • evals/reference/T2/oracle.jsonasset6,246 bytes
  • evals/reference/T2/oracle.txtdoc1,321 bytes
  • evals/reference/T2/prompt.mddoc13,086 bytes
  • evals/reference/T2/ref_lib.pyruns32,378 bytes
  • evals/reference/T2/reference.mddoc14,300 bytes
  • evals/reference/T2/results.jsonasset14,415 bytes
  • evals/reference/T2/run.pyruns8,645 bytes
  • evals/reference/T2/score.pyruns10,600 bytes
  • evals/reference/T3/check.shruns4,966 bytes
  • evals/reference/T3/fix.diffasset1,361 bytes
  • evals/reference/T3/journal.mddoc9,189 bytes
  • evals/reference/T3/prompt.mddoc6,757 bytes
  • evals/reference/T3/reference.mddoc8,944 bytes
  • evals/reference/T3/test/settings-locale-without-region.test.jsruns1,345 bytes
  • evals/reference/T4/article.mddoc6,447 bytes
  • evals/reference/T4/journal.mddoc5,949 bytes
  • evals/reference/T4/prompt.mddoc8,429 bytes
  • evals/reference/T4/reference.mddoc11,321 bytes
  • evals/reference/T4/sources.mddoc7,049 bytes
  • evals/reference/T5/journal.mddoc3,828 bytes
  • evals/reference/T5/prompt.mddoc5,542 bytes
  • evals/reference/T5/reference.mddoc14,194 bytes
  • evals/reference/T5/score.pyruns2,459 bytes
  • evals/reference/T6/journal.mddoc2,667 bytes
  • evals/reference/T6/prompt.mddoc4,424 bytes
  • evals/reference/T6/reference.mddoc5,501 bytes
  • evals/reference/oracle_churn.pyruns17,927 bytes
  • evals/runs/2026-10-21-gate/L2-A/action-log-planner.mddoc2,118 bytes
  • evals/runs/2026-10-21-gate/L2-A/action-log-verifier.mddoc5,538 bytes
  • evals/runs/2026-10-21-gate/L2-A/action-log-worker.mddoc13,535 bytes
  • evals/runs/2026-10-21-gate/L2-A/churn.pyruns60,392 bytes
  • evals/runs/2026-10-21-gate/L2-A/holdout_ids.txtdoc10,800 bytes
  • evals/runs/2026-10-21-gate/L2-A/journal.mddoc19,162 bytes
  • evals/runs/2026-10-21-gate/L2-A/planner-output.mddoc29,610 bytes
  • evals/runs/2026-10-21-gate/L2-A/post-run.mddoc2,271 bytes
  • evals/runs/2026-10-21-gate/L2-A/prompt-sections-2-6.mddoc5,260 bytes
  • evals/runs/2026-10-21-gate/L2-A/prompt-to-worker.mddoc21,901 bytes
  • evals/runs/2026-10-21-gate/L2-A/report.mddoc20,918 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/gate.jsonasset622 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/labels_D14.csvasset66,010 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/quality.jsonasset5,502 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/quality_cohorts_dev.csvasset384 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/quality_events_by_month.csvasset566 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/quality_extra.jsonasset3,739 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/refute.jsonasset1,887 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2.jsonasset34,707 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_cv_grid.csvasset2,159 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_holdout_metrics.csvasset980 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_holdout_predictions.csvasset66,276 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_importance.csvasset3,822 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_lr_coef.csvasset1,154 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_rank_stability.csvasset890 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_spearman_dev.csvasset1,529 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/stage2_stability_summary.csvasset1,240 bytes
  • evals/runs/2026-10-21-gate/L2-A/results/summary_all.jsonasset6,421 bytes
  • evals/runs/2026-10-21-gate/L2-A/verdict.mddoc11,399 bytes
  • evals/runs/2026-10-21-gate/L2-B/action-log.mddoc12,573 bytes
  • evals/runs/2026-10-21-gate/L2-B/answer.mddoc22,365 bytes
  • evals/runs/2026-10-21-gate/L2-B/checks_extra.pyruns5,486 bytes
  • evals/runs/2026-10-21-gate/L2-B/churn_pipeline.pyruns20,644 bytes
  • evals/runs/2026-10-21-gate/L2-B/features.pyruns6,539 bytes
  • evals/runs/2026-10-21-gate/L2-B/prereg.jsonasset1,914 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/audit_period_rerun.txtdoc364 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/audit_sensitivity_rerun.txtdoc752 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/bootstrap_ci.jsonasset311 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/calibration_deciles.csvasset228 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/checks_extra.jsonasset1,513 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/early_signal.jsonasset1,100 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/holdout_metrics.jsonasset947 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/holdout_metrics_production.jsonasset179 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/importance_families.csvasset497 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/importance_features.csvasset675 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/importance_robustness.jsonasset1,058 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/importance_solo.csvasset267 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/lead_time.jsonasset519 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/leakage_checks.jsonasset600 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/per_snapshot.csvasset540 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/prereg_check.jsonasset623 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/risk_list_2026-01-01.csvasset149,837 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/survival.jsonasset5,565 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/tuning.jsonasset1,421 bytes
  • evals/runs/2026-10-21-gate/L2-B/results/year_end_truncation_check.csvasset503 bytes
  • evals/runs/2026-10-21-gate/L2-B/robustness.pyruns3,204 bytes
  • evals/runs/2026-10-21-gate/L2-B/score.pyruns6,835 bytes
  • evals/runs/2026-10-21-gate/L2-B/survival.pyruns7,518 bytes
  • evals/runs/2026-10-21-gate/README.mddoc8,296 bytes
  • evals/runs/2026-10-21-gate/T3-A/action-log-planner.mddoc2,895 bytes
  • evals/runs/2026-10-21-gate/T3-A/action-log-verifier.mddoc2,567 bytes
  • evals/runs/2026-10-21-gate/T3-A/action-log-worker.mddoc11,282 bytes
  • evals/runs/2026-10-21-gate/T3-A/fix.diffasset3,158 bytes
  • evals/runs/2026-10-21-gate/T3-A/journal.mddoc60,631 bytes
  • evals/runs/2026-10-21-gate/T3-A/planner-output.mddoc23,195 bytes
  • evals/runs/2026-10-21-gate/T3-A/post-run.mddoc1,826 bytes
  • evals/runs/2026-10-21-gate/T3-A/prompt-sections-2-6.mddoc5,361 bytes
  • evals/runs/2026-10-21-gate/T3-A/prompt-to-worker.mddoc15,192 bytes
  • evals/runs/2026-10-21-gate/T3-A/report.mddoc14,680 bytes
  • evals/runs/2026-10-21-gate/T3-A/verdict.mddoc8,651 bytes
  • evals/runs/2026-10-21-gate/T3-A/worker-handback.mddoc16,042 bytes
  • evals/runs/2026-10-21-gate/T3-B/action-log.mddoc7,555 bytes
  • evals/runs/2026-10-21-gate/T3-B/answer.mddoc10,285 bytes
  • evals/runs/2026-10-21-gate/T3-B/fix.diffasset9,136 bytes
  • evals/runs/2026-10-21-gate/deviations.mddoc2,536 bytes
  • evals/runs/2026-10-21-gate/fixtures/gen_events.pyruns5,541 bytes
  • evals/runs/2026-10-21-gate/fixtures/hidden/t3-hidden.test.jsruns1,026 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/README.mddoc372 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/bin/open-screen.jsruns399 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/crash-reports/crash-48213.txtdoc528 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/crash-reports/crash-48377.txtdoc530 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/crash-reports/crash-48530.txtdoc532 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/devices/galaxy-s21-de-de.jsonasset162 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/devices/iphone-13-en-gb.jsonasset152 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/devices/pixel-7-en-us.jsonasset162 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/package.jsonruns292 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/app.jsruns427 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/currency.jsruns269 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/locale.jsruns306 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/storage.jsruns320 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/screens/home.jsruns328 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/screens/settings.jsruns1,288 bytes
  • evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/test/settings.test.jsruns1,179 bytes
  • evals/runs/2026-10-21-gate/protocol.mddoc9,307 bytes
  • evals/runs/2026-10-21-gate/scoring/L2-action-log-scorer.mddoc3,057 bytes
  • evals/runs/2026-10-21-gate/scoring/L2-mapping.txtdoc46 bytes
  • evals/runs/2026-10-21-gate/scoring/L2-score.mddoc14,158 bytes
  • evals/runs/2026-10-21-gate/scoring/T3-action-log-scorer.mddoc2,043 bytes
  • evals/runs/2026-10-21-gate/scoring/T3-mapping.txtdoc46 bytes
  • evals/runs/2026-10-21-gate/scoring/T3-score.mddoc7,939 bytes
  • evals/runs/2026-10-25-rescore-swap/L2-action-log-scorer.mddoc1,904 bytes
  • evals/runs/2026-10-25-rescore-swap/L2-mapping.txtdoc52 bytes
  • evals/runs/2026-10-25-rescore-swap/L2-score.mddoc14,194 bytes
  • evals/runs/2026-10-25-rescore-swap/README.mddoc2,459 bytes
  • evals/runs/2026-10-25-rescore-swap/T3-action-log-scorer.mddoc2,524 bytes
  • evals/runs/2026-10-25-rescore-swap/T3-mapping.txtdoc52 bytes
  • evals/runs/2026-10-25-rescore-swap/T3-score.mddoc6,990 bytes
  • evals/runs/2026-10-25-v011-verifier/L2-A/action-log.mddoc3,485 bytes
  • evals/runs/2026-10-25-v011-verifier/L2-A/verdict.mddoc10,009 bytes
  • evals/runs/2026-10-25-v011-verifier/README.mddoc1,780 bytes
  • evals/runs/2026-10-25-v011-verifier/T3-B-run1/action-log.mddoc3,053 bytes
  • evals/runs/2026-10-25-v011-verifier/T3-B-run1/verdict.mddoc6,587 bytes
  • evals/runs/2026-10-25-v011-verifier/T3-B-run2/action-log.mddoc2,376 bytes
  • evals/runs/2026-10-25-v011-verifier/T3-B-run2/verdict.mddoc5,672 bytes
  • evals/scorecard.mddoc3,015 bytes
  • evals/scorer.mddoc3,847 bytes
  • evals/tasks.mddoc7,951 bytes
  • scenarios/_index.mddoc1,895 bytes
  • scenarios/code-engineering.mddoc4,177 bytes
  • scenarios/data-analysis.mddoc4,924 bytes
  • scenarios/decision-plan.mddoc4,377 bytes
  • scenarios/design-ux.mddoc4,035 bytes
  • scenarios/generic.mddoc2,414 bytes
  • scenarios/writing-research.mddoc4,204 bytes
  • tools/gate-check.pyruns42,415 bytes
  • tools/lint-harness.pyruns22,011 bytes
  • tools/tests/fixtures/en/log-baseline.mddoc497 bytes
  • tools/tests/fixtures/en/log-conditional.mddoc1,560 bytes
  • tools/tests/fixtures/en/prompt-baseline.mddoc1,668 bytes
  • tools/tests/fixtures/en/prompt-conditional.mddoc3,127 bytes
  • tools/tests/test_gate_check.pyruns29,672 bytes
  • tools/word-budgets.tsvasset2,218 bytes
Read SKILL.md

# Be-Ambitious Harness

Three roles and one human. **Planner** turns the task into a prompt with two bars and a gate check. **Worker** executes the prompt, starting with the check. **Verifier** grades the result seeing only the goal, requirements, criteria and facts. **Human** chooses the scheme and cost limit — this is not delegated.

Each role reads only its own files. This is deliberate: instruction following breaks down past 5–6 simultaneous constraints, so structure comes from files and templates, not from rules held in mind.

## Flow

**0. Intake.** Collect four fields: `task`, `context`, `mode` (keep scope / ambitious extension / recommend), `stop_after_analysis` (yes/no). Defaults if empty: recommend, no.

**1. Routing.** Open `scenarios/_index.md`, pick a scenario by signals. Torn between two — take the one where the result lives. None fits — `generic.md`.

**2. Analysis and prompt.** Read `agents/planner.md` and the scenario file. Run the analysis; scenario hints serve their matching points, not all at once. Output questions and the prompt per `core/prompt-template.md`. With `stop_after_analysis = yes`, stop after analysis and questions; continue only after the human answers.
View the complete source on GitHub

Skills included

  • be-ambitious-harness
  • be-ambitious-harness

Repository facts

License
MIT
Created
Oct 20, 2026
Last pushed
Oct 25, 2026
Forks
0
Open issues
0
Project activity
Active

Before you install

Network or broad shell

Automated signals, not a guarantee

Checked Nov 5, 2026

Show all 173 signals by file

CITATION.cff

  • Network accessline 10
    date-released: 2026-10-25 license: MIT repository-code: "https://github.com/alexpiar/be-ambitious-harness"
  • Network accessline 11
    license: MIT repository-code: "https://github.com/alexpiar/be-ambitious-harness" abstract: >-
  • Network accessline 12
    repository-code: "https://github.com/alexpiar/be-ambitious-harness"

README.es.md

  • Network accessline 7
    ![Escenarios](https://img.shields.io/badge/escenarios-6-blue) ![Roles](https://img.shields.io/badge/roles-planificador%20%7C%20ejecutor%20%7C%20verificador-informational) ![Evaluaciones](https://img.shields.io/badge/evaluaciones-2%20de%208%
  • Network accessline 25
    **Instalación:** `git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness` y luego `/be-ambitious-harness` en Claude Code. Detalles y la versión en ruso — en [Instalación](#instalación).
  • Network accessline 31
    El 13 de octubre de 2026, Felix Rieseberg, gerente de ingeniería (engineering manager) en Anthropic que ha dirigido la ingeniería de Claude.ai, Cowork y Claude Code Desktop, publicó [*How to be more ambitious: A tutorial for humans with mod
  • Network accessline 33
    En pocos días aparecieron skills de la comunidad: [`be-ambitious` de DannyMac180](https://github.com/DannyMac180/skills) envuelve ese prompt, y [robonuggets/be-ambitious](https://github.com/robonuggets/be-ambitious) convierte la altitud, la
  • Network accessline 37
    | Modo de fallo | Qué dice la investigación | Qué hace el harness al respecto | |---|---|---| | **Saturación de restricciones** | El seguimiento fiable de instrucciones se viene abajo por encima de 5–6 restricciones simultáneas en una sola 
  • Network accessline 38
    |---|---|---| | **Saturación de restricciones** | El seguimiento fiable de instrucciones se viene abajo por encima de 5–6 restricciones simultáneas en una sola generación; en 15 modelos con k=8, cada restricción por separado se sigue cumpli
  • Network accessline 39
    | **Saturación de restricciones** | El seguimiento fiable de instrucciones se viene abajo por encima de 5–6 restricciones simultáneas en una sola generación; en 15 modelos con k=8, cada restricción por separado se sigue cumpliendo ~41% de l
  • Network accessline 40
    | **Las instrucciones negativas resultan contraproducentes** | Nombrar una palabra prohibida predispone al modelo a producirla; en los fallos, la señal de supresión es 4.4× más débil ([Rana, preprint de 2026](https://arxiv.org/abs/2601.0807
  • Network accessline 41
    | **Sesgo de autopreferencia** | Los jueces LLM puntúan más alto el texto de baja perplejidad que los evaluadores humanos, sin importar quién lo haya escrito ([Wataoka et al., NeurIPS 2024 Safe Generative AI Workshop](https://arxiv.org/abs/
  • Network accessline 59
    La investigación sobre fijación de objetivos muestra que los objetivos específicos y difíciles superan al "haz lo mejor que puedas" ([Locke & Latham, 2002](https://med.stanford.edu/content/dam/sm/s-spire/documents/PD.locke-and-latham-retros
  • Network accessline 149
    **Comprobación de entrada, 21 oct. 2026** — [registro completo](evals/runs/2026-10-21-gate/). Dos tareas de evaluación (T3: código; L2: la trampa "criterio frente a datos" sobre un conjunto de datos de pérdida de clientes), el harness frent
  • Network accessline 171
    git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness
  • Network accessline 179
    git clone https://github.com/alexpiar/be-ambitious-harness.git /tmp/be-ambitious-harness
  • Network accessline 262
    **¿Por qué un LLM no debería evaluar su propio trabajo?** Porque en el estudio de Wataoka et al., los jueces LLM puntuaron más alto que los evaluadores humanos el texto de baja perplejidad, es decir, familiar para el juez, lo hubieran escri
  • Network accessline 263
    Porque en el estudio de Wataoka et al., los jueces LLM puntuaron más alto que los evaluadores humanos el texto de baja perplejidad, es decir, familiar para el juez, lo hubieran escrito ellos o no; GPT-4 mostró la autopreferencia más fuerte 
  • Network accessline 284
    - F. Rieseberg — [How to be more ambitious: A tutorial for humans with models](https://felixrieseberg.com/how-to-be-more-ambitious-a-tutorial-for-humans-with-models/) (13 oct. 2026). Altitud / profundidad / amplitud; juez externo; el prompt
  • Network accessline 285
    - M. I. Vasileva — [Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction](https://arxiv.org/abs/2608.12426) (arXiv, 2026; póster en el 2nd Workshop on Compositional 
  • Network accessline 286
    - S. Rana — [Semantic Gravity Wells: Why Negative Constraints Backfire](https://arxiv.org/abs/2601.08070) (preprint de arXiv, 2026). Nombrar una palabra prohibida la vuelve más probable.
  • Network accessline 287
    - K. Wataoka, T. Takahashi, R. Ri — [Self-Preference Bias in LLM-as-a-Judge](https://arxiv.org/abs/2410.21819) (NeurIPS 2024 Safe Generative AI Workshop). Los jueces favorecen el texto de baja perplejidad.
  • Network accessline 289
    - G. Seijts, G. Latham — *The effect of distal learning, outcome, and proximal goals on a moderately complex task* (Journal of Organizational Behavior, 2001); *Learning versus performance goals: When should each be used?* (Academy of Manage
  • Network accessline 291
    - N. Edwards, S. Schuster — [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://arxiv.org/abs/2603.26233) (arXiv, 2026). En una versión subespecificada de SWE-bench Verified, separar la detección de la subespec

README.md

  • Network accessline 7
    ![Scenarios](https://img.shields.io/badge/scenarios-6-blue) ![Roles](https://img.shields.io/badge/roles-planner%20%7C%20worker%20%7C%20verifier-informational) ![Eval runs](https://img.shields.io/badge/eval%20runs-2%20of%208%20tasks-yellow) 
  • Network accessline 25
    **Install:** `git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness`, then `/be-ambitious-harness` in Claude Code. Details and the Russian version — [Install](#install).
  • Network accessline 33
    On 13 October 2026 Felix Rieseberg, an engineering manager at Anthropic who has led engineering for Claude.ai, Cowork and Claude Code Desktop, published [*How to be more ambitious: A tutorial for humans with models*](https://felixrieseberg.
  • Network accessline 35
    Community skills followed within days: [DannyMac180's `be-ambitious`](https://github.com/DannyMac180/skills) wraps that prompt, and [robonuggets/be-ambitious](https://github.com/robonuggets/be-ambitious) turns altitude, depth and range into
  • Network accessline 39
    | Failure mode | What the research says | What the harness does about it | |---|---|---| | **Constraint saturation** | Reliable instruction following breaks down beyond 5–6 simultaneous constraints in a single generation; across 15 models a
  • Network accessline 40
    |---|---|---| | **Constraint saturation** | Reliable instruction following breaks down beyond 5–6 simultaneous constraints in a single generation; across 15 models at k=8, individual constraints still pass ~41% of the time, but all eight ho
  • Network accessline 41
    | **Constraint saturation** | Reliable instruction following breaks down beyond 5–6 simultaneous constraints in a single generation; across 15 models at k=8, individual constraints still pass ~41% of the time, but all eight hold together on
  • Network accessline 42
    | **Negative instructions backfire** | Naming a forbidden word primes the model to produce it; in failures the suppression signal is 4.4× weaker ([Rana, 2026 preprint](https://arxiv.org/abs/2601.08070); one model, Qwen2.5-7B-Instruct) | Ins
  • Network accessline 43
    | **Self-preference bias** | LLM judges rate low-perplexity text higher than human evaluators do, regardless of who wrote it ([Wataoka et al., NeurIPS 2024 Safe Generative AI Workshop](https://arxiv.org/abs/2410.21819)) | The verifier sees 
  • Network accessline 61
    Goal-setting research shows that specific, difficult goals beat "do your best" ([Locke & Latham, 2002](https://med.stanford.edu/content/dam/sm/s-spire/documents/PD.locke-and-latham-retrospective_Paper.pdf)). But where a task first requires 
  • Network accessline 151
    **Gate check, 21 Oct 2026** — [full record](evals/runs/2026-10-21-gate/). Two eval tasks (T3: code; L2: the "criterion vs data" trap on a churn dataset), the harness against the single-skill [robonuggets/be-ambitious](https://github.com/rob
  • Network accessline 174
    git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness
  • Network accessline 180
    git clone https://github.com/alexpiar/be-ambitious-harness.git /tmp/be-ambitious-harness
  • Network accessline 267
    **Why shouldn't an LLM grade its own work?** Because in Wataoka et al.'s study LLM judges rated low-perplexity — familiar-to-the-judge — text higher than human evaluators did, whether or not they wrote it; GPT-4 showed the strongest self-pr
  • Network accessline 268
    Because in Wataoka et al.'s study LLM judges rated low-perplexity — familiar-to-the-judge — text higher than human evaluators did, whether or not they wrote it; GPT-4 showed the strongest self-preference ([Wataoka et al.](https://arxiv.org/
  • Network accessline 289
    - F. Rieseberg — [How to be more ambitious: A tutorial for humans with models](https://felixrieseberg.com/how-to-be-more-ambitious-a-tutorial-for-humans-with-models/) (13 Oct 2026). Altitude / depth / range; outside judge; the original prom
  • Network accessline 290
    - M. I. Vasileva — [Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction](https://arxiv.org/abs/2608.12426) (arXiv, 2026; poster at the 2nd Workshop on Compositional
  • Network accessline 291
    - S. Rana — [Semantic Gravity Wells: Why Negative Constraints Backfire](https://arxiv.org/abs/2601.08070) (arXiv preprint, 2026). Naming a forbidden word primes it.
  • Network accessline 292
    - K. Wataoka, T. Takahashi, R. Ri — [Self-Preference Bias in LLM-as-a-Judge](https://arxiv.org/abs/2410.21819) (NeurIPS 2024 Safe Generative AI Workshop). Judges favor low-perplexity text.
  • Network accessline 294
    - G. Seijts, G. Latham — *The effect of distal learning, outcome, and proximal goals on a moderately complex task* (Journal of Organizational Behavior, 2001); *Learning versus performance goals: When should each be used?* (Academy of Manage
  • Network accessline 296
    - N. Edwards, S. Schuster — [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://arxiv.org/abs/2603.26233) (arXiv, 2026). On an underspecified SWE-bench Verified, splitting underspecification detection from code

README.ru.md

  • Network accessline 7
    ![Сценарии](https://img.shields.io/badge/%D1%81%D1%86%D0%B5%D0%BD%D0%B0%D1%80%D0%B8%D0%B8-6-blue) ![Роли](https://img.shields.io/badge/%D1%80%D0%BE%D0%BB%D0%B8-%D0%BF%D0%BB%D0%B0%D0%BD%D0%B8%D1%80%D0%BE%D0%B2%D1%89%D0%B8%D0%BA%20%7C%20%D0%B
  • Network accessline 25
    **Установка:** `git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness` — и `/be-ambitious-harness` в Claude Code. Подробности и русская версия файлов — в разделе [Установка](#установка).
  • Network accessline 33
    13 октября 2026 года Феликс Ризеберг — инженерный менеджер Anthropic, руководивший разработкой Claude.ai, Cowork и Claude Code Desktop, — опубликовал [руководство о том, как быть амбициознее с ИИ](https://felixrieseberg.com/how-to-be-more-a
  • Network accessline 35
    За несколько дней появились скиллы сообщества: [`be-ambitious` от DannyMac180](https://github.com/DannyMac180/skills) оборачивает этот промпт, а [robonuggets/be-ambitious](https://github.com/robonuggets/be-ambitious) превращает высоту, глуб
  • Network accessline 39
    | Ограничение | Что показывают исследования | Что делает харнесс | |---|---|---| | **Насыщение ограничениями** | Надёжное соблюдение инструкций рушится после 5–6 одновременных ограничений в одном ответе; по 15 моделям при k=8 отдельные огра
  • Network accessline 40
    |---|---|---| | **Насыщение ограничениями** | Надёжное соблюдение инструкций рушится после 5–6 одновременных ограничений в одном ответе; по 15 моделям при k=8 отдельные ограничения выполняются в ~41 % случаев, а все восемь сразу — лишь в 5,
  • Network accessline 41
    | **Насыщение ограничениями** | Надёжное соблюдение инструкций рушится после 5–6 одновременных ограничений в одном ответе; по 15 моделям при k=8 отдельные ограничения выполняются в ~41 % случаев, а все восемь сразу — лишь в 5,7 % ([Vasileva
  • Network accessline 42
    | **Негативные инструкции подсвечивают запрещённое** | Упоминание запрещённого слова праймит модель на его выдачу: в 87,5 % нарушений инструкция активирует, а не подавляет целевое слово, и в провалах подавление в 4,4 раза слабее ([Rana, пре
  • Network accessline 43
    | **Самопредпочтение** | LLM-судьи ставят привычному для них тексту (с низкой перплексией) оценки выше, чем люди, — независимо от того, кто его написал ([Wataoka et al., воркшоп Safe Generative AI, NeurIPS 2024](https://arxiv.org/abs/2410.2
  • Network accessline 61
    Конкретные трудные цели работают лучше, чем «сделай как можно лучше» ([Locke & Latham, 2002](https://med.stanford.edu/content/dam/sm/s-spire/documents/PD.locke-and-latham-retrospective_Paper.pdf)). Но там, где сначала нужно освоить знания и
  • Network accessline 151
    **Входная проверка запуска, 21.10.2026** — [полная запись](evals/runs/2026-10-21-gate/). Две задачи из тестов (T3 — код; L2 — ловушка «критерий под данные» на данных об оттоке): харнесс против одиночного скилла [robonuggets/be-ambitious](ht
  • Network accessline 174
    git clone https://github.com/alexpiar/be-ambitious-harness.git /tmp/be-ambitious-harness
  • Network accessline 181
    git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness
  • Network accessline 269
    **Почему модели нельзя оценивать собственную работу?** Потому что в исследовании Wataoka et al. LLM-судьи ставили привычному для них тексту (с низкой перплексией) оценки выше, чем люди, — независимо от того, кто его написал; сильнее всего с
  • Network accessline 270
    Потому что в исследовании Wataoka et al. LLM-судьи ставили привычному для них тексту (с низкой перплексией) оценки выше, чем люди, — независимо от того, кто его написал; сильнее всего самопредпочтение проявилось у GPT-4 ([Wataoka et al.](ht
  • Network accessline 291
    - F. Rieseberg — [How to be more ambitious: A tutorial for humans with models](https://felixrieseberg.com/how-to-be-more-ambitious-a-tutorial-for-humans-with-models/) (13.10.2026). Высота, глубина, широта; внешний судья; исходный промпт.
  • Network accessline 292
    - M. I. Vasileva — [Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction](https://arxiv.org/abs/2608.12426) (arXiv, 2026; постер 2nd Workshop on Compositional Learni
  • Network accessline 293
    - S. Rana — [Semantic Gravity Wells: Why Negative Constraints Backfire](https://arxiv.org/abs/2601.08070) (препринт arXiv, 2026). Упоминание запрещённого слова его праймит.
  • Network accessline 294
    - K. Wataoka, T. Takahashi, R. Ri — [Self-Preference Bias in LLM-as-a-Judge](https://arxiv.org/abs/2410.21819) (воркшоп Safe Generative AI, NeurIPS 2024). Судьи предпочитают текст с низкой перплексией.
  • Network accessline 296
    - G. Seijts, G. Latham — *The effect of distal learning, outcome, and proximal goals on a moderately complex task* (Journal of Organizational Behavior, 2001); *Learning versus performance goals: When should each be used?* (Academy of Manage
  • Network accessline 298
    - N. Edwards, S. Schuster — [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://arxiv.org/abs/2603.26233) (arXiv, 2026). На недоопределённом SWE-bench Verified разделение детекции недоопределённости и исполнени

README.zh-CN.md

  • Network accessline 7
    ![场景](https://img.shields.io/badge/%E5%9C%BA%E6%99%AF-6-blue) ![角色](https://img.shields.io/badge/%E8%A7%92%E8%89%B2-%E8%A7%84%E5%88%92%E8%80%85%20%7C%20%E6%89%A7%E8%A1%8C%E8%80%85%20%7C%20%E9%AA%8C%E8%AF%81%E8%80%85-informational) ![评测运行](h
  • Network accessline 25
    **安装:**`git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness`,然后在 Claude Code 中输入 `/be-ambitious-harness`。详细说明及俄文版本见[安装](#安装)。
  • Network accessline 31
    2026 年 10 月 13 日,Anthropic 的工程经理 Felix Rieseberg(曾带领 Claude.ai、Cowork 和 Claude Code Desktop 的工程团队)发表了 [*How to be more ambitious: A tutorial for humans with models*](https://felixrieseberg.com/how-to-be-more-ambitious-a-tutorial-for-humans-
  • Network accessline 33
    几天之内,社区技能相继出现:[DannyMac180 的 `be-ambitious`](https://github.com/DannyMac180/skills) 封装了这段提示词;[robonuggets/be-ambitious](https://github.com/robonuggets/be-ambitious) 则把高度、深度和广度拆成步骤,然后直接动手完成工作。
  • Network accessline 37
    | 失效模式 | 研究怎么说 | Harness 如何应对 | |---|---|---| | **约束饱和** | 单次生成中同时存在的约束超过 5–6 条后,可靠的指令遵循就会崩溃;在 15 个模型、k=8 的条件下,单条约束的通过率仍有约 41%,但八条同时满足的比例只有 5.7%([Vasileva, arXiv:2608.12426](https://arxiv.org/abs/2608.12426);ICML 2026 研讨会海报) | 每个角色只拿到自己这一步的
  • Network accessline 38
    |---|---|---| | **约束饱和** | 单次生成中同时存在的约束超过 5–6 条后,可靠的指令遵循就会崩溃;在 15 个模型、k=8 的条件下,单条约束的通过率仍有约 41%,但八条同时满足的比例只有 5.7%([Vasileva, arXiv:2608.12426](https://arxiv.org/abs/2608.12426);ICML 2026 研讨会海报) | 每个角色只拿到自己这一步的文件(每个 250–630 词);结构来自模板,而不是靠上下文里
  • Network accessline 39
    | **约束饱和** | 单次生成中同时存在的约束超过 5–6 条后,可靠的指令遵循就会崩溃;在 15 个模型、k=8 的条件下,单条约束的通过率仍有约 41%,但八条同时满足的比例只有 5.7%([Vasileva, arXiv:2608.12426](https://arxiv.org/abs/2608.12426);ICML 2026 研讨会海报) | 每个角色只拿到自己这一步的文件(每个 250–630 词);结构来自模板,而不是靠上下文里记着的规则 |
  • Network accessline 40
    | **否定式指令适得其反** | 点名一个禁用词,反而会促使模型把它生成出来;在失败案例中,抑制信号弱 4.4 倍([Rana,2026 年预印本](https://arxiv.org/abs/2601.08070);仅测试了一个模型:Qwen2.5-7B-Instruct) | 指令一律采用正面表述:用“只包含模板中的各部分”代替“不要包含分析” |
  • Network accessline 41
    | **自我偏好偏差** | 无论文本出自谁手,LLM 评审给低困惑度文本的评分都高于人类评估者([Wataoka et al., NeurIPS 2024 Safe Generative AI Workshop](https://arxiv.org/abs/2410.21819)) | 验证者只看目标、需求、标准、结果和证据,并且最好运行在另一个模型上——真正能改变评审所感知困惑度的正是这一点 |
  • Network accessline 59
    目标设定研究表明,具体且有难度的目标胜过“尽力而为”([Locke & Latham, 2002](https://med.stanford.edu/content/dam/sm/s-spire/documents/PD.locke-and-latham-retrospective_Paper.pdf))。但如果任务首先需要获取新的知识或策略,具体且有难度的*绩效*目标可能还不如“尽力而为”,而具体且有难度的*学习*目标则效果更好(Seijts & Latham, 2001;
  • Network accessline 149
    **关卡检查,2026 年 10 月 21 日**——[完整记录](evals/runs/2026-10-21-gate/)。两项评测任务(T3:代码;L2:用户流失数据集上的“标准 vs 数据”陷阱),在同一个模型上让本 Harness 与单一技能 [robonuggets/be-ambitious](https://github.com/robonuggets/be-ambitious) 对比,由运行在另一个模型上的盲评评分者打分。规程和通过标准在第一次运行之前就已提交。
  • Network accessline 171
    git clone https://github.com/alexpiar/be-ambitious-harness.git ~/.claude/skills/be-ambitious-harness
  • Network accessline 179
    git clone https://github.com/alexpiar/be-ambitious-harness.git /tmp/be-ambitious-harness
  • Network accessline 262
    **为什么不应该让 LLM 给自己的工作打分?** 因为在 Wataoka 等人的研究中,LLM 评审给低困惑度文本(也就是评审自己“眼熟”的文本)打的分高于人类评估者,无论这些文本是不是它们自己写的;其中 GPT-4 的自我偏好最强([Wataoka et al.](https://arxiv.org/abs/2410.21819))。在本 Harness 中,验证者永远看不到执行者的推理过程,理想情况下还运行在另一个模型上。
  • Network accessline 263
    因为在 Wataoka 等人的研究中,LLM 评审给低困惑度文本(也就是评审自己“眼熟”的文本)打的分高于人类评估者,无论这些文本是不是它们自己写的;其中 GPT-4 的自我偏好最强([Wataoka et al.](https://arxiv.org/abs/2410.21819))。在本 Harness 中,验证者永远看不到执行者的推理过程,理想情况下还运行在另一个模型上。
  • Network accessline 284
    - F. Rieseberg — [How to be more ambitious: A tutorial for humans with models](https://felixrieseberg.com/how-to-be-more-ambitious-a-tutorial-for-humans-with-models/)(2026 年 10 月 13 日)。高度 / 深度 / 广度;外部评审;最初的那段提示词。
  • Network accessline 285
    - M. I. Vasileva — [Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction](https://arxiv.org/abs/2608.12426)(arXiv,2026;ICML 2026 的 2nd Workshop on Compositional Lear
  • Network accessline 286
    - S. Rana — [Semantic Gravity Wells: Why Negative Constraints Backfire](https://arxiv.org/abs/2601.08070)(arXiv 预印本,2026)。点名一个禁用词,反而会诱发它。
  • Network accessline 287
    - K. Wataoka, T. Takahashi, R. Ri — [Self-Preference Bias in LLM-as-a-Judge](https://arxiv.org/abs/2410.21819)(NeurIPS 2024 Safe Generative AI Workshop)。评审偏爱低困惑度的文本。
  • Network accessline 289
    - G. Seijts, G. Latham — *The effect of distal learning, outcome, and proximal goals on a moderately complex task*(Journal of Organizational Behavior,2001);*Learning versus performance goals: When should each be used?*(Academy of Management
  • Network accessline 291
    - N. Edwards, S. Schuster — [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://arxiv.org/abs/2603.26233)(arXiv,2026)。在需求描述不充分的 SWE-bench Verified 上,把“识别需求欠明确之处”与“执行代码”拆开后,一个具备不确定性感知的智能体的解决率从 61.2% 提高到 69.4%(Op

evals/fixtures/calibrate_t2pass.py

  • Executable scriptline 1
    Executable file
  • Access to secretsline 25
    [Credential-related source omitted]

evals/fixtures/gen_events.py

  • Executable scriptline 1
    Executable file

evals/reference/L2/ref_lib.py

  • Executable scriptline 1
    Executable file

evals/reference/L2/run.py

  • Executable scriptline 1
    Executable file

evals/reference/L2/score.py

  • Executable scriptline 1
    Executable file

evals/reference/README.md

  • Network accessline 3
    Эталон — решение задачи из `evals/tasks.md`, которое проходит все её условия «Провал» и скрытый оракул, если он у задачи есть. Он доказывает две вещи: задача решаема, а грейдеры настроены — правильное решение они принимают. Методика — Anthr

evals/reference/T1/gate.py

  • Executable scriptline 1
    #!/usr/bin/env python3

evals/reference/T2-pass/ref_lib.py

  • Executable scriptline 1
    Executable file

evals/reference/T2-pass/run.py

  • Executable scriptline 1
    Executable file

evals/reference/T2-pass/score.py

  • Executable scriptline 1
    Executable file

evals/reference/T2/ref_lib.py

  • Executable scriptline 1
    Executable file

evals/reference/T2/run.py

  • Executable scriptline 1
    Executable file

evals/reference/T2/score.py

  • Executable scriptline 1
    Executable file

evals/reference/T3/check.sh

  • Executable scriptline 1
    #!/usr/bin/env bash
  • Dangerous shell operationline 15
    trap 'rm -rf "$WORK"' EXIT

evals/reference/T3/test/settings-locale-without-region.test.js

  • Executable scriptline 1
    Executable file

evals/reference/T4/article.md

  • Network accessline 41
    1. Eurostat. Use of artificial intelligence in enterprises. Statistics Explained, данные извлечены в декабре 2025. https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Use_of_artificial_intelligence_in_enterprises
  • Network accessline 42
    2. U.S. Census Bureau. Large Firms With at Least 20 Employees Biggest AI Users. 26.05.2026. https://www.census.gov/library/stories/2026/05/ai-use-businesses.html
  • Network accessline 43
    3. Allen J. S. Monitoring AI Adoption in the US Economy. FEDS Notes, Board of Governors of the Federal Reserve System, 03.04.2026. https://www.federalreserve.gov/econres/notes/feds-notes/monitoring-ai-adoption-in-the-u-s-economy-20260403.ht
  • Network accessline 44
    4. KPMG. AI's Value Story Sharpens as Organizations Gain Confidence in Governance, Accountability and Workforce Adoption (Q3 2026 AI Quarterly Pulse Survey). 24.09.2026. https://kpmg.com/us/en/media/news/q3-ai-pulse-2026.html

evals/reference/T4/journal.md

  • Network accessline 16
    действие:            поиск первичных источников о доле компаний с ИИ или ИИ-агентами; каждый открыт, цитата с числом найдена в тексте страницы материалы:           веб-поиск; curl наблюдаемый результат: первичных источников с данными за пос
  • Network accessline 17
    материалы:           веб-поиск; curl наблюдаемый результат: первичных источников с данными за последние 12 месяцев — 3: Евростат (декабрь 2025), Бюро переписи США (май 2026),                      KPMG (сентябрь 2026); плюс анализ ФРС (апрел

evals/reference/T4/sources.md

  • Network accessline 7
    | № | Источник | Тип | Что подтверждает в статье | Цитата | |---|---|---|---|---| | 1 | Eurostat, «Use of artificial intelligence in enterprises», Statistics Explained; данные извлечены в декабре 2025, следующее обновление — декабрь 2026. h
  • Network accessline 8
    |---|---|---|---|---| | 1 | Eurostat, «Use of artificial intelligence in enterprises», Statistics Explained; данные извлечены в декабре 2025, следующее обновление — декабрь 2026. https://ec.europa.eu/eurostat/statistics-explained/index.php?
  • Network accessline 9
    | 1 | Eurostat, «Use of artificial intelligence in enterprises», Statistics Explained; данные извлечены в декабре 2025, следующее обновление — декабрь 2026. https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Use_of_artificia
  • Network accessline 12
    | 1 | там же | | в ИИ входят генерация текста и кода и автоматизация процессов | «technologies generating written, spoken language or programming codes»; «technologies automating different workflows or assisting in decision-making» | | 1 | 
  • Network accessline 13
    | 1 | там же | | выборка по всей экономике | «In 2025, 157 000 of the 1.53 million enterprises in the EU were surveyed.» | | 2 | U.S. Census Bureau, «Large Firms With at Least 20 Employees Biggest AI Users», America Counts, 26.05.2026; данн
  • Network accessline 14
    | 2 | U.S. Census Bureau, «Large Firms With at Least 20 Employees Biggest AI Users», America Counts, 26.05.2026; данные BTOS 14.12.2025–03.05.2026. https://www.census.gov/library/stories/2026/05/ai-use-businesses.html | официальная статисти
  • Network accessline 15
    | 2 | там же | | 37% компаний с 250+ сотрудниками | «37% of firms with at least 250 employees reported using AI in their business operations» | | 2 | там же | | выборка репрезентативна для страны | «The survey provides a biweekly, nationall
  • Network accessline 16
    | 2 | там же | | выборка репрезентативна для страны | «The survey provides a biweekly, nationally representative view of AI implementation» | | 3 | Allen J. S., «Monitoring AI Adoption in the US Economy», FEDS Notes, Board of Governors of t
  • Network accessline 17
    | 3 | Allen J. S., «Monitoring AI Adoption in the US Economy», FEDS Notes, Board of Governors of the Federal Reserve System, 03.04.2026. https://www.federalreserve.gov/econres/notes/feds-notes/monitoring-ai-adoption-in-the-u-s-economy-20260
  • Network accessline 18
    | 3 | там же | | 78% и 54% работников — в фирмах с ИИ и с языковыми моделями | «estimates that 78 percent of the labor force works at firms that have adopted AI, and about 54 percent works at firms that use LLMs» | | 4 | KPMG, «AI's Value S
  • Network accessline 19
    | 4 | KPMG, «AI's Value Story Sharpens as Organizations Gain Confidence in Governance, Accountability and Workforce Adoption», пресс-релиз Q3 2026 AI Quarterly Pulse Survey, 24.09.2026. https://kpmg.com/us/en/media/news/q3-ai-pulse-2026.htm

evals/reference/T5/score.py

  • Executable scriptline 1
    #!/usr/bin/env python3

evals/reference/oracle_churn.py

  • Executable scriptline 1
    Executable file
  • Access to secretsline 152
    [Credential-related source omitted]

evals/runs/2026-10-21-gate/L2-A/action-log-worker.md

  • Dangerous shell operationline 42
    | 34 | 23:44:56 | Bash | `R=<gate>/runs/L2-A; PY=<gate>/venv/bin/python; cd $R/out && rm -rf results tmp/results_run2 && date +%H:%M:%S && OMP_NUM_THREADS=2 $PY -I churn.py all --csv ../data/events_2025.csv --out results > tmp/all_run1.txt 

evals/runs/2026-10-21-gate/L2-A/churn.py

  • Executable scriptline 1
    Executable file
  • Access to secretsline 25
    [Credential-related source omitted]

evals/runs/2026-10-21-gate/L2-B/action-log.md

  • Access to secretsline 39
    [Credential-related source omitted]
  • Access to secretsline 62
    [Credential-related source omitted]

evals/runs/2026-10-21-gate/L2-B/checks_extra.py

  • Executable scriptline 1
    Executable file
  • Access to secretsline 16
    [Credential-related source omitted]

evals/runs/2026-10-21-gate/L2-B/churn_pipeline.py

  • Executable scriptline 1
    Executable file
  • Access to secretsline 15
    [Credential-related source omitted]

evals/runs/2026-10-21-gate/L2-B/features.py

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/L2-B/robustness.py

  • Executable scriptline 1
    Executable file
  • Access to secretsline 12
    [Credential-related source omitted]

evals/runs/2026-10-21-gate/L2-B/score.py

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/L2-B/survival.py

  • Executable scriptline 1
    Executable file
  • Network accessline 105
    elig = cten[cten >= (wk + 1) * 7].index     rate[wk] = round(float(((ses_c.uid.isin(elig)) & (ses_c.wk_before == wk)).sum() / len(elig)), 2) nc = u[~u.event]
  • Network accessline 106
    rate[wk] = round(float(((ses_c.uid.isin(elig)) & (ses_c.wk_before == wk)).sum() / len(elig)), 2) nc = u[~u.event] R["sessions_per_week_churners_by_weeks_before_cancel"] = rate
  • Network accessline 107
    nc = u[~u.event]

evals/runs/2026-10-21-gate/README.md

  • Network accessline 11
    |---|---|---| | **A — harness** | the full flow from `ru/SKILL.md`: planner → pre-flight → worker (fresh session) → verifier (fresh session) | planner and worker Claude Opus 5.5, verifier Claude Sonnet 5.5 | | **B — single skill** | [`robon
  • Network accessline 13
    | **B — single skill** | [`robonuggets/be-ambitious`](https://github.com/robonuggets/be-ambitious) `SKILL.md` at commit `dc116a3`; the skill rewrites the task and does the work | Claude Opus 5.5 |
  • Network accessline 58
    - `T3-A/`, `T3-B/`, `L2-A/`, `L2-B/` — prompts, journals, reports, verdicts, diffs, scripts, action logs. - `scoring/` — blind scores, scorer action logs, coin-flip mappings. - `*/tmp/` — the arms' working files (intermediate outputs, scrat
  • Network accessline 59
    - `scoring/` — blind scores, scorer action logs, coin-flip mappings. - `*/tmp/` — the arms' working files (intermediate outputs, scratch scripts, second-run results): moved out of the tree on 25 Oct 2026 and kept unchanged as a release asse
  • Network accessline 60
    - `*/tmp/` — the arms' working files (intermediate outputs, scratch scripts, second-run results): moved out of the tree on 25 Oct 2026 and kept unchanged as a release asset, [`gate-2026-10-21-tmp.tar.gz`](https://github.com/alexpiar/be-ambi

evals/runs/2026-10-21-gate/T3-A/action-log-worker.md

  • Dangerous shell operationline 31
    | 23 | 23:02:49 | Bash | `B=<gate>/runs/T3-A; J=$B/out/tmp/bin/jrun.sh rm -rf $B/out/tmp/clone-fix cat >> $B/out/journal.md <<EOF ### 5.5 Как у верификатора: чистые клоны (а) Новые тесты поверх 24f9d91 без фикса — клон \out/tmp/clone-base\,
  • Dangerous shell operationline 37
    | 29 | 23:03:53 | Bash | `B=<gate>/runs/T3-A; J=$B/out/tmp/bin/jrun.sh rm -rf $B/out/tmp/clone-fix $B/out/tmp/clone-base $B/out/tmp/after/* cat >> $B/out/journal.md <<EOF ## $(date -u +%H:%M:%S) — 7. Повтор самопроверки на финальном коммите

evals/runs/2026-10-21-gate/T3-B/action-log.md

  • Dangerous shell operationline 23
    | 15 | 23:15:42 | Bash | `cd <gate>/runs/T3-B && rm -rf out/tmp/before && mkdir -p out/tmp/before && cp -r repo/package.json repo/devices repo/test out/tmp/before/ && (cd repo && git archive HEAD src) / tar -x -C out/tmp/before && cd out/tm
  • Dangerous shell operationline 25
    | 17 | 23:16:05 | Bash | `cd <gate>/runs/T3-B && cat > out/tmp/mutate.sh <<'EOF' #!/usr/bin/env bash # Мутационная проверка: ломаем парсер по-разному и смотрим, ловят ли это тесты set -u SRC=$1; WORK=$2 run() { local name=$1 expr=$2 rm -rf 
  • Dangerous shell operationline 29
    | 21 | 23:17:07 | Bash | `cd <gate>/runs/T3-B && rm -rf out/tmp/count && mkdir -p out/tmp/count && cp -r repo/package.json repo/src repo/devices repo/test out/tmp/count/ && cd out/tmp/count && perl -0pi -e 's/(\s+)if \(icu\.toString\(\) !==

evals/runs/2026-10-21-gate/fixtures/gen_events.py

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/hidden/t3-hidden.test.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/bin/open-screen.js

  • Executable scriptline 1
    #!/usr/bin/env node

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/package.json

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/app.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/currency.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/locale.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/lib/storage.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/screens/home.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/src/screens/settings.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/fixtures/t3-acme-settings/test/settings.test.js

  • Executable scriptline 1
    Executable file

evals/runs/2026-10-21-gate/protocol.md

  • Network accessline 12
    |---|---|---| | **A — харнесс** | полный поток по `ru/SKILL.md`: планировщик → pre-flight (оркестратор) → воркер в новой сессии → верификатор в новой сессии | планировщик и воркер — Claude Opus 5.5; верификатор — Claude Sonnet 5.5 | | **B —
  • Network accessline 14
    | **B — одиночный скилл** | `SKILL.md` из [robonuggets/be-ambitious](https://github.com/robonuggets/be-ambitious), коммит `dc116a3ebd8cf772e4d0930626f67369bd60399d`, sha256 `b180b90b…3c5b`; скилл сам переписывает задачу и выполняет её | Cla

evals/runs/2026-10-25-v011-verifier/T3-B-run1/action-log.md

  • Dangerous shell operationline 12
    | 4 | 17:48:09 | Bash | `V=$V; cd $V/verifier; rm -rf clone; git clone -q $V/repo clone; cd clone; git checkout -q fix/settings-crash; git diff base HEAD / diff - $V/result/fix.diff && echo SAMEDIFF; git status --short; node -v echo "--- np

evals/runs/2026-10-25-v011-verifier/T3-B-run2/action-log.md

  • Dangerous shell operationline 12
    | 4 | 17:50:52 | Bash | `V=$V; mkdir -p $V/verifier; cd $V/verifier; rm -rf clone; git clone -q $V/repo clone; cd clone; git checkout -q fix/settings-crash; npm test 2>&1 / tail -12; node -e "for (const l of ['de','pt','fil']) console.log(J

tools/gate-check.py

  • Executable scriptline 1
    #!/usr/bin/env python3
  • Executable scriptline 1
    #!/usr/bin/env python3

tools/lint-harness.py

  • Executable scriptline 1
    #!/usr/bin/env python3
  • Network accessline 460
    seen = False         for n, line in enumerate(lines, 1):             for m in re.finditer(r"https://img\.shields\.io/badge/([^)\s\"'>?]+)", line):
  • Network accessline 461
    for n, line in enumerate(lines, 1):             for m in re.finditer(r"https://img\.shields\.io/badge/([^)\s\"'>?]+)", line):                 parts = split_badge(m.group(1))
  • Network accessline 462
    for m in re.finditer(r"https://img\.shields\.io/badge/([^)\s\"'>?]+)", line):

tools/tests/test_gate_check.py

  • Executable scriptline 1
    Executable file

How the signals fit together

Facts from a static read of the files at the pinned commit. They describe what the code does, not why.

Some files could not be read; the facts below still hold.

Signal confidence: 170 facts

and 120 more

Works with

Agent compatibility, from the files in this repository
AgentStatusEvidenceInstall to
Claude CodeCompatibleStandard SKILL.md SKILL.md.claude/skills/<skill-name>/
CodexCompatibleStandard SKILL.md SKILL.md.agents/skills/<skill-name>/
CursorCompatibleStandard SKILL.md SKILL.md.cursor/skills/<skill-name>/
Gemini CLICompatibleStandard SKILL.md SKILL.md.gemini/skills/<skill-name>/
GitHub CopilotCompatibleStandard SKILL.md SKILL.md.github/skills/<skill-name>/
OpenCodeCompatibleStandard SKILL.md SKILL.md.opencode/skills/<skill-name>/
  • Native The repository ships a manifest, folder or frontmatter field for this agent.
  • Compatible A standard SKILL.md that this agent can read.
  • Unknown No evidence either way.

Derived automatically from paths, manifests and frontmatter. Not tested on each agent.