Individual skill

agentforce-offline-experimentation-run

Use this skill when users want to execute this workflow when a user wants to test how a different LLM model would handle existing Agentforce agent conversations. Trigger when users mention agentforce, offline, experimentation.

skills/agentforce-offline-experimentation-run/SKILL.md

Read pinned source

Source commit a8a18ce1e4ba12e69b4073eab98ce31bfbd8271e

What was checked

Matches the observed repository HEAD: Yes
Scanner rules are current: Yes
Repository scan complete: No
Skill scan complete: No
Observed behavior: Network or broad shell
Last checked: Nov 10, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings): 1,151
Repository last updated: Nov 6, 2026
License: Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

operations Derived category
testing Derived category

Declared tags are partial or unavailable.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files

SKILL.mdcore · 9,102 bytes
scripts/bt_compare.pyruns · 4,074 bytes
scripts/bt_fetch.pyruns · 20,060 bytes
scripts/bt_patch_agent.pyruns · 4,449 bytes
scripts/btlib/__init__.pyruns · 1,920 bytes
scripts/btlib/agentpatch.pyruns · 21,936 bytes
scripts/btlib/agentsrc.pyruns · 21,986 bytes
scripts/btlib/cache.pyruns · 3,940 bytes
scripts/btlib/compare.pyruns · 31,807 bytes
scripts/btlib/fetch.pyruns · 49,201 bytes
scripts/btlib/judge.pyruns · 6,282 bytes
scripts/btlib/messages.pyruns · 3,501 bytes
scripts/btlib/patch.pyruns · 15,170 bytes
scripts/btlib/redact.pyruns · 1,276 bytes
scripts/btlib/results.pyruns · 13,091 bytes
scripts/btlib/scoring.pyruns · 15,741 bytes
scripts/btlib/sfap.pyruns · 11,096 bytes
scripts/btlib/splits.pyruns · 5,560 bytes
scripts/btlib/stdm.pyruns · 4,317 bytes
scripts/btlib/tokens.pyruns · 7,382 bytes
scripts/btlib/units.pyruns · 8,591 bytes
scripts/btlib/versions.pyruns · 9,623 bytes
scripts/build_report.pyruns · 28,993 bytes
scripts/resolve-token.shruns · 4,823 bytes
scripts/sfap_replay.pyruns · 27,991 bytes

Read excerpt


# Managing Agentforce Offline Experimentation

## When to Use This Skill

Execute this workflow when a user wants to test how a different LLM model would handle existing Agentforce agent conversations. Fetches session traces from Data Cloud STDM, replays prompts via the SFAP AI Gateway REST API with structured tool calling, scores across four metrics (topic agreement, function name accuracy, function argument accuracy via LLM-as-judge, text response quality via LLM-as-judge), and produces a run directory with units.jsonl, results.jsonl, summary.json, and an HTML report. Requires Data Cloud with Agentforce Activity data streaming enabled, an authenticated org, and a target model ID (llmgateway__ prefix for SFAP replay). Estimated runtime: 10-30 minutes for 10 sessions.

## Workflow

discover-data-space → verify-stdm-availability → discover-agents → list-available-models (if user needs help choosing) → find-sessions → get-conversation-details → get-llm-step-details → obtain-sfap-token → replay-turn-prompts (scoring happens inline) → generate-run-results → generate-html-report.
WORKFLOW GATE: Do not proceed past Phase 0 until org alias (auto-discover from authenticated orgs), agent API name (auto-discover from STDM), and target model ID (ask user) are all confirmed.
SFAP JWT expires after ~2 hours. For 20+ session replays, refresh the token between batches. Each SFAP replay call takes ~1-3 seconds; keep under 20 sessions for reasonable wall-clock time (~10-30 minutes).

## Critical Constraints