Menu

Individual skill

prefix-cache-replay

0upvotes

Replay an LLM inference request trace (Mooncake / vLLM / SGLang hash_ids format) against a block-level KV prefix cache and compute hit statistics. Use when given a request trace plus cache configuration and asked for hit rate, hit tokens, or final cache contents. Covers the longest-contiguous-prefix semantics that distinguishes KV prefix caching from full-prompt prompt caching, the policy-specific residency and eviction rules (LRU, LFU, S3FIFO), and the partial-last-block accounting rule.

tasks/llm-prefix-cache-replay/environment/skills/prefix-cache-replay/SKILL.md

Read pinned source

Source commit f860ecf02ef53412a030f8661c83d644b739ff01

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
Yes
Observed behavior
Instructions only
Last checked
Nov 9, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
1,912
Repository last updated
Nov 4, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt

# Overview

Modern LLM serving systems — vLLM, SGLang, Mooncake — pack the KV tensors of a prompt into fixed-size blocks of `block_size` tokens (typically 512). The cache is keyed by a block hash where each hash encodes both the block's own token content and the content of every block before it in the prompt. Two requests that share the first K conversation turns therefore share the first K block hashes, and the cache can reuse those blocks without recomputing attention.

This is **block-level prefix caching**. It is not the same thing as full-prompt prompt caching (Anthropic, OpenAI), where the cache stores whole prompts and looks them up by exact match. Prefix caching reuses partial prompts; prompt caching does not.

# Longest-prefix hit semantics (policy-independent)

Let a request have `hash_ids = [h_0, h_1, ..., h_{n-1}]` and `input_length = L`.

The prefix hit length is the largest integer `k` such that `h_0, h_1, ..., h_{k-1}` are **all resident in the cache** at the time the request arrives.

- `k` must start at index 0. Reuse of `h_2` when `h_0` is absent does not count.
- The scan stops at the first miss. No skip-ahead, no set intersection.
Back to top