Individual skill

vllm-serving

Serve LLMs at scale with high-throughput inference servers — continuous batching, paged attention, tensor parallelism, and production operations. Use when self-hosting models for many users or high QPS.

skills/vllm-serving/SKILL.md

Read pinned source

Source commit faf37c0859ebf627bfcb5225e8057d088691e0ae

What was checked

Matches the observed repository HEAD: Yes
Scanner rules are current: No
Repository scan complete: No
Skill scan complete: Yes
Observed behavior: Not checked yet
Last checked: Nov 8, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings): 56
Repository last updated: Oct 10, 2026
License: MIT

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

agent-skills Repository topic
ai-agents Repository topic
ai-agents-automation Repository topic
ai-skills Repository topic
awesome Repository topic
awesome-list Repository topic
chatgpt Repository topic
claude-skills Repository topic
meta-ai Repository topic
muse-ai Repository topic
operations Derived category

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files

SKILL.mdcore · 4,088 bytes

Read excerpt


# High-Throughput LLM Serving

Serving LLMs efficiently is a systems problem: GPU memory is finite, requests arrive unpredictably, 
and naive serving wastes most of the hardware. Modern inference servers use continuous batching and 
paged memory management to multiply throughput — this skill covers operating them.

## Overview

The key techniques: continuous batching (requests join and leave the batch dynamically instead of 
waiting for fixed batches), paged KV-cache management (memory allocated in blocks, eliminating 
fragmentation), and tensor parallelism (splitting large models across GPUs). Together they raise 
throughput several-fold over naive serving. Your job: configure for your workload, monitor the 
right metrics, and operate reliably.