amd/skills

Read Edit Bash Grep Glob Agent

FLYDSL_RUNTIME_ENABLE_CACHE=0 python my_kernel.py  # or: rm -rf ~/.flydsl/cache

Read,Edit,Bash,Grep,Glob,Agent,Write

RTD_VERSION=0.1.6   # <-- set to the release matching your ROCm version RTD_INSTALLER="rocprof-trace-decoder-manylinux-2.28-${RTD_VERSION}-Linux.sh" wget -q "https://github.com/ROCm/rocprof-trace-decoder/releases/download/${RTD_VERSION}/${R

RTD_INSTALLER="rocprof-trace-decoder-manylinux-2.28-${RTD_VERSION}-Linux.sh" wget -q "https://github.com/ROCm/rocprof-trace-decoder/releases/download/${RTD_VERSION}/${RTD_INSTALLER}" chmod +x "${RTD_INSTALLER}"

wget -q "https://github.com/ROCm/rocprof-trace-decoder/releases/download/${RTD_VERSION}/${RTD_INSTALLER}"

Executable file

Executable file

Executable file

AMD-AGI (federated from [AMD-AGI/Hyperloom](https://github.com/AMD-AGI/Hyperloom))

AMD-AGI (federated from [AMD-AGI/Hyperloom](https://github.com/AMD-AGI/Hyperloom))

## Step 8 - Validate and output curl commands

**8b. Immediately after validation passes, print these curl commands**

# 0. Parse and test the policy on the server itself, without registering it curl -X POST http://localhost:13305/api/v1/routing/validate \

curl -X POST http://localhost:13305/api/v1/routing/validate \

# 1. Check a model exists before registering curl http://localhost:13305/api/v1/models/<model-id>

curl http://localhost:13305/api/v1/models/<model-id>

# 2. Register the policy (idempotent - re-POST to update) curl -X POST http://localhost:13305/api/v1/pull \

curl -X POST http://localhost:13305/api/v1/pull \

# 3. Route a request and inspect the decision (-i prints response headers) curl -i -X POST http://localhost:13305/api/v1/chat/completions \

curl -i -X POST http://localhost:13305/api/v1/chat/completions \

Print **two** test curl commands (adapt Step 8b's command #3 for both), one

"Set Qwen3.5-2B-GGUF as default_model",         "Run the offline validator scripts/validate.py and report it passed",         "Output curl commands for the user to register and test the policy"

"Run the offline validator scripts/validate.py and report it passed",         "Output curl commands for the user to register and test the policy"       ],

"Output curl commands for the user to register and test the policy"

The machine-readable source of truth is `scripts/route_policy.schema.json`, a verbatim copy of the server's [`route_policy.schema.json`](https://github.com/lemonade-sdk/lemonade/blob/main/src/cpp/resources/schemas/route_policy.schema.json);

verbatim copy of the server's [`route_policy.schema.json`](https://github.com/lemonade-sdk/lemonade/blob/main/src/cpp/resources/schemas/route_policy.schema.json); `scripts/route_policy.schema.source.json` records the ref it was copied from.

[`route_policy.schema.json`](https://github.com/lemonade-sdk/lemonade/blob/main/src/cpp/resources/schemas/route_policy.schema.json);

curl -X POST http://localhost:13305/api/v1/routing/validate \

# register (idempotent - re-POST to update) curl -X POST http://localhost:13305/api/v1/pull \

curl -X POST http://localhost:13305/api/v1/pull \

# route: just name the collection as the model curl -X POST http://localhost:13305/api/v1/chat/completions \

curl -X POST http://localhost:13305/api/v1/chat/completions \

# remove curl -X POST http://localhost:13305/api/v1/delete \

curl -X POST http://localhost:13305/api/v1/delete \

{   "$schema": "https://json-schema.org/draft/2020-12/schema",

"$schema": "https://json-schema.org/draft/2020-12/schema",

"$id": "https://lemonade-sdk.github.io/schemas/route_policy.schema.json",

"repo": "lemonade-sdk/lemonade",   "ref": "main",   "url": "https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/src/cpp/resources/schemas/route_policy.schema.json",

"ref": "main",   "url": "https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/src/cpp/resources/schemas/route_policy.schema.json",   "synced_at": "2026-10-27T22:38:54Z"

"url": "https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/src/cpp/resources/schemas/route_policy.schema.json",

#!/usr/bin/env python3

def fetch(ref):

url = f"https://raw.githubusercontent.com/{REPO}/{ref}/{SCHEMA_REPO_PATH}"     with urllib.request.urlopen(url, timeout=30) as resp:         raw = resp.read().decode("utf-8")

with urllib.request.urlopen(url, timeout=30) as resp:

try:         url, raw, schema = fetch(args.ref)

url, raw, schema = fetch(args.ref)

#!/usr/bin/env python3

(OpenAI, Anthropic, or Ollama-compatible). The app launches `lemond`, the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on `http://localhost:PORT/api/v1`. The user gets local, private,

(OpenAI, Anthropic, or Ollama-compatible). The app launches `lemond`, the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on `http://localhost:PORT/api/v1`. The user gets local, private,

Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on `http://localhost:PORT/api/v1`. The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate

Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on `http://localhost:PORT/api/v1`. The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate

talks to it on `http://localhost:PORT/api/v1`. The user gets local, private,

talks to it on `http://localhost:PORT/api/v1`. The user gets local, private,

[Credential-related source omitted]

[Credential-related source omitted]

If the user instead wants a **system-wide** Lemonade Server (one install, shared across apps), do not use this skill; point them at `https://lemonade-server.ai/install_options.html` and the standard OpenAI base

If the user instead wants a **system-wide** Lemonade Server (one install, shared across apps), do not use this skill; point them at `https://lemonade-server.ai/install_options.html` and the standard OpenAI base

shared across apps), do not use this skill; point them at `https://lemonade-server.ai/install_options.html` and the standard OpenAI base URL `http://localhost:13305/api/v1`.

shared across apps), do not use this skill; point them at `https://lemonade-server.ai/install_options.html` and the standard OpenAI base URL `http://localhost:13305/api/v1`.

`https://lemonade-server.ai/install_options.html` and the standard OpenAI base

`https://lemonade-server.ai/install_options.html` and the standard OpenAI base

URL `http://localhost:13305/api/v1`.

URL `http://localhost:13305/api/v1`.

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

https://github.com/lemonade-sdk/lemonade/releases/latest

https://github.com/lemonade-sdk/lemonade/releases/latest

$rel = Invoke-RestMethod https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest

$rel = Invoke-RestMethod https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest

URL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \

URL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \

curl -L "$URL" | tar -xz --strip-components=1 -C vendor/lemonade

curl -L "$URL" | tar -xz --strip-components=1 -C vendor/lemonade

[Credential-related source omitted]

[Credential-related source omitted]

Readiness is always determined by polling the exact endpoint `GET http://127.0.0.1:<port>/api/v1/health` and checking for HTTP 200 — never

Readiness is always determined by polling the exact endpoint `GET http://127.0.0.1:<port>/api/v1/health` and checking for HTTP 200 — never

`GET http://127.0.0.1:<port>/api/v1/health` and checking for HTTP 200 — never

`GET http://127.0.0.1:<port>/api/v1/health` and checking for HTTP 200 — never

1. Set `base_url` to `http://127.0.0.1:{port}/api/v1`

1. Set `base_url` to `http://127.0.0.1:{port}/api/v1`

[Credential-related source omitted]

[Credential-related source omitted]

proc, key, port = start_lemond() client = OpenAI(     base_url=f"http://127.0.0.1:{port}/api/v1",

proc, key, port = start_lemond() client = OpenAI(     base_url=f"http://127.0.0.1:{port}/api/v1",

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

http_client=httpx.Client(timeout=120),  # required: 120s for first-run model load

http_client=httpx.Client(timeout=120),  # required: 120s for first-run model load

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

| Ollama-compatible code | `http://127.0.0.1:{port}/api/v0` | pass key anyway | 120s |

| Ollama-compatible code | `http://127.0.0.1:{port}/api/v0` | pass key anyway | 120s |

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

Executable file

Executable file

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

Executable file

Executable file

[Credential-related source omitted]

[Credential-related source omitted]

def _free_port() -> int:     with socket.socket() as s:

def _free_port() -> int:     with socket.socket() as s:

with socket.socket() as s:

with socket.socket() as s:

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

while time.monotonic() < deadline:         try:             with urllib.request.urlopen(req, timeout=1) as r:

while time.monotonic() < deadline:         try:             with urllib.request.urlopen(req, timeout=1) as r:

try:             with urllib.request.urlopen(req, timeout=1) as r:                 if r.status == 200:

try:             with urllib.request.urlopen(req, timeout=1) as r:                 if r.status == 200:

with urllib.request.urlopen(req, timeout=1) as r:

with urllib.request.urlopen(req, timeout=1) as r:

[Credential-related source omitted]

[Credential-related source omitted]

async function waitForHealth(port, key, timeoutMs) {   const url = `http://127.0.0.1:${port}/api/v1/health`;

async function waitForHealth(port, key, timeoutMs) {   const url = `http://127.0.0.1:${port}/api/v1/health`;

const url = `http://127.0.0.1:${port}/api/v1/health`;

const url = `http://127.0.0.1:${port}/api/v1/health`;

while (Date.now() < deadline) {     try {       const r = await fetch(url, { headers });

while (Date.now() < deadline) {     try {       const r = await fetch(url, { headers });

try {       const r = await fetch(url, { headers });       if (r.ok) return;

try {       const r = await fetch(url, { headers });       if (r.ok) return;

const r = await fetch(url, { headers });

const r = await fetch(url, { headers });

This is a **meta-skill**. You run it once. After that, every later request that needs image generation, text-to-speech, or speech-to-text uses the local [Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The

This is a **meta-skill**. You run it once. After that, every later request that needs image generation, text-to-speech, or speech-to-text uses the local [Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The

needs image generation, text-to-speech, or speech-to-text uses the local [Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The agent's own LLM keeps handling text; only the expensive multimodal calls move

needs image generation, text-to-speech, or speech-to-text uses the local [Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The agent's own LLM keeps handling text; only the expensive multimodal calls move

[Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The

[Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The

prints the exact OS-specific command to start it (e.g. `sudo systemctl start

prints the exact OS-specific command to start it (e.g. `sudo systemctl start

> driving an always-on `lemond` service. `lemonade` is the only valid CLI, and > this skill installs Lemonade only from the [official install > paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If

> driving an always-on `lemond` service. `lemonade` is the only valid CLI, and > this skill installs Lemonade only from the [official install > paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If

> this skill installs Lemonade only from the [official install > paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If > an older `lemonade` is already on the `PATH` it will shadow the modern CLI;

> this skill installs Lemonade only from the [official install > paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If > an older `lemonade` is already on the `PATH` it will shadow the modern CLI;

> paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If

> paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If

installs the latest version for them automatically. - The user accepts the Lemonade endpoint on this machine, whichever port the   service reports (`http://localhost:13305` if it has never been changed).

installs the latest version for them automatically. - The user accepts the Lemonade endpoint on this machine, whichever port the   service reports (`http://localhost:13305` if it has never been changed).

- The user accepts the Lemonade endpoint on this machine, whichever port the   service reports (`http://localhost:13305` if it has never been changed). - The user wants the change to be **persistent** across future turns and

- The user accepts the Lemonade endpoint on this machine, whichever port the   service reports (`http://localhost:13305` if it has never been changed). - The user wants the change to be **persistent** across future turns and

service reports (`http://localhost:13305` if it has never been changed).

service reports (`http://localhost:13305` if it has never been changed).

`sudo apt-get install`, pass `--no-install` to the setup script and confirm

`sudo apt-get install`, pass `--no-install` to the setup script and confirm

| Ubuntu/Debian | `sudo apt remove lemonade-server` |

| Ubuntu/Debian | `sudo apt remove lemonade-server` |

behalf. Use the package manager first; the download is the fallback when the package manager is absent. Full matrix, including Arch, Fedora, Debian, Snap, and Docker, is in the [install docs](https://lemonade-server.ai/docs/guide/install/).

behalf. Use the package manager first; the download is the fallback when the package manager is absent. Full matrix, including Arch, Fedora, Debian, Snap, and Docker, is in the [install docs](https://lemonade-server.ai/docs/guide/install/).

and Docker, is in the [install docs](https://lemonade-server.ai/docs/guide/install/).

and Docker, is in the [install docs](https://lemonade-server.ai/docs/guide/install/).

| OS | Install | Fallback | |---|---|---| | Windows | `winget install -e --id AMD.LemonadeServer` | Download [`lemonade.msi`](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) and run `msiexec /i lemonade.msi /

| OS | Install | Fallback | |---|---|---| | Windows | `winget install -e --id AMD.LemonadeServer` | Download [`lemonade.msi`](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) and run `msiexec /i lemonade.msi /

|---|---|---| | Windows | `winget install -e --id AMD.LemonadeServer` | Download [`lemonade.msi`](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) and run `msiexec /i lemonade.msi /qn` (silent, per-user, no el

|---|---|---| | Windows | `winget install -e --id AMD.LemonadeServer` | Download [`lemonade.msi`](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) and run `msiexec /i lemonade.msi /qn` (silent, per-user, no el

| Windows | `winget install -e --id AMD.LemonadeServer` | Download [`lemonade.msi`](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) and run `msiexec /i lemonade.msi /qn` (silent, per-user, no elevation). |

| Windows | `winget install -e --id AMD.LemonadeServer` | Download [`lemonade.msi`](https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi) and run `msiexec /i lemonade.msi /qn` (silent, per-user, no elevation). |

| Ubuntu | `sudo add-apt-repository -y ppa:lemonade-team/stable && sudo apt-get update && sudo apt-get install -y lemonade-server` | `sudo snap install lemonade-server` |

| Ubuntu | `sudo add-apt-repository -y ppa:lemonade-team/stable && sudo apt-get update && sudo apt-get install -y lemonade-server` | `sudo snap install lemonade-server` |

| macOS | `brew install --cask lemonade-server` | Download `Lemonade-<ver>-Darwin.pkg` from the [latest release](https://github.com/lemonade-sdk/lemonade/releases/latest) and run `sudo installer -pkg Lemonade-<ver>-Darwin.pkg -target /`. |

| macOS | `brew install --cask lemonade-server` | Download `Lemonade-<ver>-Darwin.pkg` from the [latest release](https://github.com/lemonade-sdk/lemonade/releases/latest) and run `sudo installer -pkg Lemonade-<ver>-Darwin.pkg -target /`. |

| macOS | `brew install --cask lemonade-server` | Download `Lemonade-<ver>-Darwin.pkg` from the [latest release](https://github.com/lemonade-sdk/lemonade/releases/latest) and run `sudo installer -pkg Lemonade-<ver>-Darwin.pkg -target /`. |

| macOS | `brew install --cask lemonade-server` | Download `Lemonade-<ver>-Darwin.pkg` from the [latest release](https://github.com/lemonade-sdk/lemonade/releases/latest) and run `sudo installer -pkg Lemonade-<ver>-Darwin.pkg -target /`. |

The Ubuntu apt package is named `lemonade-server`, but the CLI it installs is `lemonade`. The browser UI is served at `http://localhost:13305` with no extra

The Ubuntu apt package is named `lemonade-server`, but the CLI it installs is `lemonade`. The browser UI is served at `http://localhost:13305` with no extra

`lemonade`. The browser UI is served at `http://localhost:13305` with no extra

`lemonade`. The browser UI is served at `http://localhost:13305` with no extra

package; add `sudo apt install lemonade-desktop` only if the user wants the

package; add `sudo apt install lemonade-desktop` only if the user wants the

| `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`, re-asking `status` in case the service comes up on a different port). If it stays down, start it via the OS service manager: `su

| `Server is not running` | Wait a few seconds for the auto-started service (the script polls `/api/v1/health`, re-asking `status` in case the service comes up on a different port). If it stays down, start it via the OS service manager: `su

Only if the automatic install genuinely fails (no `apt-get`, no `sudo`, download blocked) should you stop and point the user at <https://lemonade-server.ai/docs/guide/install/>.

Only if the automatic install genuinely fails (no `apt-get`, no `sudo`, download blocked) should you stop and point the user at <https://lemonade-server.ai/docs/guide/install/>.

<https://lemonade-server.ai/docs/guide/install/>.

<https://lemonade-server.ai/docs/guide/install/>.

The rest of this skill writes the endpoint as `http://localhost:13305/api/v1`,

The rest of this skill writes the endpoint as `http://localhost:13305/api/v1`,

[Credential-related source omitted]

[Credential-related source omitted]

| `lemonade: command not found` | CLI not installed | Re-run `python scripts/setup_local_ai.py` (auto-installs the latest version). If it just installed on Windows, open a new shell so the user PATH refreshes, or the script will find it und

| `lemonade: command not found` | CLI not installed | Re-run `python scripts/setup_local_ai.py` (auto-installs the latest version). If it just installed on Windows, open a new shell so the user PATH refreshes, or the script will find it und

| `status` gives an "invalid choice" / usage error | An old `lemonade` (pre-v10.1.0) is shadowing the modern CLI | Uninstall it (see the Step 1a table: `winget uninstall -e --id AMD.LemonadeServer` / `sudo apt remove lemonade-server` / `bre

| `status` gives an "invalid choice" / usage error | An old `lemonade` (pre-v10.1.0) is shadowing the modern CLI | Uninstall it (see the Step 1a table: `winget uninstall -e --id AMD.LemonadeServer` / `sudo apt remove lemonade-server` / `bre

| `status` gives an "invalid choice" / usage error | An old `lemonade` (pre-v10.1.0) is shadowing the modern CLI | Uninstall it (see the Step 1a table: `winget uninstall -e --id AMD.LemonadeServer` / `sudo apt remove lemonade-server` / `bre

| `status` gives an "invalid choice" / usage error | An old `lemonade` (pre-v10.1.0) is shadowing the modern CLI | Uninstall it (see the Step 1a table: `winget uninstall -e --id AMD.LemonadeServer` / `sudo apt remove lemonade-server` / `bre

| Requests to `http://localhost:13305` are refused, but `lemonade status` says the server is running | The service is bound to a non-default port (existing config, another install channel, or a port conflict) | Use the port from `lemonade s

| Requests to `http://localhost:13305` are refused, but `lemonade status` says the server is running | The service is bound to a non-default port (existing config, another install channel, or a port conflict) | Use the port from `lemonade s

| `Server is not running` | `lemond` service stopped | Start it via the OS service manager — `sudo systemctl start lemond` / `systemctl --user start lemond` (Linux), `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS),

| `Server is not running` | `lemond` service stopped | Start it via the OS service manager — `sudo systemctl start lemond` / `systemctl --user start lemond` (Linux), `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS),

| `Server is not running` | `lemond` service stopped | Start it via the OS service manager — `sudo systemctl start lemond` / `systemctl --user start lemond` (Linux), `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS),

| `Server is not running` | `lemond` service stopped | Start it via the OS service manager — `sudo systemctl start lemond` / `systemctl --user start lemond` (Linux), `launchctl load /Library/LaunchDaemons/com.lemonade.server.plist` (macOS),

| `POST /v1/images/generations` returns 404 model not found | Image model not downloaded | `lemonade pull SD-Turbo` and retry. | | `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of spa

| `POST /v1/images/generations` returns 404 model not found | Image model not downloaded | `lemonade pull SD-Turbo` and retry. | | `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of spa

| `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of space, no write permission, quota, read-only mount). The write error may surface only in the server log while the console keeps show

| `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of space, no write permission, quota, read-only mount). The write error may surface only in the server log while the console keeps show

[Credential-related source omitted]

[Credential-related source omitted]

For full live coverage, run `lemonade list` after starting the server, or browse <https://lemonade-server.ai/models.html>.

For full live coverage, run `lemonade list` after starting the server, or browse <https://lemonade-server.ai/models.html>.

browse <https://lemonade-server.ai/models.html>.

browse <https://lemonade-server.ai/models.html>.

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

#!/usr/bin/env -S uv run --quiet

#!/usr/bin/env -S uv run --quiet

`lemond` daemon) is reachable. The port is the one `lemonade status`      reports, unless --host / --port or LEMONADE_HOST / LEMONADE_PORT say      otherwise; http://localhost:13305 is only the fallback. If no modern

`lemond` daemon) is reachable. The port is the one `lemonade status`      reports, unless --host / --port or LEMONADE_HOST / LEMONADE_PORT say      otherwise; http://localhost:13305 is only the fallback. If no modern

reports, unless --host / --port or LEMONADE_HOST / LEMONADE_PORT say      otherwise; http://localhost:13305 is only the fallback. If no modern      `lemonade` CLI is found, the latest version is installed on the user's

reports, unless --host / --port or LEMONADE_HOST / LEMONADE_PORT say      otherwise; http://localhost:13305 is only the fallback. If no modern      `lemonade` CLI is found, the latest version is installed on the user's

otherwise; http://localhost:13305 is only the fallback. If no modern

otherwise; http://localhost:13305 is only the fallback. If no modern

WINDOWS_MSI_URL = (     "https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi"

WINDOWS_MSI_URL = (     "https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi"

"https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi"

"https://github.com/lemonade-sdk/lemonade/releases/latest/download/lemonade.msi"

[Credential-related source omitted]

[Credential-related source omitted]

GITHUB_LATEST_RELEASE_API = (     "https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest"

GITHUB_LATEST_RELEASE_API = (     "https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest"

"https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest"

"https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest"

# pipeline so one sudo prompt covers the whole thing.

# pipeline so one sudo prompt covers the whole thing.

"sudo add-apt-repository -y ppa:lemonade-team/stable && "

"sudo add-apt-repository -y ppa:lemonade-team/stable && "

"sudo apt-get update && "

"sudo apt-get update && "

"sudo apt-get install -y lemonade-server"

"sudo apt-get install -y lemonade-server"

INSTALL_DOCS_URL = "https://lemonade-server.ai/docs/guide/install/"

INSTALL_DOCS_URL = "https://lemonade-server.ai/docs/guide/install/"

def _http_get(url: str, timeout_s: float) -> tuple[int, bytes]:     req = urllib.request.Request(url)

def _http_get(url: str, timeout_s: float) -> tuple[int, bytes]:     req = urllib.request.Request(url)

req = urllib.request.Request(url)

req = urllib.request.Request(url)

with urllib.request.urlopen(req, timeout=timeout_s) as r:  # noqa: S310

with urllib.request.urlopen(req, timeout=timeout_s) as r:  # noqa: S310

_print(f"downloading {url}")     try:         urllib.request.urlretrieve(url, dest)  # noqa: S310

_print(f"downloading {url}")     try:         urllib.request.urlretrieve(url, dest)  # noqa: S310

try:         urllib.request.urlretrieve(url, dest)  # noqa: S310     except (urllib.error.URLError, OSError) as exc:

try:         urllib.request.urlretrieve(url, dest)  # noqa: S310     except (urllib.error.URLError, OSError) as exc:

urllib.request.urlretrieve(url, dest)  # noqa: S310

urllib.request.urlretrieve(url, dest)  # noqa: S310

req = urllib.request.Request(

req = urllib.request.Request(

)     try:         with urllib.request.urlopen(req, timeout=15.0) as r:  # noqa: S310

)     try:         with urllib.request.urlopen(req, timeout=15.0) as r:  # noqa: S310

try:         with urllib.request.urlopen(req, timeout=15.0) as r:  # noqa: S310             data = json.loads(r.read())

try:         with urllib.request.urlopen(req, timeout=15.0) as r:  # noqa: S310             data = json.loads(r.read())

with urllib.request.urlopen(req, timeout=15.0) as r:  # noqa: S310

with urllib.request.urlopen(req, timeout=15.0) as r:  # noqa: S310

"sudo systemctl start lemond   "

"sudo systemctl start lemond   "

return "sudo launchctl load /Library/LaunchDaemons/com.lemonade.server.plist"

return "sudo launchctl load /Library/LaunchDaemons/com.lemonade.server.plist"

return "`sudo apt remove lemonade-server`, or the package manager it came from"

return "`sudo apt remove lemonade-server`, or the package manager it came from"

url = f"http://{host}:{port}/api/v1/health"

url = f"http://{host}:{port}/api/v1/health"

)     endpoint_host = "localhost" if host in {"127.0.0.1", "::1"} else host     base_root = f"http://{endpoint_host}:{port}"

)     endpoint_host = "localhost" if host in {"127.0.0.1", "::1"} else host     base_root = f"http://{endpoint_host}:{port}"

endpoint_host = "localhost" if host in {"127.0.0.1", "::1"} else host     base_root = f"http://{endpoint_host}:{port}"     replacements = {

endpoint_host = "localhost" if host in {"127.0.0.1", "::1"} else host     base_root = f"http://{endpoint_host}:{port}"     replacements = {

base_root = f"http://{endpoint_host}:{port}"

base_root = f"http://{endpoint_host}:{port}"

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

_print(                 f"FAIL: Lemonade Server is not responding at "                 f"http://{host}:{port}/api/v1/health (--no-install set)."

_print(                 f"FAIL: Lemonade Server is not responding at "                 f"http://{host}:{port}/api/v1/health (--no-install set)."

f"FAIL: Lemonade Server is not responding at "                 f"http://{host}:{port}/api/v1/health (--no-install set)."             )

f"FAIL: Lemonade Server is not responding at "                 f"http://{host}:{port}/api/v1/health (--no-install set)."             )

f"http://{host}:{port}/api/v1/health (--no-install set)."

f"http://{host}:{port}/api/v1/health (--no-install set)."

_print(                 f"FAIL: the Lemonade service did not become reachable at "                 f"http://{host}:{port}/api/v1/health."

_print(                 f"FAIL: the Lemonade service did not become reachable at "                 f"http://{host}:{port}/api/v1/health."

f"FAIL: the Lemonade service did not become reachable at "                 f"http://{host}:{port}/api/v1/health."             )

f"FAIL: the Lemonade service did not become reachable at "                 f"http://{host}:{port}/api/v1/health."             )

f"http://{host}:{port}/api/v1/health."

f"http://{host}:{port}/api/v1/health."

_print(f"server reachable at http://{host}:{port}")

_print(f"server reachable at http://{host}:{port}")

This workspace runs a local [Lemonade Server](https://lemonade-server.ai) at

This workspace runs a local [Lemonade Server](https://lemonade-server.ai) at

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

curl -sX POST {{LEMONADE_BASE_URL}}/images/generations    -H "Content-Type: application/json"    -d '{"model":"{{IMAGE_MODEL}}","prompt":"PROMPT_HERE","size":"512x512","steps":4,"response_format":"b64_json"}'    | python -c "import sys,json

curl -sX POST {{LEMONADE_BASE_URL}}/images/generations    -H "Content-Type: application/json"    -d '{"model":"{{IMAGE_MODEL}}","prompt":"PROMPT_HERE","size":"512x512","steps":4,"response_format":"b64_json"}'    | python -c "import sys,json

curl -sX POST {{LEMONADE_BASE_URL}}/images/generations \

curl -sX POST {{LEMONADE_BASE_URL}}/images/generations \

curl -sX POST {{LEMONADE_BASE_URL}}/images/generations    -H "Content-Type: application/json"    -d '{"model":"{{IMAGE_MODEL}}","prompt":"PROMPT_HERE","size":"512x512","steps":4,"response_format":"b64_json"}'    | python -c "import sys,json

curl -sX POST {{LEMONADE_BASE_URL}}/images/generations    -H "Content-Type: application/json"    -d '{"model":"{{IMAGE_MODEL}}","prompt":"PROMPT_HERE","size":"512x512","steps":4,"response_format":"b64_json"}'    | python -c "import sys,json

| python -c "import sys,json,base64; open('out.png','wb').write(base64.b64decode(json.load(sys.stdin)['data'][0]['b64_json']))"

| python -c "import sys,json,base64; open('out.png','wb').write(base64.b64decode(json.load(sys.stdin)['data'][0]['b64_json']))"

[Credential-related source omitted]

[Credential-related source omitted]

open("out.png", "wb").write(base64.b64decode(r.data[0].b64_json))

open("out.png", "wb").write(base64.b64decode(r.data[0].b64_json))

curl -sX POST {{LEMONADE_BASE_URL}}/audio/speech \

curl -sX POST {{LEMONADE_BASE_URL}}/audio/speech \

ffmpeg -y -i INPUT_AUDIO -ar 16000 -ac 1 _stt.wav curl -sX POST {{LEMONADE_BASE_URL}}/audio/transcriptions \

ffmpeg -y -i INPUT_AUDIO -ar 16000 -ac 1 _stt.wav curl -sX POST {{LEMONADE_BASE_URL}}/audio/transcriptions \

curl -sX POST {{LEMONADE_BASE_URL}}/audio/transcriptions \

curl -sX POST {{LEMONADE_BASE_URL}}/audio/transcriptions \

of the path, check the Lemonade docs) and read its most       recent lines for the underlying error — for example a download/write error       (such as `CURL code 23`) or an out-of-space message. Surface that line

of the path, check the Lemonade docs) and read its most       recent lines for the underlying error — for example a download/write error       (such as `CURL code 23`) or an out-of-space message. Surface that line

recent lines for the underlying error — for example a download/write error       (such as `CURL code 23`) or an out-of-space message. Surface that line       to the user rather than waiting through silent retries.

recent lines for the underlying error — for example a download/write error       (such as `CURL code 23`) or an out-of-space message. Surface that line       to the user rather than waiting through silent retries.

(such as `CURL code 23`) or an out-of-space message. Surface that line

(such as `CURL code 23`) or an out-of-space message. Surface that line

python -m pip install git+https://github.com/AMD-AGI/Magpie.git

For compare, replace `kernel:` with `kernels:` and provide a list of at least two entries. Configuration may also override performance, correctness, Ray, and scheduler settings. Start from the repository's [kernel config example](https://gi

Apply correctness as a hard gate before performance scoring. Keep tolerances explicit and appropriate to the datatype. Confirm optional profiler and correctness dependencies in the repository's [compatibility matrix](https://github.com/AMD-

AMD-AGI (federated from [AMD-AGI/Magpie](https://github.com/AMD-AGI/Magpie))

"<PYTHON>" -m pip index versions torch --index-url "https://download.pytorch.org/whl/<GPU_INDEX>"

"<PYTHON>" -m pip install "torch==<VERSION>" --index-url "https://download.pytorch.org/whl/<GPU_INDEX>"

"<PYTHON>" -m pip install "torch==<VERSION>" --index-url "https://download.pytorch.org/whl/cpu"

"<PYTHON>" -m pip install "amd-quark===<FULL_VERSION_WITH_VARIANT>" --index-url "<AMD_INDEX_URL>" --extra-index-url "https://pypi.org/simple"

- Always use the same absolute `"<PYTHON>" -m pip`; never use unqualified `pip` or `sudo pip`.

[Credential-related source omitted]

{   "$schema": "https://json-schema.org/draft/2020-12/schema",

"$schema": "https://json-schema.org/draft/2020-12/schema",

{   "$schema": "https://json-schema.org/draft/2020-12/schema",

"$schema": "https://json-schema.org/draft/2020-12/schema",

{   "$schema": "https://json-schema.org/draft/2020-12/schema",

"$schema": "https://json-schema.org/draft/2020-12/schema",

{   "$schema": "https://json-schema.org/draft/2020-12/schema",

"$schema": "https://json-schema.org/draft/2020-12/schema",

The accelerator decides the wheel index, not the torch version alone. Upstream publishes each combination under `https://download.pytorch.org/whl/<slug>`,

publishes each combination under `https://download.pytorch.org/whl/<slug>`,

- **Linux (x86_64 only):**      ```      curl -fsSL https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.sh | sh

```      curl -fsSL https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.sh | sh      ```

curl -fsSL https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.sh | sh

curl -fsSL https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.sh | sh

- **Windows (PowerShell):**      ```      irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex

```      irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex      ```

irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex

After install, confirm `~/.local/bin` is on `PATH` and re-run `rocm --version`.    If it still isn't available, hand the user the install page    (https://github.com/ROCm/rocm-cli) and stop.

(https://github.com/ROCm/rocm-cli) and stop.

installer cannot run, or the check still fails afterwards (an older `rocm`    earlier on `PATH` can cause that), hand the user the install page    (https://github.com/ROCm/rocm-cli) and stop.

(https://github.com/ROCm/rocm-cli) and stop.

- **Lemonade** → https://github.com/lemonade-sdk/lemonade/issues

- **Ollama** → https://github.com/ollama/ollama/issues

- **LM Studio** → in-app support (no public repo) - Anything else with no catalog match → ROCm core:   https://github.com/ROCm/ROCm/issues (this is what `route_when_no_match`

- Anything else with no catalog match → ROCm core:   https://github.com/ROCm/ROCm/issues (this is what `route_when_no_match`   returns by default).

https://github.com/ROCm/ROCm/issues (this is what `route_when_no_match`

"Execute a mutating or sudo command without first getting the user's explicit consent"

"Execute a mutating or sudo command (usermod, modprobe, and the like) without first getting the user's explicit consent"

"Execute a mutating or sudo command without first getting the user's explicit consent"

- **Lemonade** → https://github.com/lemonade-sdk/lemonade/issues

- **Ollama** → https://github.com/ollama/ollama/issues

- **pytorch** → https://github.com/pytorch/pytorch/issues (tag with the rocm label)

- **llama-cpp** → https://github.com/ggml-org/llama.cpp/issues

- otherwise **rocm-core** → https://github.com/ROCm/ROCm/issues

inheriting the bare-metal Linux ones. AMD's ROCm-on-WSL guide is still the right pointer for install questions the catalog does not cover: https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/install/installryz/wsl/howto_wsl.html

https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/install/installryz/wsl/howto_wsl.html

The rocm-systems repository has several PC sampling documentation files in this folder: projects/rocprofiler-sdk/source/docs/how-to/. You may also access them here: https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/develop/how-to/cdna3-

This reference is based on the official documentation, available [here](https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/develop/how-to/cdna3-cdna4-pc-sampling.html).

#!/usr/bin/env python3

rocprofiler-sdk (a project in [rocm-systems](https://github.com/ROCm/rocm-systems/))

#!/usr/bin/env python3

Bash, Read

eval "$(python3 scripts/cpu_tune.py)"      # or --format json to inspect

$RT rm -f vllm-epyc 2>/dev/null               # clear any leftover container from a prior run (name collision otherwise)

[Credential-related source omitted]

[Credential-related source omitted]

$RT rm -f vllm-epyc 2>/dev/null

[Credential-related source omitted]

[Credential-related source omitted]

for i in $(seq 1 120); do   $RT inspect -f '{{.State.Running}}' vllm-epyc 2>/dev/null | grep -q true || { echo "FAILED: container exited"; $RT logs --tail 50 vllm-epyc; break; }   curl -sf http://localhost:<port>/health >/dev/null 2>&1 && {

$RT inspect -f '{{.State.Running}}' vllm-epyc 2>/dev/null | grep -q true || { echo "FAILED: container exited"; $RT logs --tail 50 vllm-epyc; break; }   curl -sf http://localhost:<port>/health >/dev/null 2>&1 && { healthy=1; echo "HEALTHY"; 

curl -sf http://localhost:<port>/health >/dev/null 2>&1 && { healthy=1; echo "HEALTHY"; break; }

# 2. the served model is registered curl -sf --max-time 30 http://localhost:<port>/v1/models

curl -sf --max-time 30 http://localhost:<port>/v1/models

# primary_endpoint == chat_completions curl -sf --max-time 180 http://localhost:<port>/v1/chat/completions -H 'Content-Type: application/json' \

curl -sf --max-time 180 http://localhost:<port>/v1/chat/completions -H 'Content-Type: application/json' \

# primary_endpoint == completions  (no chat template) curl -sf --max-time 180 http://localhost:<port>/v1/completions -H 'Content-Type: application/json' \

curl -sf --max-time 180 http://localhost:<port>/v1/completions -H 'Content-Type: application/json' \

| Field | Value | |---|---| | Base URL | `http://localhost:<port>/v1` (the trailing `/v1` matters) |

|---|---| | Base URL | `http://localhost:<port>/v1` (the trailing `/v1` matters) | | Served model | `<served-model>` (the id from `/v1/models`) |

| Base URL | `http://localhost:<port>/v1` (the trailing `/v1` matters) |

| Stop | `$RT rm -f vllm-epyc` (container) or `kill <pid>` (conda) |

curl -s http://localhost:<port>/v1/chat/completions -H 'Content-Type: application/json' \

curl -s http://localhost:<port>/v1/completions -H 'Content-Type: application/json' \

[Credential-related source omitted]

[Credential-related source omitted]

with "name already in use" -- Step 6 clears it first with `$RT rm -f vllm-epyc`.

3. **sudo docker** -- only if `sudo -n docker ps` works (passwordless sudo). The

agent can still drive it unattended; `runtime` comes back as `"sudo docker"`.

`sudo usermod -aG docker $USER && newgrp docker` (or a NOPASSWD sudoers entry).

[Credential-related source omitted]

#!/usr/bin/env python3

[Credential-related source omitted]

HF = "https://huggingface.co"

GH_RAW = "https://raw.githubusercontent.com/vllm-project/vllm"

headers["Authorization"] = f"Bearer {token}"     try:         with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as r:

try:         with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as r:             return r.read().decode("utf-8"), None

with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as r:

[Credential-related source omitted]

#!/usr/bin/env python3

#!/usr/bin/env python3

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

#!/usr/bin/env python3

[Credential-related source omitted]

HF = "https://huggingface.co"

headers["Authorization"] = f"Bearer {token}"     try:         with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as r:

try:         with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as r:             return json.load(r), None

with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as r:

[Credential-related source omitted]

#!/usr/bin/env python3

"(sudo usermod -aG docker $USER, then re-login) or start the daemon; "

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

Bash, Read

Bash, Read

- **`data/recipes_cache.json`** -- model configs synced from   [vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry

- **`data/recipes_cache.json`** -- model configs synced from   [vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry

[vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry

[vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

If a Docker container is on that port, stop it with `docker rm -f <name>`.

If a Docker container is on that port, stop it with `docker rm -f <name>`.

while docker inspect --format='{{.State.Running}}' <container_name> 2>/dev/null | grep -q true; do   curl -sf http://localhost:<port>/health && echo "READY" && exit 0

while docker inspect --format='{{.State.Running}}' <container_name> 2>/dev/null | grep -q true; do   curl -sf http://localhost:<port>/health && echo "READY" && exit 0

curl -sf http://localhost:<port>/health && echo "READY" && exit 0

curl -sf http://localhost:<port>/health && echo "READY" && exit 0

curl -s http://localhost:<port>/v1/chat/completions \

curl -s http://localhost:<port>/v1/chat/completions \

curl -s http://<host>:<port>/v1/chat/completions \

curl -s http://<host>:<port>/v1/chat/completions \

Use `localhost` for health/warmup curl URLs (curl runs on the remote host).

Use `localhost` for health/warmup curl URLs (curl runs on the remote host).

group. Fix: `sudo usermod -aG video,render $USER` (requires re-login).

group. Fix: `sudo usermod -aG video,render $USER` (requires re-login).

{             "note": "Pinned transformers commit required for Seed-OSS tokenizer compatibility",             "command": "uv pip install git+https://github.com/huggingface/transformers.git@56d68c6706ee052b445e1e476056ed92ac5eb383"

{             "note": "Pinned transformers commit required for Seed-OSS tokenizer compatibility",             "command": "uv pip install git+https://github.com/huggingface/transformers.git@56d68c6706ee052b445e1e476056ed92ac5eb383"

"note": "Pinned transformers commit required for Seed-OSS tokenizer compatibility",             "command": "uv pip install git+https://github.com/huggingface/transformers.git@56d68c6706ee052b445e1e476056ed92ac5eb383"           }

"note": "Pinned transformers commit required for Seed-OSS tokenizer compatibility",             "command": "uv pip install git+https://github.com/huggingface/transformers.git@56d68c6706ee052b445e1e476056ed92ac5eb383"           }

"command": "uv pip install git+https://github.com/huggingface/transformers.git@56d68c6706ee052b445e1e476056ed92ac5eb383"

"command": "uv pip install git+https://github.com/huggingface/transformers.git@56d68c6706ee052b445e1e476056ed92ac5eb383"

},         "strategy_overrides": {},         "guide": "## Overview\n\nSeed-OSS-36B is a dense language model from ByteDance Seed with a unique **thinking\nbudget** feature for controlled reasoning, and up to 512K context. Users can choose\n

},         "strategy_overrides": {},         "guide": "## Overview\n\nSeed-OSS-36B is a dense language model from ByteDance Seed with a unique **thinking\nbudget** feature for controlled reasoning, and up to 512K context. Users can choose\n

"strategy_overrides": {},         "guide": "## Overview\n\nSeed-OSS-36B is a dense language model from ByteDance Seed with a unique **thinking\nbudget** feature for controlled reasoning, and up to 512K context. Users can choose\ntensor-para

"strategy_overrides": {},         "guide": "## Overview\n\nSeed-OSS-36B is a dense language model from ByteDance Seed with a unique **thinking\nbudget** feature for controlled reasoning, and up to 512K context. Users can choose\ntensor-para

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"id": "modal",               "blurb": "Serverless deploy via the Gemma 4 Modal script bundled with this recipe.",               "script": "https://github.com/vllm-project/recipes/blob/main/Google/gemma4-modal.py",

"id": "modal",               "blurb": "Serverless deploy via the Gemma 4 Modal script bundled with this recipe.",               "script": "https://github.com/vllm-project/recipes/blob/main/Google/gemma4-modal.py",

"blurb": "Serverless deploy via the Gemma 4 Modal script bundled with this recipe.",               "script": "https://github.com/vllm-project/recipes/blob/main/Google/gemma4-modal.py",               "install": "curl -O https://raw.githubuse

"blurb": "Serverless deploy via the Gemma 4 Modal script bundled with this recipe.",               "script": "https://github.com/vllm-project/recipes/blob/main/Google/gemma4-modal.py",               "install": "curl -O https://raw.githubuse

"script": "https://github.com/vllm-project/recipes/blob/main/Google/gemma4-modal.py",               "install": "curl -O https://raw.githubusercontent.com/vllm-project/recipes/main/Google/gemma4-modal.py\npip install modal\nmodal setup\nmoda

"script": "https://github.com/vllm-project/recipes/blob/main/Google/gemma4-modal.py",               "install": "curl -O https://raw.githubusercontent.com/vllm-project/recipes/main/Google/gemma4-modal.py\npip install modal\nmodal setup\nmoda

"install": "curl -O https://raw.githubusercontent.com/vllm-project/recipes/main/Google/gemma4-modal.py\npip install modal\nmodal setup\nmodal deploy gemma4-modal.py\nmodal run gemma4-modal.py\n"

"install": "curl -O https://raw.githubusercontent.com/vllm-project/recipes/main/Google/gemma4-modal.py\npip install modal\nmodal setup\nmodal deploy gemma4-modal.py\nmodal run gemma4-modal.py\n"

}         },         "guide": "## Overview\n\n[Gemma 4 26B-A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) is the Mixture-of-Experts member of Google's Gemma 4 family \u2014 26B total parameters with only 4B active per token via 128 

}         },         "guide": "## Overview\n\n[Gemma 4 26B-A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) is the Mixture-of-Experts member of Google's Gemma 4 family \u2014 26B total parameters with only 4B active per token via 128 

},         "guide": "## Overview\n\n[Gemma 4 26B-A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) is the Mixture-of-Experts member of Google's Gemma 4 family \u2014 26B total parameters with only 4B active per token via 128 fine-grain

},         "guide": "## Overview\n\n[Gemma 4 26B-A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) is the Mixture-of-Experts member of Google's Gemma 4 family \u2014 26B total parameters with only 4B active per token via 128 fine-grain

[Credential-related source omitted]

[Credential-related source omitted]

}         },         "guide": "## Overview\n\n[Gemma 4](https://ai.google.dev/gemma/docs) is Google's most capable open model family, featuring a unified multimodal architecture that natively processes text, images, and audio. Gemma 4 model

}         },         "guide": "## Overview\n\n[Gemma 4](https://ai.google.dev/gemma/docs) is Google's most capable open model family, featuring a unified multimodal architecture that natively processes text, images, and audio. Gemma 4 model

},         "guide": "## Overview\n\n[Gemma 4](https://ai.google.dev/gemma/docs) is Google's most capable open model family, featuring a unified multimodal architecture that natively processes text, images, and audio. Gemma 4 models support 

},         "guide": "## Overview\n\n[Gemma 4](https://ai.google.dev/gemma/docs) is Google's most capable open model family, featuring a unified multimodal architecture that natively processes text, images, and audio. Gemma 4 models support 

[Credential-related source omitted]

[Credential-related source omitted]

}         },         "guide": "## Overview\n\n[Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it) is the smallest member of Google's Gemma 4 family \u2014 an effective-2B unified multimodal model that natively processes text, images

}         },         "guide": "## Overview\n\n[Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it) is the smallest member of Google's Gemma 4 family \u2014 an effective-2B unified multimodal model that natively processes text, images

},         "guide": "## Overview\n\n[Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it) is the smallest member of Google's Gemma 4 family \u2014 an effective-2B unified multimodal model that natively processes text, images, and audi

},         "guide": "## Overview\n\n[Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it) is the smallest member of Google's Gemma 4 family \u2014 an effective-2B unified multimodal model that natively processes text, images, and audi

[Credential-related source omitted]

[Credential-related source omitted]

}         },         "guide": "## Overview\n\n[Gemma 4 E4B](https://huggingface.co/google/gemma-4-E4B-it) is Google's effective-4B unified multimodal model \u2014 text + images + audio in a single model, with structured thinking/reasoning, 

}         },         "guide": "## Overview\n\n[Gemma 4 E4B](https://huggingface.co/google/gemma-4-E4B-it) is Google's effective-4B unified multimodal model \u2014 text + images + audio in a single model, with structured thinking/reasoning, 

},         "guide": "## Overview\n\n[Gemma 4 E4B](https://huggingface.co/google/gemma-4-E4B-it) is Google's effective-4B unified multimodal model \u2014 text + images + audio in a single model, with structured thinking/reasoning, function c

},         "guide": "## Overview\n\n[Gemma 4 E4B](https://huggingface.co/google/gemma-4-E4B-it) is Google's effective-4B unified multimodal model \u2014 text + images + audio in a single model, with structured thinking/reasoning, function c

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models fro

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models fro

"strategy_overrides": {},         "guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models from Google, based on the Gemma 3 fam

"strategy_overrides": {},         "guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models from Google, based on the Gemma 3 fam

"guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models from Google, based on the Gemma 3 family. TranslateGemma models handle 

"guide": "## Overview\n\n[TranslateGemma](https://huggingface.co/collections/google/translategemma) is a family of lightweight, state-of-the-art open translation models from Google, based on the Gemma 3 family. TranslateGemma models handle 

}         },         "guide": "## Overview\n\n[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) with imp

}         },         "guide": "## Overview\n\n[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) with imp

},         "guide": "## Overview\n\n[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) with improvements 

},         "guide": "## Overview\n\n[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) with improvements 

"guide": "## Overview\n\n[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) with improvements over the or

"guide": "## Overview\n\n[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) with improvements over the or

}         },         "guide": "## Overview\n\n[MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) and a 19

}         },         "guide": "## Overview\n\n[MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) and a 19

},         "guide": "## Overview\n\n[MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) and a 196K context

},         "guide": "## Overview\n\n[MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) and a 196K context

"guide": "## Overview\n\n[MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) and a 196K context per sequen

"guide": "## Overview\n\n[MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) is part of the MiniMax\nM2 series of advanced MoE language models. It retains the M2 architecture (10B active,\n230B total) and a 196K context per sequen

}         },         "guide": "## Overview\n\n[MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) is the latest release\nin the MiniMax M2 series. Like earlier M2 variants, it ships with 10B active\nparameters out of 230B total, a

}         },         "guide": "## Overview\n\n[MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) is the latest release\nin the MiniMax M2 series. Like earlier M2 variants, it ships with 10B active\nparameters out of 230B total, a

},         "guide": "## Overview\n\n[MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) is the latest release\nin the MiniMax M2 series. Like earlier M2 variants, it ships with 10B active\nparameters out of 230B total, and support

},         "guide": "## Overview\n\n[MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) is the latest release\nin the MiniMax M2 series. Like earlier M2 variants, it ships with 10B active\nparameters out of 230B total, and support

"guide": "## Overview\n\n[MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) is the latest release\nin the MiniMax M2 series. Like earlier M2 variants, it ships with 10B active\nparameters out of 230B total, and supports a 196K co

"guide": "## Overview\n\n[MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) is the latest release\nin the MiniMax M2 series. Like earlier M2 variants, it ships with 10B active\nparameters out of 230B total, and supports a 196K co

}         },         "guide": "## Overview\n\n[MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is an advanced MoE language\nmodel from [MiniMax](https://www.minimax.io/). Highlights:\n\n- Superior intelligence \u2014 #1 among open-

}         },         "guide": "## Overview\n\n[MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is an advanced MoE language\nmodel from [MiniMax](https://www.minimax.io/). Highlights:\n\n- Superior intelligence \u2014 #1 among open-

},         "guide": "## Overview\n\n[MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is an advanced MoE language\nmodel from [MiniMax](https://www.minimax.io/). Highlights:\n\n- Superior intelligence \u2014 #1 among open-source mod

},         "guide": "## Overview\n\n[MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is an advanced MoE language\nmodel from [MiniMax](https://www.minimax.io/). Highlights:\n\n- Superior intelligence \u2014 #1 among open-source mod

"guide": "## Overview\n\n[MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is an advanced MoE language\nmodel from [MiniMax](https://www.minimax.io/). Highlights:\n\n- Superior intelligence \u2014 #1 among open-source models globall

"guide": "## Overview\n\n[MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is an advanced MoE language\nmodel from [MiniMax](https://www.minimax.io/). Highlights:\n\n- Superior intelligence \u2014 #1 among open-source models globall

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[InternVL3.5](https://github.com/OpenGVLab/InternVL) is a vision-language model developed\nby Shanghai AI Laboratory. It supports single-image and 

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[InternVL3.5](https://github.com/OpenGVLab/InternVL) is a vision-language model developed\nby Shanghai AI Laboratory. It supports single-image and 

"strategy_overrides": {},         "guide": "## Overview\n\n[InternVL3.5](https://github.com/OpenGVLab/InternVL) is a vision-language model developed\nby Shanghai AI Laboratory. It supports single-image and multi-image prompts, plus an\nopti

"strategy_overrides": {},         "guide": "## Overview\n\n[InternVL3.5](https://github.com/OpenGVLab/InternVL) is a vision-language model developed\nby Shanghai AI Laboratory. It supports single-image and multi-image prompts, plus an\nopti

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

{             "note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

{             "note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

"note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"           },

"note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"           },

"command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

"command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

},         "strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL-1.5](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5) is the next\ngeneration of PaddleOCR-VL \u2014 same 0.9B architecture (NaViT-style dynamic-resolu

},         "strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL-1.5](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5) is the next\ngeneration of PaddleOCR-VL \u2014 same 0.9B architecture (NaViT-style dynamic-resolu

"strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL-1.5](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5) is the next\ngeneration of PaddleOCR-VL \u2014 same 0.9B architecture (NaViT-style dynamic-resolution\nvisio

"strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL-1.5](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5) is the next\ngeneration of PaddleOCR-VL \u2014 same 0.9B architecture (NaViT-style dynamic-resolution\nvisio

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

{             "note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

{             "note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

"note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"           },

"note": "PaddlePaddle runtime (install in a separate venv from vllm to avoid conflicts)",             "command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"           },

"command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

"command": "uv pip install paddlepaddle-gpu==3.2.1 --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/"

},         "strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) is a SOTA resource-efficient\nmodel for document parsing. Its core (PaddleOCR-VL-0.9B) combines a NaViT-s

},         "strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) is a SOTA resource-efficient\nmodel for document parsing. Its core (PaddleOCR-VL-0.9B) combines a NaViT-s

"strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) is a SOTA resource-efficient\nmodel for document parsing. Its core (PaddleOCR-VL-0.9B) combines a NaViT-style dynami

"strategy_overrides": {},         "guide": "## Overview\n\n[PaddleOCR-VL](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) is a SOTA resource-efficient\nmodel for document parsing. Its core (PaddleOCR-VL-0.9B) combines a NaViT-style dynami

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

{             "note": "vLLM-Omni must be installed from source and pins vllm==0.18.0 for diffusion support",             "command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && uv pip install -e . vllm==0.18.0

{             "note": "vLLM-Omni must be installed from source and pins vllm==0.18.0 for diffusion support",             "command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && uv pip install -e . vllm==0.18.0

"note": "vLLM-Omni must be installed from source and pins vllm==0.18.0 for diffusion support",             "command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && uv pip install -e . vllm==0.18.0"           }

"note": "vLLM-Omni must be installed from source and pins vllm==0.18.0 for diffusion support",             "command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && uv pip install -e . vllm==0.18.0"           }

"command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && uv pip install -e . vllm==0.18.0"

"command": "git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && uv pip install -e . vllm==0.18.0"

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nQwen-Image is a diffusion-based text-to-image model. This recipe documents the Qwen-Image family served via **vLLM-Omni**:\n\n| Model | HuggingFace

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nQwen-Image is a diffusion-based text-to-image model. This recipe documents the Qwen-Image family served via **vLLM-Omni**:\n\n| Model | HuggingFace

"strategy_overrides": {},         "guide": "## Overview\n\nQwen-Image is a diffusion-based text-to-image model. This recipe documents the Qwen-Image family served via **vLLM-Omni**:\n\n| Model | HuggingFace | Description |\n|-------|-------

"strategy_overrides": {},         "guide": "## Overview\n\nQwen-Image is a diffusion-based text-to-image model. This recipe documents the Qwen-Image family served via **vLLM-Omni**:\n\n| Model | HuggingFace | Description |\n|-------|-------

"guide": "## Overview\n\nQwen-Image is a diffusion-based text-to-image model. This recipe documents the Qwen-Image family served via **vLLM-Omni**:\n\n| Model | HuggingFace | Description |\n|-------|-------------|-------------|\n| **Qwen-Im

"guide": "## Overview\n\nQwen-Image is a diffusion-based text-to-image model. This recipe documents the Qwen-Image family served via **vLLM-Omni**:\n\n| Model | HuggingFace | Description |\n|-------|-------------|-------------|\n| **Qwen-Im

},         "strategy_overrides": {},         "guide": "## Overview\n\nThis guide describes how to run Qwen2.5-VL series on the targeted accelerated stack. Since BF16 is the commonly used precision for Qwen2.5-VL training, using BF16 in infe

},         "strategy_overrides": {},         "guide": "## Overview\n\nThis guide describes how to run Qwen2.5-VL series on the targeted accelerated stack. Since BF16 is the commonly used precision for Qwen2.5-VL training, using BF16 in infe

"strategy_overrides": {},         "guide": "## Overview\n\nThis guide describes how to run Qwen2.5-VL series on the targeted accelerated stack. Since BF16 is the commonly used precision for Qwen2.5-VL training, using BF16 in inference ensur

"strategy_overrides": {},         "guide": "## Overview\n\nThis guide describes how to run Qwen2.5-VL series on the targeted accelerated stack. Since BF16 is the commonly used precision for Qwen2.5-VL training, using BF16 in inference ensur

"guide": "## Overview\n\nThis guide describes how to run Qwen2.5-VL series on the targeted accelerated stack. Since BF16 is the commonly used precision for Qwen2.5-VL training, using BF16 in inference ensures the best accuracy.\n\n## Prereq

"guide": "## Overview\n\nThis guide describes how to run Qwen2.5-VL series on the targeted accelerated stack. Since BF16 is the commonly used precision for Qwen2.5-VL training, using BF16 in inference ensures the best accuracy.\n\n## Prereq

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507) is the flagship instruct MoE model in the Qwen3 series, with 235B total paramete

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507) is the flagship instruct MoE model in the Qwen3 series, with 235B total paramete

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507) is the flagship instruct MoE model in the Qwen3 series, with 235B total parameters and 22B active parameters. This

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507) is the flagship instruct MoE model in the Qwen3 series, with 235B total parameters and 22B active parameters. This

"guide": "## Overview\n\n[Qwen3](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507) is the flagship instruct MoE model in the Qwen3 series, with 235B total parameters and 22B active parameters. This guide covers deploying the model 

"guide": "## Overview\n\n[Qwen3](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507) is the flagship instruct MoE model in the Qwen3 series, with 235B total parameters and 22B active parameters. This guide covers deploying the model 

},         "strategy_overrides": {},         "guide": "## Overview\n\nQwen3-ASR is a speech-to-text model that achieves accurate and robust recognition across 11 languages and multiple accents. It supports prompting the model with text cont

},         "strategy_overrides": {},         "guide": "## Overview\n\nQwen3-ASR is a speech-to-text model that achieves accurate and robust recognition across 11 languages and multiple accents. It supports prompting the model with text cont

"strategy_overrides": {},         "guide": "## Overview\n\nQwen3-ASR is a speech-to-text model that achieves accurate and robust recognition across 11 languages and multiple accents. It supports prompting the model with text context in any 

"strategy_overrides": {},         "guide": "## Overview\n\nQwen3-ASR is a speech-to-text model that achieves accurate and robust recognition across 11 languages and multiple accents. It supports prompting the model with text context in any 

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Coder](https://github.com/QwenLM/Qwen3-Coder) is an advanced large language model created by the Qwen team. `Qwen3-Coder-480B-A35B-Instruct` is the flagship coder 

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Coder](https://github.com/QwenLM/Qwen3-Coder) is an advanced large language model created by the Qwen team. `Qwen3-Coder-480B-A35B-Instruct` is the flagship coder 

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Coder](https://github.com/QwenLM/Qwen3-Coder) is an advanced large language model created by the Qwen team. `Qwen3-Coder-480B-A35B-Instruct` is the flagship coder MoE with 48

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Coder](https://github.com/QwenLM/Qwen3-Coder) is an advanced large language model created by the Qwen team. `Qwen3-Coder-480B-A35B-Instruct` is the flagship coder MoE with 48

"guide": "## Overview\n\n[Qwen3-Coder](https://github.com/QwenLM/Qwen3-Coder) is an advanced large language model created by the Qwen team. `Qwen3-Coder-480B-A35B-Instruct` is the flagship coder MoE with 480B total / 35B active parameters. 

"guide": "## Overview\n\n[Qwen3-Coder](https://github.com/QwenLM/Qwen3-Coder) is an advanced large language model created by the Qwen team. `Qwen3-Coder-480B-A35B-Instruct` is the flagship coder MoE with 480B total / 35B active parameters. 

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Next](https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list) is an advanced LLM from the Qwen team featuring:\n- 

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Next](https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list) is an advanced LLM from the Qwen team featuring:\n- 

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Next](https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list) is an advanced LLM from the Qwen team featuring:\n- A hybrid at

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-Next](https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list) is an advanced LLM from the Qwen team featuring:\n- A hybrid at

"guide": "## Overview\n\n[Qwen3-Next](https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list) is an advanced LLM from the Qwen team featuring:\n- A hybrid attention mechanism\n- A highly spar

"guide": "## Overview\n\n[Qwen3-Next](https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list) is an advanced LLM from the Qwen team featuring:\n- A hybrid attention mechanism\n- A highly spar

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-VL](https://github.com/QwenLM/Qwen3-VL) is the most powerful vision-language model in the Qwen series, delivering upgrades to text understanding & generation, visu

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-VL](https://github.com/QwenLM/Qwen3-VL) is the most powerful vision-language model in the Qwen series, delivering upgrades to text understanding & generation, visu

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-VL](https://github.com/QwenLM/Qwen3-VL) is the most powerful vision-language model in the Qwen series, delivering upgrades to text understanding & generation, visual percepti

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3-VL](https://github.com/QwenLM/Qwen3-VL) is the most powerful vision-language model in the Qwen series, delivering upgrades to text understanding & generation, visual percepti

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) is the smallest\nmember of the Qwen3.5 family \u2014 same hybrid gated delta networks arch

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) is the smallest\nmember of the Qwen3.5 family \u2014 same hybrid gated delta networks arch

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) is the smallest\nmember of the Qwen3.5 family \u2014 same hybrid gated delta networks architecture\nand 262K context, at a s

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) is the smallest\nmember of the Qwen3.5 family \u2014 same hybrid gated delta networks architecture\nand 262K context, at a s

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

}         },         "guide": "## Overview\n\n[Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) is a mid-tier\nmember of the Qwen3.5 family, sharing the gated delta networks MoE architecture\nwith 122B total parameters and 

}         },         "guide": "## Overview\n\n[Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) is a mid-tier\nmember of the Qwen3.5 family, sharing the gated delta networks MoE architecture\nwith 122B total parameters and 

},         "guide": "## Overview\n\n[Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) is a mid-tier\nmember of the Qwen3.5 family, sharing the gated delta networks MoE architecture\nwith 122B total parameters and 10B activa

},         "guide": "## Overview\n\n[Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) is a mid-tier\nmember of the Qwen3.5 family, sharing the gated delta networks MoE architecture\nwith 122B total parameters and 10B activa

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) is the flagship dense\nmodel of the Qwen3.5 family. It uses the same gated delta networks hy

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) is the flagship dense\nmodel of the Qwen3.5 family. It uses the same gated delta networks hy

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) is the flagship dense\nmodel of the Qwen3.5 family. It uses the same gated delta networks hybrid\nattention as its MoE sibling

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) is the flagship dense\nmodel of the Qwen3.5 family. It uses the same gated delta networks hybrid\nattention as its MoE sibling

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) is a miniature dense\nQwen3.5 model \u2014 the full gated delta networks architecture, vision 

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) is a miniature dense\nQwen3.5 model \u2014 the full gated delta networks architecture, vision 

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) is a miniature dense\nQwen3.5 model \u2014 the full gated delta networks architecture, vision encoder,\nand 262K context, in a f

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) is a miniature dense\nQwen3.5 model \u2014 the full gated delta networks architecture, vision encoder,\nand 262K context, in a f

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) is the smallest\nMoE in the Qwen3.5 family, sharing the gated delta networks archite

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) is the smallest\nMoE in the Qwen3.5 family, sharing the gated delta networks archite

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) is the smallest\nMoE in the Qwen3.5 family, sharing the gated delta networks architecture with\n35B total parameters a

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) is the smallest\nMoE in the Qwen3.5 family, sharing the gated delta networks architecture with\n35B total parameters a

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

}         },         "guide": "## Overview\n\n[Qwen3.5](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) is a multimodal mixture-of-experts model featuring a gated delta networks architecture with 397B total parameters and 17B active paramete

}         },         "guide": "## Overview\n\n[Qwen3.5](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) is a multimodal mixture-of-experts model featuring a gated delta networks architecture with 397B total parameters and 17B active paramete

},         "guide": "## Overview\n\n[Qwen3.5](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) is a multimodal mixture-of-experts model featuring a gated delta networks architecture with 397B total parameters and 17B active parameters. This g

},         "guide": "## Overview\n\n[Qwen3.5](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) is a multimodal mixture-of-experts model featuring a gated delta networks architecture with 397B total parameters and 17B active parameters. This g

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) is the compact dense\nentry in the Qwen3.5 family \u2014 same gated delta networks architectur

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) is the compact dense\nentry in the Qwen3.5 family \u2014 same gated delta networks architectur

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) is the compact dense\nentry in the Qwen3.5 family \u2014 same gated delta networks architecture, vision\nencoder, 262K context, 

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) is the compact dense\nentry in the Qwen3.5 family \u2014 same gated delta networks architecture, vision\nencoder, 262K context, 

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) is a dense multimodal\nmodel from the Qwen3.5 family \u2014 same gated delta networks hybrid a

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) is a dense multimodal\nmodel from the Qwen3.5 family \u2014 same gated delta networks hybrid a

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) is a dense multimodal\nmodel from the Qwen3.5 family \u2014 same gated delta networks hybrid attention,\nvision encoder, 262K co

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) is a dense multimodal\nmodel from the Qwen3.5 family \u2014 same gated delta networks hybrid attention,\nvision encoder, 262K co

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) is the flagship dense\nmodel of the Qwen3.6 family. It uses the same gated delta networks hy

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) is the flagship dense\nmodel of the Qwen3.6 family. It uses the same gated delta networks hy

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) is the flagship dense\nmodel of the Qwen3.6 family. It uses the same gated delta networks hybrid\nattention as its MoE sibling

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) is the flagship dense\nmodel of the Qwen3.6 family. It uses the same gated delta networks hybrid\nattention as its MoE sibling

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) is the smaller sibling\nof Qwen3.5, sharing the same gated-delta-networks MoE architecture but with 35B tot

},         "strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) is the smaller sibling\nof Qwen3.5, sharing the same gated-delta-networks MoE architecture but with 35B tot

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) is the smaller sibling\nof Qwen3.5, sharing the same gated-delta-networks MoE architecture but with 35B total\nparamet

"strategy_overrides": {},         "guide": "## Overview\n\n[Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) is the smaller sibling\nof Qwen3.5, sharing the same gated-delta-networks MoE architecture but with 35B total\nparamet

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n**Qwen3Guard-Gen** is a lightweight text-only guardrail model. This guide describes how to run the 8B variant \u2014 as well as the 4B and 0.6B var

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n**Qwen3Guard-Gen** is a lightweight text-only guardrail model. This guide describes how to run the 8B variant \u2014 as well as the 4B and 0.6B var

"strategy_overrides": {},         "guide": "## Overview\n\n**Qwen3Guard-Gen** is a lightweight text-only guardrail model. This guide describes how to run the 8B variant \u2014 as well as the 4B and 0.6B variants \u2014 on GPU using vLLM.\n\

"strategy_overrides": {},         "guide": "## Overview\n\n**Qwen3Guard-Gen** is a lightweight text-only guardrail model. This guide describes how to run the 8B variant \u2014 as well as the 4B and 0.6B variants \u2014 on GPU using vLLM.\n\

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

{             "note": "vllm-omni pinned to a specific commit that includes Wan2.2 text-to-video support",             "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git@ef01223c42be10ee260b9f6e5ec31894cd09d86e"

{             "note": "vllm-omni pinned to a specific commit that includes Wan2.2 text-to-video support",             "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git@ef01223c42be10ee260b9f6e5ec31894cd09d86e"

"note": "vllm-omni pinned to a specific commit that includes Wan2.2 text-to-video support",             "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git@ef01223c42be10ee260b9f6e5ec31894cd09d86e"           }

"note": "vllm-omni pinned to a specific commit that includes Wan2.2 text-to-video support",             "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git@ef01223c42be10ee260b9f6e5ec31894cd09d86e"           }

"command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git@ef01223c42be10ee260b9f6e5ec31894cd09d86e"

"command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git@ef01223c42be10ee260b9f6e5ec31894cd09d86e"

},         "strategy_overrides": {},         "guide": "## Overview\n\nWan2.2 is a video generation family served via **vLLM-Omni** with optional **Cache-DiT**\nacceleration:\n\n- `Wan-AI/Wan2.2-T2V-A14B-Diffusers` \u2014 Text-to-Video (MoE,

},         "strategy_overrides": {},         "guide": "## Overview\n\nWan2.2 is a video generation family served via **vLLM-Omni** with optional **Cache-DiT**\nacceleration:\n\n- `Wan-AI/Wan2.2-T2V-A14B-Diffusers` \u2014 Text-to-Video (MoE,

"strategy_overrides": {},         "guide": "## Overview\n\nWan2.2 is a video generation family served via **vLLM-Omni** with optional **Cache-DiT**\nacceleration:\n\n- `Wan-AI/Wan2.2-T2V-A14B-Diffusers` \u2014 Text-to-Video (MoE, 14B active

"strategy_overrides": {},         "guide": "## Overview\n\nWan2.2 is a video generation family served via **vLLM-Omni** with optional **Cache-DiT**\nacceleration:\n\n- `Wan-AI/Wan2.2-T2V-A14B-Diffusers` \u2014 Text-to-Video (MoE, 14B active

"guide": "## Overview\n\nWan2.2 is a video generation family served via **vLLM-Omni** with optional **Cache-DiT**\nacceleration:\n\n- `Wan-AI/Wan2.2-T2V-A14B-Diffusers` \u2014 Text-to-Video (MoE, 14B active)\n- `Wan-AI/Wan2.2-I2V-A14B-Diffu

"guide": "## Overview\n\nWan2.2 is a video generation family served via **vLLM-Omni** with optional **Cache-DiT**\nacceleration:\n\n- `Wan-AI/Wan2.2-T2V-A14B-Diffusers` \u2014 Text-to-Video (MoE, 14B active)\n- `Wan-AI/Wan2.2-I2V-A14B-Diffu

},         "strategy_overrides": {},         "guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and

},         "strategy_overrides": {},         "guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and

"strategy_overrides": {},         "guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and agentic\nw

"strategy_overrides": {},         "guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and agentic\nw

"guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and agentic\nworkflows, it features hybrid atten

"guide": "## Overview\n\n[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a MoE language model\nwith 309B total parameters and 15B active. Designed for high-speed reasoning and agentic\nworkflows, it features hybrid atten

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters\nand 42B active per token. It uses 384 routed experts (top-8) wi

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters\nand 42B active per token. It uses 384 routed experts (top-8) wi

"strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters\nand 42B active per token. It uses 384 routed experts (top-8) with hybrid\nattention (full-attenti

"strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters\nand 42B active per token. It uses 384 routed experts (top-8) with hybrid\nattention (full-attenti

"guide": "## Overview\n\nMiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters\nand 42B active per token. It uses 384 routed experts (top-8) with hybrid\nattention (full-attention + sliding-window 128 at 6:1 rat

"guide": "## Overview\n\nMiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters\nand 42B active per token. It uses 384 routed experts (top-8) with hybrid\nattention (full-attention + sliding-window 128 at 6:1 rat

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5 is Xiaomi's native omnimodal MoE model with 310B total parameters\nand 15B active per token, supporting text, image, video, and audio und

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5 is Xiaomi's native omnimodal MoE model with 310B total parameters\nand 15B active per token, supporting text, image, video, and audio und

"strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5 is Xiaomi's native omnimodal MoE model with 310B total parameters\nand 15B active per token, supporting text, image, video, and audio understanding\nwithin a unified archi

"strategy_overrides": {},         "guide": "## Overview\n\nMiMo-V2.5 is Xiaomi's native omnimodal MoE model with 310B total parameters\nand 15B active per token, supporting text, image, video, and audio understanding\nwithin a unified archi

"guide": "## Overview\n\nMiMo-V2.5 is Xiaomi's native omnimodal MoE model with 310B total parameters\nand 15B active per token, supporting text, image, video, and audio understanding\nwithin a unified architecture. Built on the MiMo-V2-Flas

"guide": "## Overview\n\nMiMo-V2.5 is Xiaomi's native omnimodal MoE model with 310B total parameters\nand 15B active per token, supporting text, image, video, and audio understanding\nwithin a unified architecture. Built on the MiMo-V2-Flas

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Trinity-Large-Thinking](https://huggingface.co/arcee-ai/Trinity-Large-Thinking) is\nArcee AI's reasoning-focused Trinity Large checkpoint \u2014 a

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\n[Trinity-Large-Thinking](https://huggingface.co/arcee-ai/Trinity-Large-Thinking) is\nArcee AI's reasoning-focused Trinity Large checkpoint \u2014 a

"strategy_overrides": {},         "guide": "## Overview\n\n[Trinity-Large-Thinking](https://huggingface.co/arcee-ai/Trinity-Large-Thinking) is\nArcee AI's reasoning-focused Trinity Large checkpoint \u2014 a sparse MoE model designed for\nlo

"strategy_overrides": {},         "guide": "## Overview\n\n[Trinity-Large-Thinking](https://huggingface.co/arcee-ai/Trinity-Large-Thinking) is\nArcee AI's reasoning-focused Trinity Large checkpoint \u2014 a sparse MoE model designed for\nlo

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

},         "strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 is Baidu's MoE language model family. This recipe covers the text-only\nvariants:\n\n- `baidu/ERNIE-4.5-21B-A3B-PT` \u2014 21B total / 3B active (fits on 1x80GB

},         "strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 is Baidu's MoE language model family. This recipe covers the text-only\nvariants:\n\n- `baidu/ERNIE-4.5-21B-A3B-PT` \u2014 21B total / 3B active (fits on 1x80GB

"strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 is Baidu's MoE language model family. This recipe covers the text-only\nvariants:\n\n- `baidu/ERNIE-4.5-21B-A3B-PT` \u2014 21B total / 3B active (fits on 1x80GB)\n- `baidu

"strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 is Baidu's MoE language model family. This recipe covers the text-only\nvariants:\n\n- `baidu/ERNIE-4.5-21B-A3B-PT` \u2014 21B total / 3B active (fits on 1x80GB)\n- `baidu

"guide": "## Overview\n\nERNIE 4.5 is Baidu's MoE language model family. This recipe covers the text-only\nvariants:\n\n- `baidu/ERNIE-4.5-21B-A3B-PT` \u2014 21B total / 3B active (fits on 1x80GB)\n- `baidu/ERNIE-4.5-300B-A47B-PT` \u2014 30

"guide": "## Overview\n\nERNIE 4.5 is Baidu's MoE language model family. This recipe covers the text-only\nvariants:\n\n- `baidu/ERNIE-4.5-21B-A3B-PT` \u2014 21B total / 3B active (fits on 1x80GB)\n- `baidu/ERNIE-4.5-300B-A47B-PT` \u2014 30

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 VL is Baidu's multimodal MoE model with heterogeneous experts (separate text\nand vision experts). Because of the heterogeneous architect

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 VL is Baidu's multimodal MoE model with heterogeneous experts (separate text\nand vision experts). Because of the heterogeneous architect

"strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 VL is Baidu's multimodal MoE model with heterogeneous experts (separate text\nand vision experts). Because of the heterogeneous architecture, **torch.compile and CUDA\ngra

"strategy_overrides": {},         "guide": "## Overview\n\nERNIE 4.5 VL is Baidu's multimodal MoE model with heterogeneous experts (separate text\nand vision experts). Because of the heterogeneous architecture, **torch.compile and CUDA\ngra

"guide": "## Overview\n\nERNIE 4.5 VL is Baidu's multimodal MoE model with heterogeneous experts (separate text\nand vision experts). Because of the heterogeneous architecture, **torch.compile and CUDA\ngraphs are not supported**.\n\n- `bai

"guide": "## Overview\n\nERNIE 4.5 VL is Baidu's multimodal MoE model with heterogeneous experts (separate text\nand vision experts). Because of the heterogeneous architecture, **torch.compile and CUDA\ngraphs are not supported**.\n\n- `bai

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR-2 is a frontier OCR model exploring optical context compression for LLMs.\nIt iterates on DeepSeek-OCR with better grounding and markd

"hardware_overrides": {},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR-2 is a frontier OCR model exploring optical context compression for LLMs.\nIt iterates on DeepSeek-OCR with better grounding and markd

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR-2 is a frontier OCR model exploring optical context compression for LLMs.\nIt iterates on DeepSeek-OCR with better grounding and markdown conversion, and supports\nprom

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR-2 is a frontier OCR model exploring optical context compression for LLMs.\nIt iterates on DeepSeek-OCR with better grounding and markdown conversion, and supports\nprom

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR is a frontier OCR model exploring optical context compression for LLMs.\nIt is optimized for document parsing, free-form OCR, and markdown generation from im

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR is a frontier OCR model exploring optical context compression for LLMs.\nIt is optimized for document parsing, free-form OCR, and markdown generation from im

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR is a frontier OCR model exploring optical context compression for LLMs.\nIt is optimized for document parsing, free-form OCR, and markdown generation from images,\nand 

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-OCR is a frontier OCR model exploring optical context compression for LLMs.\nIt is optimized for document parsing, free-form OCR, and markdown generation from images,\nand 

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-R1 is a 671B-parameter Mixture-of-Experts reasoning model (37B activated per\ntoken) that shares its architecture with DeepSeek-V3, so the same launch recipes ap

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-R1 is a 671B-parameter Mixture-of-Experts reasoning model (37B activated per\ntoken) that shares its architecture with DeepSeek-V3, so the same launch recipes ap

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-R1 is a 671B-parameter Mixture-of-Experts reasoning model (37B activated per\ntoken) that shares its architecture with DeepSeek-V3, so the same launch recipes apply\nto bot

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-R1 is a 671B-parameter Mixture-of-Experts reasoning model (37B activated per\ntoken) that shares its architecture with DeepSeek-V3, so the same launch recipes apply\nto bot

"guide": "## Overview\n\nDeepSeek-R1 is a 671B-parameter Mixture-of-Experts reasoning model (37B activated per\ntoken) that shares its architecture with DeepSeek-V3, so the same launch recipes apply\nto both. DeepSeek publishes a refreshed 

"guide": "## Overview\n\nDeepSeek-R1 is a 671B-parameter Mixture-of-Experts reasoning model (37B activated per\ntoken) that shares its architecture with DeepSeek-V3, so the same launch recipes apply\nto both. DeepSeek publishes a refreshed 

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.1 is a hybrid MoE model that supports both thinking and non-thinking modes.\nYou can dynamically switch between the two modes from the client by passing\n`ext

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.1 is a hybrid MoE model that supports both thinking and non-thinking modes.\nYou can dynamically switch between the two modes from the client by passing\n`ext

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.1 is a hybrid MoE model that supports both thinking and non-thinking modes.\nYou can dynamically switch between the two modes from the client by passing\n`extra_body={\"

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.1 is a hybrid MoE model that supports both thinking and non-thinking modes.\nYou can dynamically switch between the two modes from the client by passing\n`extra_body={\"

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

[Credential-related source omitted]

{             "note": "DeepGEMM pinned build for MQA logits (FP8 MoE kernels)",             "command": "uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation"

{             "note": "DeepGEMM pinned build for MQA logits (FP8 MoE kernels)",             "command": "uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation"

"note": "DeepGEMM pinned build for MQA logits (FP8 MoE kernels)",             "command": "uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation"           }

"note": "DeepGEMM pinned build for MQA logits (FP8 MoE kernels)",             "command": "uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation"           }

"command": "uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation"

"command": "uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation"

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.2-Exp is a sparse-attention MoE preview. Its main architecture is similar to\nDeepSeek-V3.1, with a sparse attention mechanism. Only Hopper and Blackwell data

},         "strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.2-Exp is a sparse-attention MoE preview. Its main architecture is similar to\nDeepSeek-V3.1, with a sparse attention mechanism. Only Hopper and Blackwell data

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.2-Exp is a sparse-attention MoE preview. Its main architecture is similar to\nDeepSeek-V3.1, with a sparse attention mechanism. Only Hopper and Blackwell data center\nGP

"strategy_overrides": {},         "guide": "## Overview\n\nDeepSeek-V3.2-Exp is a sparse-attention MoE preview. Its main architecture is similar to\nDeepSeek-V3.1, with a sparse attention mechanism. Only Hopper and Blackwell data center\nGP

"guide": "## Overview\n\nDeepSeek-V3.2-Exp is a sparse-attention MoE preview. Its main architecture is similar to\nDeepSeek-V3.1, with a sparse attention mechanism. Only Hopper and Blackwell data center\nGPUs are supported for now.\n\n## Pr