Menu

Individual skill

opik-annotate

0upvotes

Set up human review in Opik — create an annotation queue for a project's traces or threads and give it a custom annotator view, an HTML page that shows annotators each item the way they need to judge it (the email beside the reply, the conversation as chat bubbles) instead of raw JSON. Writes through the Opik MCP when connected, the Opik SDK otherwise. Returns the queue, its link, and what annotators will see. Use for "build a custom annotator view", "set up an annotation queue", "make the review queue show the email and the reply side by side", "change what annotators see", "remove the custom view". Not for automatic scoring (use online-eval) or for finding which traces to review (use diagnose).

plugins/opik/skills/opik-annotate/SKILL.md

Read pinned source

Source commit 9f66464c5fb5f58f70f277f48cbccdb93025bfdd

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
No
Repository scan complete
No
Skill scan complete
No
Observed behavior
Not checked yet
Last checked
Nov 6, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
226
Repository last updated
Nov 6, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt

# Annotate — A Review Queue Annotators Can Read

**Definition of done:** the **queue exists with the view saved, and the user knows where to check it** — the write returned without a `warning`, and the answer carries the queue's link. A view is built from the project's **real** traces or threads, never from a guessed shape. If the queue or the view can't be saved, stop at the **first** genuine blocker and return **exactly one** next step.

Operate: **look at real items first, cover every shape you saw, show text not JSON, keep scoring in Opik's own panel.** This skill writes to Opik only and changes no application code.

## Inputs

`/opik-annotate <what annotators judge>` ("the drafted reply", "whether the agent solved the ticket"), `/opik-annotate <queue name or id>` (change that queue's view), or a request to remove a view. Infer the rest; optional overrides: project (default: configured) · scope (default: `trace`; `thread` for whole conversations) · feedback definitions (default: the queue's own).

## Activation — the only in-scope work

### 1. Resolve the project and the queue
With the Opik MCP connected: `list('annotation_queue', project_name=…)` shows the project's queues with their scope and item count; `read('annotation_queue', id_or_name)` returns one with its instructions, feedback definitions and view summary. The view's html is stated by size; `read('annotation_queue', id, fields=['annotator_view.html'])` returns it whole — read it before changing a view, and change only what was asked.