Individual skill
tao-generate-referring-expressions
Four-step image referring-expression pipeline: turns images plus KITTI bounding-box labels into region descriptions, scene captions, grounded referring expressions, and (optionally) verified expressions via VLM distillation. Use when the user wants to generate referring-expression annotations from images with KITTI labels, build region descriptions, produce grouped grounding phrases tied to bboxes, run a double-check verification pass on grounding expressions, auto-label traffic / scene images for referring datasets, or run the image_referring_expression pipeline. Triggers include 'referring expression', 'region description', 'KITTI labels', 'spatial relationship annotation', 'auto-label image referring expression', 'image_referring_expression'.
skills/tao-generate-referring-expressions/SKILL.md
Source commit 25340619336e89bcc40365c28bfe9e790430468e
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 2, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 3,886
- Repository last updated
- Nov 6, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent-skills Repository topic
- agentic-ai Repository topic
- ai-agents Repository topic
- claude-code Repository topic
- codex Repository topic
- coding-agents Repository topic
- cuda Repository topic
- developer-tools Repository topic
- llm Repository topic
- omniverse Repository topic
- image Declared skill tag
- referring-expression Declared skill tag
- kitti Declared skill tag
- bounding-boxes Declared skill tag
- auto-label Declared skill tag
- vlm Declared skill tag
- engineering Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
BENCHMARK.mddoc · 7,501 bytesSKILL.mdcore · 10,415 bytesSKILL.mdcore · 10,415 bytesconfig/skillspector-baseline.yamlasset · 834 bytesevals/evals.jsonasset · 849 bytesreferences/configuration.mddoc · 12,993 bytesreferences/skill_info.yamlasset · 1,504 bytesreferences/vllm_server.mddoc · 6,922 bytesskill-card.mddoc · 4,864 bytesskill.oms.sigasset · 5,577 bytes
Read excerpt
# Image Referring Expression Pipeline
> **Standalone install?** If this session was not initialized by the TAO skill bank plugin, run the `tao-setup` skill first (host preflight, credentials, cross-skill discovery).
Generate referring-expression and grounding annotations from images with KITTI-format bounding box labels. A single VLM (Gemini or any OpenAI-compatible endpoint) runs four steps: per-object region descriptions, holistic image captions, grouped grounding expressions tied to bboxes, and an optional double-check verification pass.
## Purpose
Transform `(image, KITTI labels)` pairs into a unified `annotations.jsonl` containing rich, grounded referring expressions. The VLM acts as a "teacher" annotator: Steps 0-1 see the image; Step 2 groups Step 0 outputs into grouping phrases with bbox lists; Step 3 (optional) re-examines those bboxes against the image and corrects mismatches.
## Pipeline Architecture
```
Step 0: Region expression ──┐