Menu

Individual skill

orchestrating-llm-attacks-with-pyrit

0upvotes

Build automated multi-turn adversarial attacks against conversational LLM targets using Microsoft PyRIT's RedTeamingOrchestrator, CrescendoOrchestrator (gradual escalation), and TreeOfAttacksWithPruningOrchestrator (adaptive branching), with scorer feedback loops and persisted conversation memory. Use when single-shot LLM scanning is insufficient and you need multi-turn, scorer-driven AI red-team campaigns against a chatbot or agent.

skills/orchestrating-llm-attacks-with-pyrit/SKILL.md

Read pinned source

Source commit 54a798831d2266a3ca61ce68a7acb80b81160d57

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
No
Observed behavior
Network or broad shell
Last checked
Nov 9, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
35,893
Repository last updated
Aug 31, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt
# Orchestrating LLM Attacks with PyRIT

> **Legal and Authorized-Use Notice:** PyRIT generates adversarial and potentially harmful prompts to test AI systems. Use it only against models and endpoints you own or are explicitly authorized to assess. Multi-turn orchestrators consume large numbers of tokens against both the target and the adversarial/scoring models; account for cost and terms of service. Unauthorized use is prohibited.

## Overview

PyRIT (Python Risk Identification Tool for generative AI) is an open-source automation framework from Microsoft's AI Red Team, distributed at github.com/microsoft/PyRIT. Where a single-shot scanner sends one prompt and checks the answer, PyRIT automates *multi-turn* adversarial conversations: an attacker model and a scorer model collaborate in a loop to drive a target model toward a defined objective (for example, eliciting restricted content, leaking a system prompt, or making an agent perform an unauthorized tool call). This mirrors how real adversaries iterate against a chatbot rather than relying on one magic prompt.

PyRIT is built from composable primitives. **Targets** (`pyrit.prompt_target`) wrap the systems being probed and the helper models — `OpenAIChatTarget`, `AzureMLChatTarget`, `HTTPTarget`, and others. **Orchestrators / attacks** (`pyrit.orchestrator`) implement attack strategies; all multi-turn strategies subclass `MultiTurnOrchestrator`. The headline strategies are `RedTeamingOrchestrator` (a generic adversarial-chat loop), `CrescendoOrchestrator` (the Crescendo technique — start benign and escalate gradually so each turn looks reasonable in isolation), and `TreeOfAttacksWithPruningOrchestrator` (TAP — branch multiple attack lines in parallel, expand the branches the scorer rates as progressing, and prune dead ends). **Scorers** (`pyrit.score`) such as `SelfAskTrueFalseScorer` decide whether the objective was met and feed that judgment back into the loop. **Converters** mutate prompts (base64, translation, ASCII art) to evade filters, and **memory** persists every turn for later analysis.

This skill maps to MITRE ATLAS **AML.T0051 (LLM Prompt Injection)** and **AML.T0054 (LLM Jailbreak)** because PyRIT operationalizes both at scale across conversation turns, and supports NIST AI RMF **MEASURE-2.7** by producing repeatable, scored security measurements of an AI system.

## When to Use

- When single-shot scanning (e.g. garak) finds a model robust to one-prompt attacks and you need to test multi-turn escalation (Crescendo) or adaptive branching (TAP).
Back to top