Menu

Individual skill

detecting-data-and-model-poisoning

0upvotes

Identify poisoned training data and backdoored ML models across the pipeline using IBM's Adversarial Robustness Toolbox (activation clustering, spectral signatures, trigger reconstruction), Cleanlab for label-quality issues, and supply-chain checks like weight-hash verification and safetensors enforcement. Use before training or deploying on third-party/user-contributed data or downloaded checkpoints, during ML supply-chain reviews, or when investigating model misbehavior tied to specific inputs (suspected backdoor trigger).

skills/detecting-data-and-model-poisoning/SKILL.md

Read pinned source

Source commit 54a798831d2266a3ca61ce68a7acb80b81160d57

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
No
Observed behavior
Network or broad shell
Last checked
Nov 9, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
35,926
Repository last updated
Aug 31, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt
# Detecting Data and Model Poisoning

> **Authorized-use-only notice:** This skill includes routines that craft poisoned samples and backdoor triggers for *defensive validation*. Generate and use poisoned data and backdoored models only in isolated test environments you control. Never deploy a backdoored model or distribute poisoned datasets.

## Overview

Data poisoning and model backdooring attack the *integrity* of an ML system at training time rather than at inference. In **data poisoning** (MITRE ATLAS **AML.T0020 Poison Training Data**), an adversary injects manipulated samples into the training, fine-tuning, or RAG corpus so the resulting model misbehaves — degraded accuracy, targeted misclassification, or an attacker-chosen bias. In **model backdooring** (MITRE ATLAS **AML.T0018 Backdoor ML Model**), the model behaves normally on clean inputs but produces an attacker-chosen output whenever a hidden *trigger* (a pixel patch, a rare token, a phrase) is present. Both are amplified by **ML supply-chain compromise (AML.T0010)**: poisoned public datasets, trojaned pre-trained weights downloaded from a hub, or a malicious model serialization. This is OWASP **LLM04:2025 Data and Model Poisoning**.

Detection spans the pipeline. On the *data* side: provenance and integrity checks, statistical outlier and label-flip detection, and de-duplication of suspiciously near-identical samples. On the *model* side: activation-clustering and spectral-signature analysis (which exploit the fact that poisoned samples activate the network differently than clean ones) and trigger reconstruction. On the *supply-chain* side: verifying weights hashes/signatures and refusing unsafe serialization formats (pickle-based `.bin`/`.pt`) in favor of safetensors. This skill implements all three using IBM's **Adversarial Robustness Toolbox (ART)**, **Cleanlab** for label-quality issues, and integrity tooling.

## When to Use

- Before training/fine-tuning on third-party or user-contributed data.
- Before deploying a model built on a downloaded pre-trained checkpoint.
- During an ML supply-chain security review.
Back to top