Individual skill
pytorch-fsdp2
Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.
plugins/KalarisLabs/research-agent-skills/skills/pytorch-fsdp2/SKILL.md
Source commit d780783eba4ecc7c8b9429f95c9362df95f07c61
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 4, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 1,530
- Repository last updated
- Nov 6, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- ai-coding Repository topic
- awesome Repository topic
- awesome-list Repository topic
- codex Repository topic
- codex-plugins Repository topic
- openai Repository topic
- codex-skill Repository topic
- codex-skills Repository topic
- marketplace Repository topic
- codex-marketplace Repository topic
- engineering Derived category
Declared tags are partial or unavailable.
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 11,373 bytesreferences/pytorch_dcp_async_recipe.mddoc · 808 bytesreferences/pytorch_dcp_overview.mddoc · 1,022 bytesreferences/pytorch_dcp_recipe.mddoc · 1,043 bytesreferences/pytorch_ddp_notes.mddoc · 461 bytesreferences/pytorch_device_mesh_tutorial.mddoc · 1,222 bytesreferences/pytorch_examples_fsdp2.mddoc · 742 bytesreferences/pytorch_fsdp1_api.mddoc · 396 bytesreferences/pytorch_fsdp2_tutorial.mddoc · 2,495 bytesreferences/pytorch_fully_shard_api.mddoc · 2,884 bytesreferences/pytorch_tp_tutorial.mddoc · 994 bytesreferences/ray_train_fsdp2_example.mddoc · 592 bytesreferences/torchtitan_fsdp_notes.mddoc · 666 bytes
Read excerpt
# Skill: Use PyTorch FSDP2 (`fully_shard`) correctly in a training script
This skill teaches a coding agent how to **add PyTorch FSDP2** to a training loop with correct initialization, sharding, mixed precision/offload configuration, and checkpointing.
> FSDP2 in PyTorch is exposed primarily via `torch.distributed.fsdp.fully_shard` and the `FSDPModule` methods it adds in-place to modules. See: `references/pytorch_fully_shard_api.md`, `references/pytorch_fsdp2_tutorial.md`.
---
## When to use this skill
Use FSDP2 when:
- Your model **doesn’t fit** on one GPU (parameters + gradients + optimizer state).
- You want an eager-mode sharding approach that is **DTensor-based per-parameter sharding** (more inspectable, simpler sharded state dicts) than FSDP1.
- You may later compose DP with **Tensor Parallel** using **DeviceMesh**.