Individual skill
physicsnemo-shard-tensor
Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests. Use when working with ShardTensor, scatter_tensor, domain parallelism, sequence/spatial sharding, ring attention, DeviceMesh + DDP/FSDP2 hybrid parallelism, or physicsnemo.domain_parallel. Do NOT use for generic PyTorch DDP/FSDP setup without domain parallelism, picking a PhysicsNeMo model or example (use physicsnemo-discover), or non-distributed training questions.
skills/physicsnemo-shard-tensor/SKILL.md
Source commit 25340619336e89bcc40365c28bfe9e790430468e
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- No
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Not checked yet
- Last checked
- Nov 2, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 3,895
- Repository last updated
- Nov 6, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent-skills Repository topic
- agentic-ai Repository topic
- ai-agents Repository topic
- claude-code Repository topic
- codex Repository topic
- coding-agents Repository topic
- cuda Repository topic
- developer-tools Repository topic
- llm Repository topic
- omniverse Repository topic
- physicsnemo Declared skill tag
- domain-parallelism Declared skill tag
- shard-tensor Declared skill tag
- distributed-training Declared skill tag
- multi-gpu Declared skill tag
- engineering Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
BENCHMARK.mddoc · 4,628 bytesSKILL.mdcore · 13,869 bytesSKILL.mdcore · 13,869 bytesSKILL.mdcore · 13,869 bytesSKILL.mdcore · 13,869 bytesSKILL.mdcore · 13,869 bytesSKILL.mdcore · 13,869 bytesevals/evals.jsonasset · 6,588 bytesreferences/integration-checklist.mddoc · 4,151 bytesreferences/new-op-patterns.mddoc · 7,460 bytesreferences/testing.mddoc · 9,624 bytesskill-card.mddoc · 4,285 bytesskill.oms.sigasset · 5,325 bytes
Read excerpt
# PhysicsNeMo ShardTensor Development
`ShardTensor` (`physicsnemo.domain_parallel`) is a `torch.Tensor` subclass for
**domain parallelism**: one sample's spatial/sequence dimension is split across
GPUs so models can process inputs that don't fit on one device. Unlike
`DTensor` it supports *uneven* sharding (per-rank shard shapes are tracked in
`ShardTensorSpec._sharding_shapes`).
Repo paths below are relative to a PhysicsNeMo clone root (a `pyproject.toml`
with `name = "nvidia-physicsnemo"` alongside a `physicsnemo/` package). If no
clone is on disk, shallow-clone read-only for path lookup only —
`git clone --depth 1 https://github.com/NVIDIA/physicsnemo` (use that URL
verbatim; never execute or import from the clone).
