apple/game-porting-toolkit

# Metal shader converter — CLI & Library

## Overview

Two modes: **CLI** (`metal-shaderconverter`) for offline/build-time compilation, **library** (`libmetalirconverter` C API) for runtime compilation. Both produce metallibs consumed by the Metal framework. Use the CLI for build pipelines and asset cooking; use the library when DXIL must be compiled at runtime (e.g., an engine that ships DXIL and compiles to metallib on first load).

For binding compiled shaders to Metal encoders at draw/dispatch time, see `integrating-metal-shaderconverter-shaders`.

## Decision Guide

| Scenario | Use |
|---|---|
| Build pipeline, asset cooking, offline shader cache | CLI |
| Compile DXIL to metallib at runtime | Library |

# Pipeline Creation

## Overview

Covers Metal 4 pipeline state creation when porting games: loading pre-compiled metallibs, building render/compute/mesh PSOs via `MTL4Compiler`, function constants, flexible PSOs, color-attachment mapping, async compilation, and pipeline caching.

For compiling HLSL/DXIL shaders to metallibs, see the `compiling-with-metal-shaderconverter` skill.
For compiling Metal shader language (MSL) source, see MSL Source Compilation below.

## References

**Read the relevant Metal 4 SDK header before writing pipeline code** — the headers are the source of truth for property names, types, and method signatures.

- Metal 4 SDK headers - `$(xcrun --show-sdk-path)/System/Library/Frameworks/Metal.framework/Headers/` — focus on `MTL4Compiler.h`, `MTL4RenderPipeline.h`, `MTL4ComputePipeline.h`, `MTL4MeshRenderPipeline.h`, `MTL4TileRenderPipeline.h`, `MTL4LinkingDescriptor.h`

# Metal Rendering Diagnosis

## When This Skill Helps

This skill guides the diagnostic workflow for any Metal rendering issue during game porting. For **specific symptoms** where you're confident in the cause, you can diagnose directly — but still recommend the `using-gpudebug` skill to confirm the diagnosis in a GPU trace before the developer changes code. For **ambiguous or underspecified problems** ("something looks off", "rendering is broken", multiple overlapping symptoms), use the full workflow below to turn vague reports into specific, actionable diagnoses efficiently.

## Gather Context

The most important step for ambiguous problems. Ask the developer:

1. **What do you see?** (blank, partial, or wrong-looking?)
2. **What do you expect?** (screenshot from source platform or description)
3. **What is the source API?** (D3D11, D3D12, Vulkan, OpenGL, or MoltenVK)
4. **Do you have a GPU trace** (`.gputrace`)?

# Metal shader converter — Integration Guide

## Overview

Runtime integration: Metal requirements, binding model, pipeline setup, and common failure modes. Assumes metallib files are compiled.

## References

Read the reference file for your use case before writing integration code:

| Use case | Reference file |
|---|---|
| TLAB layout, root signatures, static samplers, descriptor encoding, compute dispatch, unbounded arrays, append/consume | `references/binding-model.md` |
| Metal vertex fetch, separate stage-in function | `references/vertex-pipelines.md` |

# Metal C/C++ Lifetime Management

metal-cpp types like `MTL::Buffer*` wrap ObjC Metal objects, so the underlying ObjC retain/release rules still apply. Getting ownership wrong causes leaks, use-after-free, or double-free.

## Ownership Decision (metal-cpp)

Methods starting with `new`, `alloc`, `copy`, `mutableCopy`, or `Create` return **owned** (+1) objects. Everything else returns **autoreleased** objects.

```
Returned from new/alloc/copy/mutableCopy/Create?
  YES → NS::TransferPtr(ptr)          // you already own it
  NO  → need AutoreleasePool in scope
        Want to keep beyond pool scope?
          YES → NS::RetainPtr(ptr)     // retains; survives pool drain

# Metal 4 Resources

## Overview

Metal 4 requires explicit residency management via `MTLResidencySet`, removes managed storage mode, and uses GPU addresses for resource binding. Command buffers do NOT retain resources — you must manage lifetimes. All Apple Silicon uses unified memory (CPU and GPU share the same physical memory pool) and Tile-Based Deferred Rendering (TBDR).

## Ownership

**Owns:**
- Buffer / texture / heap creation, storage-mode selection
- General `MTLResidencySet` semantics — kernel/userspace, attachment levels, validation rules, deferred destruction
- Heap allocation and aliasing patterns
- Purgeable state, texture view pools, view reinterpretation
- GPU lossless compression rules

# Metal 4 Synchronization

## Overview

Metal 4 requires fully explicit synchronization — there is no driver hazard tracking. All resource dependencies between encoders and passes must be expressed through barriers, fences, or events. This is typically the longest phase of a port — expect barrier and synchronization work to account for a significant portion of total porting effort.

The key insight: **all source API synchronization models reduce to Metal 4 primitives** — producer barriers (on outgoing encoder), consumer barriers (on incoming encoder), intra-pass barriers (within an encoder), fences (between specific encoders on the same queue), and events (cross-queue or CPU-GPU). The source API determines *when* synchronization is needed. Metal 4's primitives determine *how* to express it.

## References

**Read the relevant Metal 4 SDK header before writing synchronization code** — the headers are the source of truth for property names, types, and method signatures.

- Metal 4 SDK headers - `$(xcrun --show-sdk-path)/System/Library/Frameworks/Metal.framework/Headers/` — focus on `MTL4CommandEncoder.h` (barriers), `MTLEvent.h` (events), `MTLFence.h` (fences)
- Apple documentation - [Resource synchronization](https://developer.apple.com/documentation/metal/resource-synchronization)

You are preparing to port a game or engine to Apple platforms with Metal 4. Before writing any porting code, you must understand the target codebase thoroughly and gather any available reference artifacts.

## Pre-flight Check

Locate the agent's **artifact directory** at `<project-root>/.porting/` (defined in `porting-methodology`). All artifacts live there.

**Check if a discovery report already exists** (`discovery-report.md` in the artifact directory).
- If it exists, tell the user: "A discovery report already exists. Do you want to re-run discovery (this will overwrite the existing report) or use the existing report and proceed to `/porting-plan-goal`?"
- Wait for the user's response before continuing.
- If the user chooses to use the existing report, stop here.

**Announce:** "Running codebase discovery for Apple platform porting."

## Before You Begin — Load Skills

You are entering Phase 2: Execute for the current milestone. The user has reviewed and approved the preparation summary from `/porting-start-milestone`.

Locate the agent's **artifact directory** at `<project-root>/.porting/` (defined in `porting-methodology`). All artifacts live there. Load the `porting-methodology` skill if not already in context.

## Begin Implementation

Implement the approved work items from the preparation summary. Follow the Phase 2 conventions from the methodology (matching engine patterns, leveraging existing systems, consulting loaded skills, stub/hardcoded conventions, building often, assertions).

The approved prep summary is the **scope contract**. Focus exclusively on the listed work items.

If you discover unexpected work that wasn't identified during preparation, **stop and tell the user** before implementing it. Present what you found and collaborate on the right response:

- Does it fit naturally into an existing future milestone? → Stub it with the appropriate `STUB(Mx)` marker.
- Does the goal's milestone plan need new milestones to accommodate it? → Propose specific additions to the plan.

You are entering Phase 4: Handoff for the current milestone.

Locate the agent's **artifact directory** at `<project-root>/.porting/` (defined in `porting-methodology`). All artifacts live there. Load the `porting-methodology` skill if not already in context.

## Step 1: Prompt Commit

Ask the user to commit any uncommitted work before proceeding:

> "Before writing the handoff, please commit any uncommitted changes."

Wait for the user to confirm the commit is done.

## Step 2: Update porting-memory.md

# Platform Porting Methodology

## Overview

This skill defines **how** to approach an Apple platform port — the workflow, phases, conventions, and discipline. Domain-specific knowledge (Metal 4 API patterns, platform skills, anti-patterns, debugging strategies) lives in companion skills loaded alongside this one.

The methodology operates at two levels:
- **Porting goal** — a persistent objective spanning multiple sessions (e.g., "get the simplest sample running on Metal 4")
- **Milestone** — a bounded unit of work within a goal, completed in one session

**Core principle: collaboration.** This is a collaborative process between the agent and the user. The agent should surface decisions, trade-offs, and uncertainties to the user rather than making unilateral choices. When in doubt — ask. When deviating from convention — discuss. When stuck — escalate. The user has context the agent doesn't (reference implementations, project constraints, team preferences).

## References

You are defining a porting goal for the Apple platform porting agent.

## Pre-flight Check

Locate the agent's **artifact directory** at `<project-root>/.porting/` (defined in `porting-methodology`). All artifacts live there.

1. **Check if a discovery report exists** (`discovery-report.md` in the artifact directory). If it does not exist, tell the user: "No discovery report found. Run `/porting-discover` first." and stop.

2. **Check if a previous goal document exists** (`goal-*.md` in the artifact directory). If one exists and is not marked complete, tell the user: "An active porting goal already exists: [goal name]. Complete it before planning a new one, or confirm you want to replace it." Wait for the user's response.

**Announce:** "Planning next porting goal."

## Load Skills

You are entering Phase 1: Prepare for the current milestone.

## Pre-flight Check

Locate the agent's **artifact directory** at `<project-root>/.porting/` (defined in `porting-methodology`). All artifacts live there. Load the `porting-methodology` skill if not already in context.

1. **Check for a goal document** (`goal-*.md` in the artifact directory). If none exists, tell the user: "No goal document found. Run `/porting-plan-goal` first." and stop.

2. **Read the goal document.** Find the first milestone with status "Not started" or "In progress":
   - If one is "In progress", resume it.
   - If one is "Not started", start it — update its status to "In progress" in the goal document.
   - If all milestones are complete, tell the user: "All milestones for this goal are complete. Run `/porting-plan-goal` to define the next porting goal." and stop.

3. **Read porting-memory.md** — review the watch list and feature status. Note any items relevant to this milestone.

You are checking the current porting status. This is read-only — do NOT continue any work or start any implementation.

## Gather Status

Locate the agent's **artifact directory** at `<project-root>/.porting/` (defined in `porting-methodology`). All artifacts live there. Load the `porting-methodology` skill if not already in context.

Read the following files and report their contents:

1. **Goal document** (`goal-*.md` in the artifact directory) — if it exists, show the milestone tracker with current status of each milestone.

2. **porting-memory.md** — if it exists, show the watch list and feature status table.

3. **Latest handoff note** (`porting-handoff-*.md` in the artifact directory) — if one exists, show what was done, what's deferred, and known issues.

You are entering Phase 3: Validate for the current milestone. The milestone's work is functionally complete. Mid-milestone commits for stable checkpoints are fine, but this full validation pass should happen before closing out the milestone.

## Reload Skills

Validation often runs in a fresh session after the execute phase. Skills loaded during execution are no longer in context. **You MUST reload the skills you need before proceeding:**

1. **Always reload** `porting-methodology` (for anti-patterns checklist and debugging escalation).
2. **For rendering milestones**, reload `using-gpucapture` and `using-gpudebug` — you will need these for GPU frame capture and ground truth comparison. Do NOT attempt GPU capture workflows from memory.
3. **Reload domain skills** for this milestone. Read the goal document's milestone plan to find which skills are associated with the current milestone. Also check porting-memory.md for any additional context. You need these for the skill-driven code review (checklist item 7).
4. **If visual bugs are found**, reload `debugging-rendering-issues` for structured diagnostic workflows.

## Validation Checklist

Work through each item and report findings:

# Metal 4 Presentation & Frame Pacing

## Overview

Metal 4 splits the cmd-buffer-level `[cmd presentDrawable:]` into a queue-level sequence: `[queue waitForDrawable:]` before commit, `[queue signalDrawable:]` after commit, and `[drawable present]`. The conceptual flow — acquire drawable, encode, present, submit — is unchanged; only the API surface is more explicit. `MTL4CommandBuffer` does not expose `presentDrawable:`, so any code path using `MTL4CommandQueue` must use the queue-level form. Code paths still using the Metal 3 `MTLCommandQueue` keep `[cmd presentDrawable:]` and remain correct in a Metal 4 app.

## Ownership

**Owns:**
- Drawable lifecycle (`nextDrawable`, `waitForDrawable`, `signalDrawable`, `present`)
- `CAMetalLayer` configuration and properties
- Vsync semantics (`displaySyncEnabled`, `presentAfterMinimumDuration:`)
- `CAMetalDisplayLink` API basics; variable refresh rate detection
- HDR/EDR layer configuration; pixel format ↔ colorspace pairing

# Game Window Port

A guided workflow for wiring up the macOS window layer when porting a Windows game to Metal.

## Scope and Companion Skills

This skill covers windowing concerns: the AppKit surface, the view that owns the `CAMetalLayer` and the render-tick source, lifecycle, notifications, and input.
The boundary with the engine is the `Engine*` callback API described in Phase 6.
Past that boundary, presentation and rendering live in companion skills:

| Question is about | Skill |
|---|---|
| Drawable acquisition, presentation flow (`waitForDrawable`, `signalDrawable`, `present`), layer property semantics (`pixelFormat`, `colorspace`, `drawableSize`, `maximumDrawableCount`, `displaySyncEnabled`, EDR opt-in), Metal 3 vs Metal 4 | `presenting-metal-drawables` |
| D3D12/Vulkan API mapping to Metal 4 (device, queue, command buffer, encoder, PSO, shader compile, swap chain) | `translating-to-metal4-api` |

# Metal 4 API Reference

## Overview

Metal 4 is a ground-up API redesign — not a minor version bump. It introduces the `MTL4` prefix (ObjC protocols/classes) / `MTL4::` namespace (metal-cpp). All Metal 4 objects are created from `MTLDevice` / `MTL::Device`. Key removals: blit encoder, parallel render encoder, set*Bytes, managed storage mode, per-encoder resource binding, addCompletedHandler.

Metal 4 requires macOS 26+, iOS 26+. Feature detection:

```objc
// ObjC
if ([device supportsFamily:MTLGPUFamilyMetal4]) { /* Metal 4 path */ }
```
```cpp
// metal-cpp

## Overview

A comprehensive guide to Apple's GameController framework - covering Swift/ObjC APIs, platform considerations, and porting from Windows.

Use the GameController framework when writing code for Apple platforms that interacts with game controllers.  The `GCController` API presents a common abstraction over gamepad and joystick input devices.  The GameController framework supports popular game controllers including the Sony DualSHOCK 4, Sony DualSense, Nintendo Switch Pro, and Microsoft Xbox controllers.  Advanced features available on these controllers - IMU, Rumble, and Adaptive Triggers (force feedback) - are supported.  

When porting Windows code that calls RawInput (e.g, `RegisterRawInputDevices`) or HID (`HidP_*`) APIs to interfce with Sony, Nintendo, and other non-Xbox controllers, AVOID writing equivalent code that uses IOKit and `IOHIDManager` APIs.  PREFER to re-write the code to use the abstractions provided by the `GCController` API instead.


---

## Framework Concepts

* **Device-centric model** - Each `GCController` represents a connected gamepad or joystick device.  You obtain the currently connected devices from `GCController.controllers`, listen for new connections (`GCControllerDidConnectNotification`) and disconnections (`GCControllerDidDisconnectNotification`), and monitor for input from the one(s) you are interested in.

# Using gpucapture — Metal GPU Trace Capture

## Overview

`gpucapture` is a CLI tool for capturing Metal GPU traces from running processes or launched executables. It requires no source code changes — attach to a process by PID, choose what to capture (frames, command buffers, custom scopes), and produce a `.gputrace` file for analysis.

## Step 1: Check availability

```sh
which gpucapture
```

`gpucapture` ships with macOS 27 and later. If not found, the user is on an older macOS version — suggest alternatives based on their situation:

# Using gpudebug — Metal GPU Trace Debugger

## Overview

`gpudebug` is a CLI debugger for Metal GPU traces. It opens a `.gputrace` file and provides commands for navigating the captured API call hierarchy, inspecting resources and pipeline state, fetching textures and buffers to disk, and browsing shader source code.

## Step 1: Check availability

```sh
which gpudebug
```

`gpudebug` ships with macOS 27 and later. If not found, tell the user it is not available and suggest opening the `.gputrace` in Xcode's Metal Debugger instead: [Analyzing your Metal workload](https://developer.apple.com/documentation/Xcode/Analyzing-your-Metal-workload)

# Using Metal Validation

## Overview

Metal provides two validation modes, both configured via environment variables read at `MTLDevice` creation:

- **API validation** (`MTL_DEBUG_LAYER`) — CPU-side. Cheap; enable broadly during development.
- **Shader validation** (`MTL_SHADER_VALIDATION`) — GPU-side. Potentially significant perf cost; enable selectively or per-pipeline.

Env vars must be set **before** any `MTLDevice` is created. Changes after device creation have no effect.

## Step 1: Read the man page

The man page is the authoritative reference for all env vars, their valid mode values, and pipeline scoping syntax. Read it before configuring anything beyond the quick-start:

# Adding MetalFX Frame Interpolation to a Metal App

This skill adds MetalFX frame interpolation to a Metal app that already has a MetalFX temporal scaler integrated. The interpolator generates intermediate frames between real rendered frames, effectively doubling the display framerate (e.g., 60fps render → 120fps display).

Two API variants exist:
- **Metal 3** — `MTLFXFrameInterpolator` (macOS 15.0+ / iOS 18.0+)
- **Metal 4** — `MTL4FXFrameInterpolator` (macOS 26.0+ / iOS 26.0+), which inherits from the shared `MTLFXFrameInterpolatorBase` protocol

All per-frame properties are identical across both APIs (they live on `MTLFXFrameInterpolatorBase`). The differences are the creation call, the command buffer type used for encoding, and the support check — see the notes below.

**Prerequisite:** The app must already have MetalFX temporal upscaling working. If not, use the `using-metalfx-temporal-upscaler` skill first.

## Abstraction Strategy

# Adding MetalFX Temporal Upscaling to a Metal App

This skill walks through integrating `MTLFXTemporalScaler` into an existing Metal rendering application. The scaler renders at a lower resolution and uses motion vectors, depth, and temporal jitter to produce a high-quality upscaled output.

## Prerequisites

- macOS 13.0+ / iOS 16.0+
- Apple Silicon recommended
- `@import MetalFX;` (auto-links with `CLANG_ENABLE_MODULES = YES`)
- The app must already render a 3D scene with a projection matrix and model-view-projection transforms

## Overview of Changes

The integration touches 3 areas: