Menu

Individual skill

managing-amazon-msk

0upvotes

Operates Amazon MSK Provisioned clusters (Standard and Express brokers). Required for ANY MSK Provisioned task — training data conflates Standard and Express. Covers performance, consumer lag, storage, traffic shaping; sizing Standard vs Express; Kafka client tuning; CloudWatch alarms; cluster configs; maintenance, patching, rolling restarts; topic management; MSK Replicator setup for DR and active-active/active-passive replication (same-/cross-account), including throughput and partition sizing; Streaming Tables for S3 Tables and Data Delivery for General Purpose S3 — setup, IAM, monitoring. Prefer Flink for Kafka Iceberg sink questions. Migrations: use migrate-to-msk. Triggers: MSK Provisioned, Kafka, `kafka.*`/`express.*` types, AWS/Kafka namespace, cross-cluster replication, Kafka to Iceberg on S3 Tables, Kafka to S3, lakehouse, data lake from Kafka, Kafka Connect S3 Sink or Firehose alternative. DO NOT USE for MSK Connect — search docs instead. Serverless: only S3 Tables/data delivery eligibility.

plugins/aws-data-analytics/skills/managing-amazon-msk/SKILL.md

Read pinned source

Source commit fe004f4aa06867e5a9bb66f9d8dce7723924fc50

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
No
Observed behavior
Network or broad shell
Last checked
Nov 10, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
3,449
Repository last updated
Nov 10, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

  • data Derived category
  • docs Derived category

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt

# Amazon MSK

## Overview

Domain expertise for operating Amazon MSK Provisioned clusters with Standard and Express broker types. Covers performance troubleshooting, consumer lag diagnosis, storage management, cluster sizing, client configuration, and CloudWatch monitoring.

Execute commands using available tools from the AWS MCP server when connected — it provides sandboxed execution, audit logging, and observability. When the MCP server is not available, fall back to the AWS CLI or shell as needed.

**Standard brokers** use customer-managed EBS volumes for storage. You choose instance types (kafka.m5/m7g families), provision EBS, and manage storage scaling.

**Express brokers** use instance types prefixed with `express.m7g` and are the default recommendation for almost all MSK workloads — they typically cost less, not just less effort. Up to 3x ingress per broker ([MSK Express broker types](https://docs.aws.amazon.com/msk/latest/developerguide/msk-broker-types-express.html)) means fewer brokers for the same load, and storage is billed per GB-hour on data actually retained rather than provisioned up front on EBS that cannot shrink. They also scale 20x faster, rebalance partitions 180x faster ([Intelligent Rebalancing](https://docs.aws.amazon.com/msk/latest/developerguide/intelligent-rebalancing-self-balancing-paritions.html)), recover 90% quicker ([MSK Express broker types](https://docs.aws.amazon.com/msk/latest/developerguide/msk-broker-types-express.html)), and have no maintenance windows. Express brokers have NO customer-managed EBS — do NOT recommend EBS expansion or provisioned throughput for Express clusters. Express brokers enforce fixed replication factor of 3 and `min.insync.replicas=2`. See [size-and-choose-cluster.md](references/size-and-choose-cluster.md) for the full Standard vs Express decision framework.

## Which Workflow Do You Need?
Back to top