Individual skill

k8s-prometheus-troubleshooting

Use when Prometheus on Kubernetes does not collect or alert as expected: a ServiceMonitor or PodMonitor whose targets never appear, targets that are down ("server returned HTTP status 404 Not Found", "connection refused", "context deadline exceeded"), a PrometheusRule that does not load or alerts that do not fire or reach Alertmanager, a Prometheus or Alertmanager custom resource that is not Available, "InvalidConfiguration" events from the Prometheus Operator, or Prometheus pods that are OOMKilled or out of disk. Covers the Prometheus Operator (kube-prometheus, kube-prometheus-stack). Investigates with kubectl get, describe, events, logs and Prometheus' HTTP API through the API server, and proposes fixes. Reads first and asks before changing anything.

skills/k8s-prometheus-troubleshooting/SKILL.md

Read pinned source

Source commit 6919c56810a3995714beecedaa8b516db82f41bc

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
No
Repository scan complete
No
Skill scan complete
No
Observed behavior
Not checked yet
Last checked
Nov 5, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
36
Repository last updated
Nov 6, 2026
License
Apache-2.0

Observed agent integration: claude-code

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt

# Prometheus troubleshooting

Find out why Prometheus does not scrape a target, does not load a rule, or does not
alert, and propose a fix. **Read first, ask before changing.** Run the investigation
commands freely. Run a command under an "Ask first" line only after the user approves
that exact command; if your credentials do not allow it, give it to the user to run.
Never run a "Proposed fix"; hand it to the user.

With the **Prometheus Operator**, a `Prometheus` custom resource selects
**ServiceMonitors**, **PodMonitors**, **Probes**, **ScrapeConfigs** and
**PrometheusRules** by label and namespace, and the operator turns them into
Prometheus' configuration. A ServiceMonitor selects Services by label and names one of
their ports. Something can go wrong at each step: the selection, the Service, the port,
the scrape itself.