Individual skill
databricks-streaming-guardian
Guard production Databricks data pipelines — Delta Lake, Liquid Clustering, Structured Streaming, Auto Loader, and DLT — against the twelve foot-guns that fire at scale: OPTIMIZE/auto-compaction conflicts, Liquid-Clustering merge conflicts, VACUUM breaking a streaming checkpoint, RocksDB OOM, Auto Loader schema-evolution stops, and DLT refresh data loss. Includes a PreToolUse hook that blocks DROP/CREATE-OR-REPLACE/VACUUM on a table with active streaming consumers. Use when a Delta MERGE/OPTIMIZE fails with a concurrency exception, a stream breaks after VACUUM or a table replace, an Auto Loader stream stops on a new column, a DLT refresh drops data, or before running a destructive op on a streamed-from table. Trigger with "ConcurrentAppendException", "ConcurrentDeleteDeleteException", "DELTA_FILE_NOT_FOUND", "streaming checkpoint broke", "vacuum broke my stream", "autoloader UnknownFieldException", "dlt full refresh".
skills/.curated/databricks-streaming-guardian/SKILL.md
Source commit 0500b275e963c027f6b410e8c49c0446945c574e
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- No
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Not checked yet
- Last checked
- Nov 12, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 2,917
- Repository last updated
- Nov 12, 2026
- License
- MIT
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- ai Repository topic
- automation Repository topic
- claude-code Repository topic
- devops Repository topic
- mcp Repository topic
- ai-agents Repository topic
- anthropic Repository topic
- claude-code-plugins Repository topic
- developer-tools Repository topic
- llm Repository topic
- saas Declared skill tag
- databricks Declared skill tag
- streaming Declared skill tag
- delta Declared skill tag
- data-ops Declared skill tag
- engineering Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 14,292 bytesSKILL.mdcore · 14,292 bytesSKILL.mdcore · 14,292 bytesSKILL.mdcore · 14,292 bytesSKILL.mdcore · 14,292 bytesSKILL.mdcore · 14,292 bytesagents/merge-rewriter.mddoc · 3,520 bytesdocs/ADR.mddoc · 12,613 bytesdocs/ONE-PAGER.mddoc · 5,417 bytesdocs/PRD.mddoc · 13,229 byteseval-spec.yamlasset · 8,331 byteshooks/streaming-guard-hook.pyruns · 8,093 bytesreferences/autoloader-schema-evolution.mddoc · 13,679 bytesreferences/checkpoint-recovery.mddoc · 16,515 bytesreferences/concurrency-conflicts.mddoc · 15,311 bytesreferences/dlt-rebuild-safety.mddoc · 13,662 bytesreferences/rocksdb-state-store-tuning.mddoc · 15,091 bytesscripts/pre-optimize-check.shruns · 3,391 bytesscripts/recover-streaming-source.pyruns · 11,183 bytes
Read excerpt
# Databricks Streaming Guardian
The data-ops spine of the pack. Delta Lake, Liquid Clustering, Structured
Streaming, and DLT each ship a different set of foot-guns that fire most visibly
when production data flows through them at scale — and most of them are documented
platform *decisions* that surprise engineers, not bugs. This skill's job is
friction at trigger time (a hook that blocks the genuinely-irreversible op) plus
deterministic recovery when something already broke.
## Overview
Twelve foot-guns, grouped by the surface that triggers them. Eleven are owned
outright (D01–D10, D12); the twelfth — D11, DLT rebuild cost — is shared with
`databricks-cost-leak-hunter`: this skill checks the rebuild cost as part of