Skill

Find stale warehouse tables to cut storage costs

Finds stale, removable tables to cut warehouse storage costs by delegating classification and safety scoring to one Monte Carlo tool.

Works with monte carlosnowflakebigqueryredshiftdatabricks

80
Spark score
out of 100
Updated 22 days ago
Version 1.14.0

Add to Favorites

Why it matters

Identify unused, unread, and zombie tables in your data warehouse that can be safely removed to reduce storage costs, with automatic classification by waste pattern, table category, and safety tier.

Outcomes

What it gets done

01

Classify tables into waste patterns: unread, write-only, dead-end, static, zombie, or other stale

02

Categorize tables as temporary/staging, archive/snapshot, production, or uncategorized for prioritized review

03

Score removal safety by checking criticality, consumers, monitors, and importance flags

04

Check downstream lineage to verify whether active tables or BI dashboards depend on a candidate

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/ag-monte-carlo-storage-cost-analysis | bash

Overview

Monte Carlo Storage Cost Analysis Skill

This skill finds stale, removable tables to cut warehouse storage costs by delegating classification and safety scoring to one Monte Carlo tool, presenting its formatted output verbatim across 6 waste patterns and 4 table categories, with lineage-checked removal guidance. Use it for storage cost, waste, or table-cleanup questions on Snowflake, BigQuery, Redshift, or Databricks. Not for general table exploration, monitor management, data quality incidents, or performance/query-cost questions.

What it does

A Monte Carlo skill that analyzes a data warehouse for stale tables that can be removed to reduce storage costs, delegating classification, safety scoring, and formatting entirely to the analyze_storage_costs MCP tool rather than reasoning about it itself. That tool fetches candidates, classifies them into six waste patterns (Unread, Write-only, Dead-end, Static waste, Zombie, Other stale) and four table categories (Temporary/Staging, Archive/Snapshot, Production, Other), computes safety tiers, and returns pre-formatted output as two regions: a PRESENT_AS_IS block (condensed summary, Top-N table, drill-down prompt) that the skill copies verbatim with no reformatting, and a CATEGORY_DETAILS block with per-category tables that stay hidden until the user asks about a specific category. Critically, once analyze_storage_costs succeeds, the skill calls no other tool to cross-check or search - the result is treated as the final answer, and markdown-linked MCONs in the tables must be preserved exactly, never flattened to bare strings. Category drill-down requests are matched by keyword (temporary/staging/tmp/stg, archive/snapshot/backup/old, uncategorized/other/unknown, production/prod/critical/important) and answered from the already-returned data without re-invoking the tool. A separate lineage-check path uses get_asset_lineage with downstream direction when the user asks whether a specific table is safe to remove: no tracked relationships suggests the consumers are likely BI dashboards rather than other tables (still worth verifying with dashboard owners), stale downstream tables support removing both, and active downstream tables get flagged as risky. Each result row's Usage & Risk cell combines an activity value (no reads; days-since-read plus zero reads; or recent reads with total count and distinct users) with risk flags appended in fixed order when they fire - criticality level, consumer count (including non-table BI assets that the lineage tool itself can't see, which is called out explicitly as a source of undercounting), a Databricks-computed importance-score threshold, and active monitoring status.

When to use - and when NOT to

Use it when someone asks about storage costs, waste, or cleanup opportunities, wants to find unused or stale tables, or asks which tables can be dropped. Do NOT use it for querying or exploring table contents generally, creating or modifying monitors (the monitoring-advisor skill), investigating data quality incidents (the prevent skill), or pipeline performance and query cost questions (the performance-diagnosis skill).

Inputs and outputs

Input is a warehouse (named, or auto-selected/prompted-for if only one or several supported warehouses exist) among Snowflake, BigQuery, Redshift, or Databricks. Output is a verbatim-presented cost-waste summary and Top-N table, expandable per-category tables on request, and lineage-informed removal guidance - always a recommendation, never an executed DROP TABLE or other destructive action.

Integrations

Runs through Monte Carlo's bundled MCP server, primarily the single analyze_storage_costs tool for the full analysis pipeline and get_asset_lineage for follow-up lineage checks only - both routed through fully-qualified plugin tool names, never a separately-configured monte-carlo-mcp server.

Who it's for

Data platform teams looking to cut warehouse storage spend who want a pre-scored, safety-tiered list of candidate tables to drop - covering temporary, archived, and zombie tables - without manually cross-referencing read activity, lineage, and monitoring status themselves.

Source README

Monte Carlo Storage Cost Analysis Skill

This skill analyzes a data warehouse for stale tables that can be removed to reduce storage costs. It delegates classification, safety scoring, and formatting to the analyze_storage_costs MCP tool, then presents the pre-formatted result verbatim and handles follow-up questions (category drill-downs, lineage checks).

Monte Carlo tool routing (required): Always call Monte Carlo MCP tools through this plugin's
bundled server, whose fully-qualified tool names are
mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__<tool> (e.g.
mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__get_alerts). Bare tool names used in this skill
(get_alerts, search, get_table, …) refer to that bundled server. If the session also has a
separately-configured monte-carlo-mcp server, do not route to it - it may point at a
different endpoint or credentials.

Reference file (use the Read tool to access it):

  • Output contract and category keywords: references/output-structure.md

When to activate this skill

Activate when the user:

  • Asks about storage costs, waste, or cleanup opportunities
  • Wants to find unused, unread, or stale tables
  • Asks "which tables can I drop?" or "what's costing us money?"
  • Mentions storage optimization, cost reduction, or warehouse cleanup
  • Wants to identify zombie tables, dead-end pipelines, or temporary/archive tables

When NOT to activate this skill

Do not activate when the user is:

  • Just querying data or exploring table contents
  • Creating or modifying monitors (use the monitoring-advisor skill)
  • Investigating data quality incidents (use the prevent skill)
  • Looking at pipeline performance or query cost (use the performance-diagnosis skill)

Prerequisites

The following MCP tools must be available (connect to Monte Carlo's MCP server):

  • analyze_storage_costs -- runs the full analysis pipeline and returns pre-formatted output
  • get_asset_lineage -- used only for follow-up lineage checks

The analyze_storage_costs tool supports Snowflake, BigQuery, Redshift, and Databricks warehouses only. Other warehouse types are out of scope.

Workflow

Important: These steps are internal instructions for you. Do NOT expose step numbers, step names, or the procedural structure to the user. Just act naturally.

Step 1: Identify the warehouse

You need a warehouse to proceed.

  • If the user specified a warehouse (by name or UUID), use it.
  • If not: call analyze_storage_costs with no warehouse_id. The tool will either auto-pick when only one supported warehouse exists, or return a list of supported warehouses - let the user choose one, then call the tool again with the chosen warehouse_id.

Step 2: Run the analysis

Call analyze_storage_costs with:

  • warehouse_id: the warehouse UUID

The tool fetches candidates, classifies them into waste patterns (Unread, Write-only, Dead-end, Static waste, Zombie, Other stale) and table categories (Temporary, Archive/Snapshot, Production, Other), computes safety tiers, and returns a formatted analysis.

  • If the tool returns an error, report it to the user and stop.
  • If no candidates are found, tell the user and stop.

Step 3: Present the initial summary

The tool output contains two regions:

  1. A `` block with a condensed summary, a Top-N table, and a drill-down prompt.
  2. A block with per-category tables wrapped in markers. Do NOT present these yet.

Present ONLY the `` block - copy it verbatim, preserving every column, row, and value. Add a brief intro sentence if needed, then paste the block unchanged. The user will see the summary and top tables, then choose a category to drill into.

CRITICAL - do NOT call any other tool after analyze_storage_costs succeeds. No search, no get_table, no troubleshooting agents, no cross-checks. The analysis result IS the final answer; your only remaining job is to present the `` block verbatim.

CRITICAL - preserve markdown-linked MCONs verbatim. The pre-formatted tables already contain properly linked MCONs (e.g., [`db:schema.table`](https://getmontecarlo.com/assets/MCON++...)). Never output bare MCON strings as plain text.

Step 4: Handle follow-up requests

Category drill-downs. When the user asks about a specific category ("show me temporary tables", "what about production?", "tell me more about archive"):

  1. Find the matching `` section in the analyze_storage_costs result already in the conversation. Do NOT re-invoke analyze_storage_costs - the data is already there.
  2. Present that section's content verbatim - every column, row, and value.
  3. After presenting, remind the user of remaining categories they haven't explored yet.

Category keywords (see references/output-structure.md for the full list):

  • "temporary", "staging", "tmp", "stg" → CATEGORY:temporary
  • "archive", "snapshot", "backup", "old" → CATEGORY:archive_snapshot
  • "uncategorized", "other", "unknown" → CATEGORY:other
  • "production", "prod", "critical", "important" → CATEGORY:production

If the user says "show me everything" or "all categories", present all category sections in order: temporary → archive → uncategorized → production.

Lineage checks. When the user asks what consumes a specific table ("check lineage for X", "is it safe to remove Y?", "what depends on this table?"):

  1. Call get_asset_lineage with mcons: [<table mcon>] and direction: "DOWNSTREAM".
  2. If has_relationships: false → the table's consumers are likely BI dashboards or tools (not other tables). Mention this - it may still be safe to remove, but the user should verify with dashboard owners.
  3. If downstream tables exist AND are also stale → recommend removing both.
  4. If downstream tables are active → flag as risky, do NOT recommend removal.

Note: The N consumers flag in the Usage & Risk column counts ALL consumers, including BI dashboards (Looker, Tableau, Power BI) and other non-table assets. The lineage tool only returns table-to-table edges, so lineage results may show fewer consumers than the count. When that happens, explain the gap to the user.

Reading the Usage & Risk column

Each row's final Usage & Risk cell combines read-side activity with risk flags. Format:

{activity}                          # no flags fire
{activity}; {flag1, flag2, ...}     # one or more flags fire

Activity values (always present):

  • No reads -- no recorded reads
  • 180d · 0 reads -- last read N days ago, zero total reads
  • 2d · 580 reads / 14 users -- recent reads, total reads and distinct reading users

A low days since read is only meaningful when paired with the read count - a single backup job or security scanner can make a cold table look "1d". Always weigh staleness against reads + users.

Risk flags (appended after ; in this fixed order when any fire):

  • high criticality / medium criticality -- pre-computed criticality
  • N consumers -- has active consumers (tables, views, or BI dashboards); verify before removing
  • high importance score -- is_important is a thresholded importance_score ≥ 0.6 computed upstream in Databricks, not a user-applied tag
  • has monitors -- actively monitored by Monte Carlo

Table categories

Tables are automatically classified for prioritized review:

  • Temporary/Staging -- Short-lived ETL/test tables (safest to drop)
  • Archive/Snapshot -- Historical copies, date-suffixed tables (verify retention policies)
  • Production -- Monitored, critical, or lineage-important tables (highest risk)
  • Other -- No strong signal either way (needs manual review)

Scope limitations

  • Storage costs only -- not compute, query optimization, or billing
  • One warehouse per analysis
  • Snowflake, BigQuery, Redshift, and Databricks only
  • Recommendations only -- never execute DROP TABLE or destructive actions

Limitations

  • Use this skill only when the task clearly matches its upstream source and local project context.
  • Verify commands, generated code, dependencies, credentials, and external service behavior before applying changes.
  • Do not treat examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.

FAQ

Common questions

Discussion

Questions & comments · 0

Sign In Sign in to leave a comment.