Snowflake: Lessons Learned Over the Years

Snowflake has become the backbone of a lot of the data platforms I've worked on, and along the way I've picked up a few habits that make a real difference between a warehouse that scales gracefully and one that quietly burns through credits. Here they are.


Warehouses are not one-size-fits-all. It's tempting to run everything, ETL loads, BI dashboards, ad hoc analyst queries, through a single virtual warehouse. Don't. Separate warehouses by workload so a heavy transformation job doesn't queue behind someone's Tableau refresh, and so you can size and auto-suspend each one independently. A small warehouse for BI traffic and a larger one for batch loads will almost always cost less than one oversized warehouse trying to do both.


Auto-suspend aggressively, auto-resume freely. Snowflake bills by the second, so there's rarely a good reason to leave a warehouse running idle. Set auto-suspend to a minute or two for most workloads. The resume time is fast enough that users barely notice, and the credit savings add up quickly across dozens of warehouses.


Cluster keys are not a default setting. Clustering can dramatically speed up pruning on very large tables, but it also costs credits to maintain. Only add a clustering key when a table is genuinely large and queried with predictable filter patterns, like a date range. Adding clustering to every table "just in case" is a common way to quietly inflate your monthly bill.


Use resource monitors before someone else asks you to. Resource monitors let you cap credit usage at the account or warehouse level and get notified before you blow through budget. Set them up early rather than after finance asks why last month's Snowflake invoice tripled.


Zero-copy cloning is underused. Need a full copy of production data for testing or a one-off analysis? Clone it. Snowflake's zero-copy clone creates a full logical copy instantly without duplicating storage, and you only pay for the data that changes afterward. It's one of the most practical features for setting up safe dev and test environments.


Time Travel is a safety net, not a backup strategy. Time Travel lets you query or restore data as it existed at a past point in time, which is great for recovering from an accidental delete or bad merge. But it has a retention limit tied to your edition, so don't rely on it as your only recovery mechanism for anything critical, pair it with proper backups or Fail-safe awareness.


Watch your micro-partitions, not just your row counts. A table with a small number of rows can still perform poorly if it's been updated in a way that creates a lot of small, poorly pruned micro-partitions. Periodically check clustering depth on your largest tables rather than assuming row count alone tells you how healthy a table is.


None of these require exotic tuning, just some intentional defaults. Getting them right early on saves both compute cost and a lot of firefighting later. Happy querying :)


Comments

Popular posts from this blog

How to fetch data from SnowFlake using Excel VBA?

How to clear Tableau Desktop Qualified Associate Exam (TDQA 10).

All you need to know about the Google Data Studio