Data Engineering Fundamentals, Part 5: Data Visualisation Fundamentals

This is the final post in a 5-part series on data engineering fundamentals. Part 4 covered data warehousing with Snowflake, . If you've followed the whole series, you've now gone from raw SQL and Python, through ELT pipelines, into a properly modeled warehouse, and now to the layer where people actually see and use the data.

Why visualization is the last mile that matters most

You can build a technically flawless pipeline and a perfectly modeled warehouse, but if the person making a business decision can't see the answer clearly and quickly, none of that engineering work translates into value. Visualization is where data engineering meets decision-making.

Core concepts that apply across every tool

Dimensions vs measures. Dimensions are the categorical fields you slice and group by, region, product, date. Measures are the numeric fields you aggregate, sales, count, average order value. Every visualization tool is built around this same distinction, understand it once and it transfers everywhere.

Calculated fields. Almost every real dashboard needs some logic beyond what's in the raw data, a profit margin, a year-over-year change, a custom category grouping. Get comfortable writing these directly in your viz tool, not just in the underlying SQL.

Level of Detail (LOD) style calculations. Sometimes you need an aggregation at a different granularity than what's on your chart, average order value per customer, shown alongside individual transaction rows. Tableau calls these LOD expressions, Power BI handles this through DAX context, and Qlik through its Set Analysis. Different syntax, same underlying concept.

Filtering vs context. Understand the difference between a filter that removes rows entirely from a view versus a highlight or set that keeps rows visible but marks them differently. This distinction trips up a lot of beginners building their first interactive dashboards.

Choosing between Tableau, Power BI, and Qlik

Tableau is generally the easiest entry point for building polished, exploratory dashboards quickly, and it has the largest community and hiring pool of the three, which matters if you're building a career around it.

Power BI has deep, native integration with the Microsoft ecosystem, Excel, Azure, Fabric, and is often the default choice inside organizations already standardized on Microsoft tools, with a licensing model that tends to be more budget-friendly at scale.

Qlik's associative model lets users click on any data point and see everything related light up while unrelated data greys out, a genuinely different and powerful way to explore data once you're used to it, though its ecosystem has increasingly shifted toward Qlik Sense rather than the older QlikView.

None of these is universally "better," the right choice depends on your organization's existing tools, your team's skills, and how your users prefer to explore data.

Getting started practically

Pick one tool and build with real (or realistic) data, not a sample dataset that's already clean. Connect it to a properly modeled warehouse if you can, ideally one you built yourself following the star schema concepts from Part 4, so you're practicing the full pipeline end to end, not just the visualization step in isolation.

Bringing the series together

If you've worked through all five parts, SQL, Python, ELT/ETL concepts, Snowflake and data warehousing, and now visualization, you've covered the full path data actually travels in a real organization: extracted, transformed, modeled, and finally understood. That's the same path I've walked over 15 years, and it still holds up as the core skillset that matters, regardless of which specific tools come and go.

One important thing to flag though, everything in this series has been about batch processing, data moving in scheduled chunks, hourly, daily, or on-demand runs. That covers a huge portion of real-world data engineering, but not all of it.

When your data volume grows into the terabytes or beyond, or when a single machine's memory and compute simply can't handle the transformation you need, that's where distributed computing and Apache Spark come in. Spark splits both the data and the processing work across many machines at once, which is what makes it possible to transform genuinely massive datasets in a reasonable amount of time, something a traditional single-node approach just can't do.

And beyond batch entirely, there's streaming, processing data continuously as it arrives, event by event, rather than waiting for the next scheduled run. Think fraud detection, live dashboards, or real-time recommendation engines, use cases where waiting even an hour for the next batch simply isn't good enough.

Both of these, distributed computing with Spark and real-time streaming, are big enough topics that they deserve their own dedicated series rather than a rushed mention here. I'll be covering both in detail in an upcoming series, so if batch processing was your foundation, consider this the bridge to where things get genuinely large-scale and real-time.

If this series was useful, I'd love to hear which part resonated most, feel free to reach out.