Most data teams build pipelines. I build products.

Three questions from my Lalamove interview — on data freshness, metric accuracy, and why the data mart is your most important product. Written for both engineers and the executives who depend on their work.


Data infrastructure fails in three predictable ways: dashboards go stale, metrics report the wrong number, and nobody trusts the data enough to act on it. These are the three questions I was asked — and my answers come back to one idea: the data mart is a product, not plumbing.

01 / Data Freshness
How do you keep data fresh enough that decisions made on it are still based on what's actually happening?
Executive summary

Decisions made on stale data cost more than the engineering to prevent it.

1h
Refresh every hour
Catches bad data before dashboards see it
10%
60-min freshness at 10% of streaming cost
The problem
Your dashboard refreshes once a day. An exec sees yesterday's numbers and makes today's decision. By the time the data is right, the decision is made.
The fix
We update every hour, automatically. Only new data gets added; nothing gets deleted or rewritten. If something breaks, we catch it before it reaches any dashboard.

Stale data occurs when a dashboard shows you a number from hours or days ago but makes it appear current. The risk isn't just inconvenience. It's those decisions that get made on outdated reality.

The fix is incremental updates: instead of rewriting the entire dataset every hour, we only process what's new. Think of it like syncing new emails instead of downloading your entire inbox from scratch every time you open Outlook. Faster, cheaper, and if something goes wrong, easy to reprocess just the broken window.

Insight: Hourly updates catch most data problems before they reach a dashboard at a fraction of the cost of a real-time system. For executive reporting, that's usually the right trade-off.

How we know the data is trustworthy, hour by hour

What we monitor What triggers an alert Why it matters
Data volume Drops more than 20% vs normal Means something upstream broke
Data age More than 90 min since last update Means the pipeline stalled
Data consistency Key fields are missing or duplicated Means reports will be wrong
02 / The wrong number
How do you make sure business metrics roll up correctly across days, weeks, and months — without silently reporting the wrong number?
Executive summary

The wrong aggregation shows leadership a number that's 3× too high. Silently.

What most teams do
Sum daily active drivers → report 840 weekly actives
inflation
What actually happened
340 unique drivers worked that week

Three types of business metrics — and why they need different rules

Metric type Plain-English rule Lalamove example
Can be added up Add across any time period or region Total orders, total revenue, total km driven
Can only be added some ways Add across regions — but NOT across time periods Drivers logged in today (Tuesday's count + Monday's count ≠ this week's count)
Must be recalculated fresh Never add — always recalculate from scratch for the time window you need % of on-time deliveries, average order value, customer satisfaction score

Most BI tools will happily add up any number you give them. The problem is that some numbers, like the count of active drivers or your on-time delivery rate, don't work that way. Add them up across days, and you get a number that's plausible but wrong. The fix is to calculate these metrics fresh for each time window (daily, weekly, monthly) rather than letting the dashboard sum them automatically.

Insight: A driver who works Monday, Wednesday, and Friday counts once in the weekly report but three times if you sum the daily reports. That gap between what the system reports and what actually happened is where trust in data breaks down.

03 / Trust & adoption
"The data mart is the product." What does that mean, and how does it change how you build?
Executive summary

A data mart is only valuable if people trust it enough to use it.

Most data teams build for engineers. I build for the people who open the dashboard at 8am and need to make a decision by 9am.

What it means to treat data as a product

If data is just plumbing… If data is a product…
It works until it breaks, then someone scrambles It has an SLA — if it's late, the right person is already fixing it
Column names are for engineers Column names are readable by anyone
Changes ship whenever they're ready Changes come with a migration window so nothing downstream breaks
Success = it runs Success = people use it and trust it

When I call the data mart the product, I mean: the analyst who opens it at 8 am is my user. Their needs define what I build, not the technical elegance of the pipeline behind it.

In practice, that means three things:

First, it has to be reliable. Not 'usually works' reliable — SLA reliable. If the dashboard is late, someone is already on it before the user notices.

Second, it has to be understandable. A metric called 'completed_order_count' tells a PM something. A metric called 'cnt_co' tells them nothing. Documentation ships with the data, not six months later.

Third, it has to be trustworthy over time. When we change something, we give users time to adapt. No surprise schema changes that break their reports on a Monday morning.

Insight: A mart nobody queries is just expensive storage. Adoption and trust are the real success metrics, not whether the pipeline ran on time.