Good Data. Better Judgment.

There is a phrase I hear constantly in product circles: data-driven.

It is well intentioned and sounds right. It sounds rigorous and defensible. It signals that decisions are grounded in evidence rather than gut feel or politics. There is appeal in the statement and sends a signal to use data as opposed to gut or “just winging it”.

I do want to say though that I have come to believe that data-driven, as a philosophy, is incomplete. In the domains I care most about, the ones that shape how people behave, connect, and experience the world, it can quietly lead us somewhere we never intended to go.

The alternative I keep coming back to is simpler: be data-informed.

The difference is not semantic. It comes down to who holds the final judgment, you or the dashboard.

A Problem the Metrics Couldn’t See

I recall a moment a few years back when I inherited a team struggling with poor performance metrics. The data told a clear story where a critical part of the system was failing repeatedly, and that failure was dragging everything down.

The first step was to fix the failing service and improve those obvious failing metrics. We fixed those failures. The metrics improved. Job done, right?

No.

There was a second problem that no metric had surfaced. Years of unreliability had created real distrust between the people building the product and the people depending on it. You could feel it in every meeting. Not anger. Just people going through the motions. This was critical work, work that mattered to the business in a real way, but it had been overlooked for so long that the team had stopped expecting anyone to notice. They kept showing up. They just didn’t expect things to get better. The data had nothing to say about any of that, because nobody had ever thought to measure being forgotten.

So alongside the technical fix, I started spending real time with the business and customer stakeholders. Not in formal reviews. Just present, consistent, listening to what was actually frustrating them. That investment changed the relationship in ways the performance data alone never would have predicted.

Had I only followed the quantitative data, I would have solved half the problem and spent a long time wondering why things still felt broken.

Data tells you when something is broken. It is much quieter when something is simply wrong.

When the Numbers Made the Case and I Said No

Another time, a clear business case landed on my desk. A manual process, a high error rate, a real cost. The math was airtight and the efficiency gains were genuine. Most product leaders looking at those numbers would have greenlit the project without a second thought.

But when I sat down with the people closest to the work, I found something the spreadsheet hadn’t accounted for. This wasn’t just a workflow to them. There was craft in it, identity in it, a creativity that would have been lost had a programmatic/cost-effective approach been taken. The inefficiency we wanted to eliminate was tied up in something people genuinely cared about.

We walked away from the project.

That wasn’t a comfortable call to make. The data had built a strong case. But data can only measure what you decided to measure ahead of time. It cannot tell you what you forgot to ask about. That part only comes from spending real time with real people.

What I Took from a Recent Conversation on Taste

I LOVE Lenny’s Podcast and I truly enjoyed Tony’s book, Build.

A few weeks back, these forces came together on Lenny’s Podcast (highly recommend you check it out!). Tony Fadell talked about taste, judgment, and building in the AI era. One of his arguments stuck with me was that the most consequential product decisions are opinion-based, not data-based.

While I appreciate the intent of the statement I do worry that it may be misread as “ignore the data.” However, that said, I do think that the intent here is aligned with the belief that data informs the decision, judgment owns it.

The best product leaders I know don’t distrust data. They take it seriously, interrogate it, and then bring something to it that no dashboard can provide. Context. Pattern recognition built from experience. The ability to notice what’s missing from the picture. None of that comes from nowhere. It comes from spending enough time close to the work that you start noticing the gap between what the metrics say and what’s actually happening.

I’ve watched this play out in academic settings too. Smart, rigorous people whose research pointed one direction while their instinct pointed another. The right call was to trust the instinct, not because the data was wrong, but because they understood something about the problem the data couldn’t fully hold.

What Data-Informed Actually Means

I want to be clear about something. This isn’t a case against data. I’ve spent years building systems that are deeply rooted in measurement and research. Data is essential. It reveals patterns you can’t see from the inside. It challenges your assumptions. It keeps teams honest.

What I’m pushing back on is the feeling that the data holds all of the truth and context, the idea that if the numbers say something, the decision has already been made for you.

Being data-informed means the data earns its seat at the table. You take it seriously and let it shape your thinking. But you also bring your own read of the situation which means also looking at the qualitative signals, the human context, the things you’ve noticed that don’t have a metric attached to them yet. You weigh all of it and you own the call.

The data doesn’t get the final vote. You do.

Why This Matters Most in Behavioral Systems

Most of my work sits at the intersection of technology and human behavior at scale. In this space, the gap between what data measures and what actually matters to people is about as wide as it gets.

You can track enforcement rates, moderation capability, reporting. It’s much harder to track whether someone felt heard.

You can measure engagement. It’s much harder to measure whether a community feels worth belonging to.

You can confirm a system is functioning. It’s much harder to confirm the experience is right.

None of that makes the unmeasured stuff soft. Often it’s the entire point. Getting to it takes product thinking: asking the question the dashboard isn’t asking, staying curious when the numbers look fine but something still feels off, and being willing to make a call the data alone can’t justify.

That’s the difference between a system that functions and one that’s actually good.

Data-driven gets you a product that works. Data-informed gets you a product that matters.