LearnProduct Judgment

What a Product Decision Actually Is

What separates a decision from a commitment, why "we scoped it" isn't "we validated it", and the four risks any idea has to survive.

Chapter 1 · 4 ideas · 12 checks · 14 min

Start here

It is a Thursday review. Rhea, one of your product managers, is walking the room through a proposal for smart reminders, a feature that nudges users about work they left unfinished. She has mockups. She has an engineering estimate: four people, six weeks. She has a rollout schedule and a slide listing the risks, each with a mitigation next to it.

Nobody in the room has a question. The document answers everything it raises. You approve it.

Six weeks later it ships. It works: no outages, no complaints, the settings page is clean. At the next quarterly review someone asks whether it helped. The answer takes a while to arrive, and when it does it is a version of: engagement is up about two percent, but the marketing campaign ran in the same window, and we didn't measure the before, so, honestly, we don't know.

Nothing went wrong. Everyone was competent. The document was genuinely good.

Here is the uncomfortable part: the thing that failed had already failed on that Thursday, in the room, before a line of code existed. The proposal was thorough about one question and silent about three others, and its thoroughness is exactly what stopped anyone noticing.

So: what was missing at the moment of approval? Not detail; there was plenty of that. Something else.

Core concepts

Each idea is a short read, one picture and one quick check.

Idea 1 of 4

The four risks

Put Rhea's proposal next to a two-page one written in plain prose. Hers runs to twelve pages: architecture, data model, an estimate per engineer, mockups. It feels safer. But nine of its twelve pages answer one question: can the team build it?

Any product idea has to survive four separate risks before it is worth building:

  • Value: will anyone choose this? Not "would it be nice", but will a real person pick it over what they do today, including doing nothing?
  • Usability: can they work out how to use it without being taught?
  • Feasibility: can you build it with the people, data and time you have?
  • Business viability: does it work for your own company? Support load, legal exposure, brand, margin, whether sales can explain it.

The four are independent. Passing one tells you nothing about the others. A feature can be well built and easy to use, and nobody wants it.

Picture it

Smart reminders · where the pages went

  • Architecture, data model · 5 pagesfeasibility
  • Estimate and rollout · 4 pagesfeasibility
  • Mockups, three screens · 3 pagesusability, partly
  • "Users want this" · 1 line (circled)value, assumed
  • Support, legal, pricing · none (circled)viability, missing

Value got one line. Viability, none.

Count where the pages went. Rhea's proposal was rigorous about whether the team could build it, and said one line about whether anyone would choose it.

Check yourself

Each question is one you could ask about the smart reminders proposal. Which risk does it test?

  1. Will users act on a reminder, or swipe it away like every other notification?

  2. Can the reminder service handle every account in a single overnight batch?

  3. Will a new user find the setting that turns reminders off?

  4. How many extra support tickets will reminders create, and who answers them?

  5. Do users who left work unfinished ever come back to it on their own?

  6. Does emailing reminders need consent the sign-up form never asked for?

Go deeper

Why independence matters. A feature can be beautifully engineered, obvious to use, and still nobody wants it (value: fatal). A feature can be wanted, usable and buildable and still sink you because it triples support tickets or attracts customers your pricing can't carry (viability: fatal).

How to use it. When any proposal reaches you, do not evaluate it. First take inventory. Go through the four risks and sort each into addressed, assumed or not mentioned. Then ask one question about the biggest thing in the "assumed" pile: what would we do this week to find out?

How to spot the gap. Count where the pages went. A document that is overwhelmingly about how and how long has answered feasibility and quietly assumed the rest. As Cagan puts it, a proposal like that has not been evaluated; it has been scheduled. Feasibility attracts the effort because it is the one risk a competent team can settle by itself, in a room, by thinking hard.

One thing that has changed. Feasibility risk has genuinely fallen for a lot of software: building really is faster than it was. That does not remove the risk mix, it reweights it. When building is cheap, value becomes the binding constraint, and business viability becomes the quiet killer, because support load, legal exposure and brand damage did not get cheaper just because code generation did. An idea that looks newly attractive because "we can build this in a week now" is usually still failing on the exact risk it failed on before.

Idea 2 of 4

Problems, not solutions

Open any quarterly roadmap, the plan for what a team will work on over the next few months, and you will usually find a list like Smart reminders · Bulk import · Notification preferences · Mobile onboarding v2. Ask of any one item: what is this supposed to change, and by how much? Most roadmaps cannot answer.

A list of features is a list of solutions, each chosen before anyone did the work that would justify choosing it. The document has quietly turned a team that understands the customer into a delivery queue.

The alternative is not "no roadmap". It is handing a team the problem and the outcome to move, and letting the people closest to the customer choose the solution. "Get a new user to their first real result in under ten minutes" is a problem. "Ship onboarding v2 by March" is a queue ticket.

Check yourself

Each line is from a roadmap. Does it hand the team a problem, or a solution?

  1. Ship mobile onboarding v2 by March.

  2. Get new users to their first real result in under ten minutes (today it takes 26).

  3. Add bulk import for spreadsheets.

  4. Halve the share of trial teams who stall after inviting one colleague.

  5. Smart reminders: designs signed off, ships 30 July.

Go deeper

How to use it. For each item, write down the outcome it is meant to move and the evidence that it will. Items that can answer are bets; keep them. Items that cannot are commitments to someone, usually a customer or an executive, and those are fine as long as everyone says so out loud. What is not fine is a commitment wearing a bet's clothes.

How to spot it. Look at the success criteria. If they are shipping dates, you are looking at a feature roadmap, whatever the header says.

One thing that has changed. Generated roadmaps are very good at producing plausible feature lists and very bad at supplying the problem behind them. The format survives the generation; the justification does not. Asking "what outcome does this move?" of an AI-drafted plan is the fastest way to find out whether anyone thought about it before it was formatted.

Idea 3 of 4

Outcome over output

A launch retrospective, the review a team runs after shipping, opens with a slide: fourteen releases, velocity up 30%, zero rollbacks. The room is pleased, and it should be; that is real work. Then someone asks what changed for users, and nobody knows, because nobody measured the before.

Output is what you shipped. Outcome is the change in behaviour that shipping caused. Output is always achievable: a working team will produce some, and it is measurable, immediate and credited to named people. Outcome is noisy, lagging and shared. Under pressure the measurable thing wins, and a team can run for years reporting output without finding out whether any of it worked.

An outcome claim also needs a baseline: the reading taken before the change. Without one, "improved" means nothing.

Check yourself

Each line is from the same quarterly update. Output or outcome?

  1. We shipped fourteen releases this quarter.

  2. Velocity is up 30% on last quarter.

  3. New users reaching a first result in week one went from 41% to 52%.

  4. Onboarding v2 is live for every account.

  5. Teams using templates invite a second colleague two days sooner than teams without.

Check yourself

Back to Rhea's smart reminders. Engagement is up about 2%, a marketing campaign ran in the same weeks, and nobody took a reading before launch. What can the review honestly say?

Go deeper

How to use it. Take every stated accomplishment and translate it into the behaviour it was meant to change, then check whether that behaviour was measured against a baseline. "We shipped onboarding v2" becomes "did more new users reach their first real result, and how would we know?" If the artifact only reports shipping, the review has not happened yet. It has been announced.

How to spot it. A metric with no baseline and no comparison group. Also: any claim of improvement made in a window where something else obviously changed, like Rhea's two percent, sharing a calendar with a marketing campaign.

One thing that has changed. This is where the AI productivity paradox lives. Teams adopting AI tooling frequently ship substantially more while their outcomes stay flat, which is not a mystery: the constraint was never how fast the team could produce. More output at constant outcome is a signal that you are solving the wrong problem faster.

Idea 4 of 4

Decisions and commitments

The three ideas so far meet in one distinction, and it is the reason this chapter comes first.

A commitment names a thing and a date. It can be tracked, reported and delivered. Delivering it proves you delivered it.

A decision names which risk you are taking, what you expect to change, and what would tell you that you were wrong. It can turn out badly while having been made well, and that is the point: a bet you can lose is the only kind that teaches you anything.

"We scoped it" is not "we validated it." Rhea's proposal was an excellent commitment. It was never a decision, and the Thursday room approved it as one.

Picture it

Smart reminders · the same approval, as a decision (example)

  • The bet: people return to work a nudge reminds them of. (highlighted)← the risk we're taking
  • Expect: share back within 7 days, 31% → 36%. (highlighted)← what should change
  • We were wrong if: under 33% after four weeks. (highlighted)← how we'd know
  • Four engineers, six weeks, ships 30 July.the commitment part

Three lines that make it losable.

An illustrative rewrite. The plan stays; a decision adds three lines a commitment never needs: the risk, the expected change and what would prove it wrong. They only count as a test if they are written before the result is in.

Check yourself

Each line is from an approval note. Is it part of the decision, or the commitment?

  1. Ships to all accounts on 30 July.

  2. If fewer than one in ten trial teams use the import in week one, we drop it.

  3. Owned by the growth squad, four engineers.

  4. We think value is the risk here; usability and support load are untested.

  5. Expect weekly active teams to rise from 1,200 to 1,350.

Go deeper

How to use it. Before approving anything, ask for the three lines: the risk being taken, the change expected (with the before number), and the result that would mean it was wrong. If the author can't write them, the item may still be worth doing, but call it what it is: a commitment.

How to spot it. Read the approval minutes. If they record an owner and a date and nothing that could come back "no", the room scheduled work. It did not make a decision.

One thing that has changed. When a draft costs nothing, a proposal can arrive complete in every section a commitment needs and still without the three lines. Ask for them; it is the cheapest check in the review.

Worked examples

Make the call yourself first. The reasoning opens after you choose.

Example 1 of 3

The twelve-page proposal

Devraj founded a company that makes scheduling software for veterinary clinics: nine engineers, roughly two hundred paying clinics. His head of engineering brings him a proposal for a triage assistant, a system that reads incoming appointment requests and suggests urgency levels. The document is excellent: model evaluation, a fallback when confidence is low, an audit log, cost per clinic, and a ten-week estimate with named owners. On value it has one line, "clinics have told us triage is painful". On what happens when a suggestion is confidently wrong, it says nothing.

Check yourself

What should Devraj do with the proposal?

Example 2 of 3

The roadmap item that cannot say its outcome

Lena runs product at a forty-person company selling expense software. Her Q3 roadmap has six items and her CEO likes it. Before the planning meeting she asks of each item what outcome it moves. Five answer, more or less. One does not: Slack integration. The honest answer is that a large customer, worth 8% of revenue, asked for it in a renewal call.

Check yourself

What should Lena write under the Slack integration?

Example 3 of 3

The retro that had not happened

A team ships a redesigned checkout flow. The retro reports: shipped on time, support tickets flat, conversion up 4%. Someone asks what conversion was doing in the eight weeks before. When the number is pulled, it was already drifting up, and the trend accounts for most of the 4%.

Check yourself

What should the retro conclude?

What great operators do

The rules this chapter's scenarios test for, and how to tell when someone follows them.

Rules to keep

Before you open each card, say what you'd look for.

Common failure patterns

How smart people get this wrong. Find each one in the wild.

Trap 1 of 3

Feasibility theatre

What it looks like. A proposal that covers engineering approach, estimates and architecture exhaustively, with value and business viability handled in a sentence each. It reads rigorous because it is rigorous, about the wrong risk.

Check yourself

Tap the lines that stand in for the risks this proposal never tested.

Proposal · AI-written replies for the help centre

Found 0 of 2.

Why smart people do it. Feasibility is the only one of the four risks that is fully knowable from inside the building. You can resolve it by thinking hard, which makes it attractive to competent people who like resolving things. The other three require going outside and being told something you did not want to hear.

The correction. Force the four-risk inventory before discussing the plan. State which risks are addressed, which are assumed, and what would test the biggest assumed one this week.

Trap 2 of 3

The feature roadmap

What it looks like. A quarterly plan that is a list of features with dates. The success criteria are the dates. An outcome may be named once at the top and is never connected to any specific item.

Check yourself

Tap the lines that make this a feature roadmap.

Q3 plan · growth team

Found 0 of 3.

Why smart people do it. Stakeholders ask "what are you shipping," and a feature list is the only answer that fits in a status meeting. It also feels like commitment and competence, which it is, just not the kind being claimed.

The correction. For each item, write the outcome it moves and the evidence that it will. What survives are bets. What doesn't are obligations, and those should be renegotiated explicitly rather than smuggled through as strategy.

Trap 3 of 3

Output mistaken for progress

What it looks like. A retro or update listing shipped work as accomplishment. Velocity up, releases up, the team visibly productive. No behaviour metric appears, or one appears without a baseline.

Check yourself

Tap the lines that report output as if it were progress.

Quarterly update · onboarding squad

Found 0 of 2.

Why smart people do it. Output is measurable, attributable and immediate. Outcomes are noisy, lagging and shared across teams. Under pressure the measurable thing wins, every time, and nobody involved is being dishonest.

The correction. Pair every shipped item with the behaviour it targeted and the observed change. Where nothing was measured, say so. An unmeasured launch is an unevaluated bet, not a success, and the sentence "we don't know yet" is a sign of a serious team.

Cheat sheet

The whole chapter on one card. Save it or share it.

Judgment GymCheat sheet

What a Product Decision Actually Is

  1. 1Every idea has four risks. Name the ones you tested.Value, usability, feasibility, business viability. Passing one says nothing about the others.
  2. 2Count where the pages went.A proposal that is all build plan has been scheduled, not evaluated.
  3. 3Give the team a problem, not a feature."Under ten minutes to a first result" beats "ship onboarding v2 by March".
  4. 4Label commitments as commitments.A promise to a customer is fine. Dressing it up as a bet is not.
  5. 5Report the outcome, not the shipment.What did users do differently, measured against the before?
  6. 6A decision says what would prove it wrong.Write the risk, the expected change and the failing result before you approve.
Train the judgment AI can't do for you.@judgmentgym
Download image

Memes

The failure patterns, one picture each. For the group chat.

Your turn

Make the call

The checkpoint is a launch memo asking leadership to approve a general availability date: the point a feature stops being a limited test and is switched on for everyone. It is well written, it has results from a beta (the limited test with some customers), and it has a risks table with four rows and a mitigation beside each. You say which of its soft claims would change the decision if it were false, then approve, hold or set conditions on the date. A second, shorter case hands you a quarterly plan of eleven features and asks you to make it one the CEO can hold the team to.

The trap. The memo's risks table looks like the four-risk inventory, and it isn't. Its rows are about delivery: docs slipping, volume spiking, a support runbook, endpoint variability. No row asks whether accounts will adopt the feature or whether it moves win rate. Its success metric, at 30 and 90 days, says what winning looks like but not what would show the decision was wrong. Holding the launch until everything is proven is the other trap: the missing checks are small, and the memo says what waiting costs.