Validation

Phase 1 of the playbook: proving (or disproving) in 1–2 weeks that the problem from ideation exists, hurts, would be paid for, and belongs to people you can reach. It covers the evidence ladder, customer interviews done properly (The Mom Test, customer discovery, Jobs to Be Done), recruiting, synthesis, smoke tests and pre-sales, assumption mapping, and the validated problem brief that hands off to product design. Later-stage product–market fit measurement is in launch and iterate.

Entry criteria and timebox

What you are validating

Validation is a ladder. Each rung needs different evidence, and you can't skip rungs: a great landing page conversion rate means little if nobody has the problem badly enough to change behavior.

Most failed validations fail at rung 2: the problem is real but mild. People describe it readily and have never lifted a finger to fix it.

Signal strength: from compliments to money

Rob Fitzpatrick's The Mom Test (2013) is the most practical guide to customer conversations. Two of its ideas define this phase:

• Bad data comes in three forms: compliments, fluff (generic claims, hypotheticals and promises about the future) and ideas (feature requests). None of it is evidence.
• Commitment and advancement. A meeting only produced evidence if the other person gave something up (commitment) or moved to the next concrete step of a real buying process (advancement). The book's currencies of commitment are time, reputation risk and money.

The Mom Test rules

Named because the questions should be ones even your mother couldn't lie to you about. The three rules, verbatim:

1. Talk about their life instead of your idea.
2. Ask about specifics in the past instead of generics or opinions about the future.
3. Talk less and listen more.

More rules of thumb from the book, paraphrased: keep conversations casual rather than formal; if they haven't looked for a solution, they don't care enough; anchor fluff back to specifics; and don't tell them your idea until you've learned about their life (preferably, not at all in the first meeting).

Customer discovery

Steve Blank's The Four Steps to the Epiphany (2005) introduced customer development, the method the Lean Startup grew out of. Its core principle is usually summarized as "there are no facts inside your building, so get outside": every assumption about customers is a hypothesis until tested with customers. Blank defines a startup as "an organization formed to search for a repeatable and scalable business model."

The practical consequence: an interview in a café or on a video call with a real target customer is worth more than a week of desk research. "Getting out of the building" today mostly means video calls, but it still means strangers in the segment, not your network's opinions.

Jobs to Be Done interviews

Jobs to Be Done (JTBD) frames a purchase as hiring a product to make progress in a particular circumstance. The most complete account is Competing Against Luck (Clayton Christensen, Taddy Hall, Karen Dillon and David S. Duncan, 2016). The standard illustration, as told by the Christensen Institute: a fast-food chain found that morning customers "hired" milkshakes to make a long commute more bearable and stave off hunger, so the real competitors were bananas and bagels, not other milkshakes.

The switch interview. Developed by Bob Moesta and colleagues at the Re-Wired Group (see Moesta's Demand-Side Sales 101, 2020), it reconstructs the story of a recent switch from an old way to a new one, as a timeline: first thought → passive looking → active looking → deciding → using. Interview people who recently switched (or tried to) in your problem area: the memory is fresh and the causes are concrete.

The four forces of progress decide whether someone switches:

A switch happens when push + pull outweigh anxiety + habit. Most founders only design for pull (features). Your interviews should tell you how strong the push is (rung 2 of the ladder) and what anxieties you'll have to remove.

SWITCH INTERVIEW PROMPTS (for someone who recently changed
how they handle the problem)
1. First thought: when did you first realize the old way
   wasn't working? What happened that day?
2. Passive looking: what did you notice or read after that?
3. Trigger: what made you actively start looking? Why then?
4. Active looking: what options did you consider? Where did
   you look? Who did you ask?
5. Deciding: what nearly stopped you? What tipped it?
6. Using: first week with the new way - what surprised you?
   What do you still do the old way?

Recruiting interviewees

How many: plan for 10–20 problem interviews in the segment. This is a heuristic, not a statistical rule; the real stopping condition is saturation, when three or four interviews in a row tell you nothing new. If answers are still all over the place after 15, the segment is probably too broad: narrow it and continue.

Outreach templates. Keep them short, specific and honest: you are researching a problem, not selling.

WARM INTRO REQUEST (to someone who knows the target)
 
Hi [name] - I'm researching how small engineering teams
handle [problem area], before building anything. Do you
know 1-2 people who [specific trait, e.g. "ship Postgres
migrations weekly without a DBA"]? I'd love 20 minutes to
hear how they do it today. No pitch - I don't have a
product. Happy to send a blurb you can forward.
Thanks, [you]

COLD MESSAGE (email or LinkedIn; under 90 words)
 
Subject: How does [team/company] handle [problem]?
 
Hi [name] - I saw [specific: your post / your talk / your
team's job ad] about [topic]. I'm a developer researching
how teams like yours handle [problem area]. Could I ask you
about the last time [specific event] happened? 20 minutes,
any time next week, and I'll share what I learn across the
interviews. No product, no pitch.
[you, one-line credibility: "ex-[company] backend engineer"]

COMMUNITY POST (check the community's rules first)
 
Title: How do you handle [problem] on small teams?
 
I'm researching [problem area] and would love to hear
stories: the last time [event] happened, what you did,
and what it cost you. I'm not selling anything. If you're
up for a 20-minute call, reply or DM me - I'll post an
anonymised summary of what I learn back here.

FOLLOW-UP (no reply after 4-5 days; send once)
 
Hi [name] - bumping this in case it got buried. Even a
2-line reply about how you handle [problem] today would
help. Totally fine if not.

AFTER THE INTERVIEW (same day)
 
Thanks for your time today, [name]. The bit about [specific
thing they said] was really useful. Two quick asks:
1. Who else should I talk to about this? An intro would be
   great.
2. I'm going to try solving [problem] for a handful of
   teams in the next few weeks. Could I show you an early
   version on [date]?

The last message contains two commitment asks: an introduction (reputation) and a follow-up meeting (time). Log the answers; they are data.

Interview script

Aim for 20–30 minutes, record only with permission, and take notes in the other person's words. Two people is ideal: one asks, one writes.

PROBLEM INTERVIEW SCRIPT (20-30 min)
 
0. SET-UP (2 min)
   "Thanks for this. I'm researching how people handle
   [area]. I'm not selling anything. There are no right
   answers - the more specific, the better. OK if I take
   notes?"
 
1. CONTEXT (3 min)
   - Tell me about your role. What does a normal week look
     like?
   - Where does [area] fit in?
 
2. THE LAST TIME (10 min)  <- most of the value is here
   - Tell me about the last time [problem event] happened.
   - What were you trying to do? What went wrong?
   - What did you do next? And then?
   - How long did that take? Who else was involved?
   - What did it cost (time, money, stress, customers)?
 
3. WORKAROUNDS AND SPEND (5 min)
   - How do you handle this today?
   - What have you tried? Why did you stop using it?
   - Are you paying for anything related to this?
   - Have you looked for a solution? Where? What happened?
 
4. PRIORITY (3 min)
   - Where does this rank among the problems you deal with?
   - What would happen if you never solved it?
 
5. WRAP-UP AND COMMITMENT (3 min)
   - Is there anything I should have asked?
   - Who else should I talk to? Could you introduce me?
   - Can I come back to you when I have something to show?
 
RULES: no pitch until the end (if at all). Never ask "would
you". When they generalize ("we usually..."), ask for the
last specific time. Silence is fine - wait.

Synthesis

Synthesise the same day, while memory is fresh. Then look for patterns every 4–5 interviews and at the end.

INTERVIEW NOTES (one per interview)
 
Who: [role, company size, segment]      Date:    Source:
Matches segment? Y / N   Knows me? Y / N
Problem raised unprompted?  Y / N   (before I mentioned it)
Last occurrence: [when, what happened - their words]
Frequency: [per week/month]
Workaround: [what they do]    Cost: [time / money / risk]
Tried solutions: [what, why abandoned]
Current spend: [money or headcount]
Anxieties / habits (JTBD):
Best quote: "..."
Commitment obtained: none / time / reputation / money
Next step agreed:
Surprises / new hypotheses:

FREQUENCY x INTENSITY GRID
 
              low intensity          high intensity
            +-----------------------+-----------------------+
 high freq  | "Vitamin": annoying,  | TARGET: frequent and  |
            | tolerated; hard to    | painful; people are   |
            | charge for            | already hacking fixes |
            +-----------------------+-----------------------+
 low freq   | Ignore                | "Painkiller, rarely": |
            |                       | can work if the cost  |
            |                       | per event is huge     |
            +-----------------------+-----------------------+

Worked example (idea A, continued)

Illustrative results for the migration checker from ideation, against the criteria written there ("continue if at least half raise the problem unprompted and at least 3 give a real commitment"):

Decision: re-segment, don't kill. Narrow to 5–20 engineer teams with no DBA, run 5 more interviews there, and add a pre-order ask. That is the kind of adjustment the pre-set criteria make possible without self-deception.

Smoke tests and quantitative signals

Interviews tell you why; smoke tests tell you how many. Run them in parallel with interviews, not after.

Buffer's two-page MVP is the textbook small version. Joel Gascoigne's account ("Idea to Paying Customers in 7 Weeks", 2011) describes a first page explaining the product and a second collecting emails; he then added a pricing page between them, so a visitor had to click a plan before leaving their email, which tested both demand and price.

Setting thresholds without invented benchmarks

Published conversion-rate and click-through benchmarks vary enormously by platform, industry, traffic source and year, and many come from vendors selling the tools. Don't import a benchmark. Derive the threshold from your own economics, and write it down before the test starts.

SMOKE TEST CARD                                 Date: ______
Hypothesis:  [segment] will [action] because [problem]
Test:        [landing page / fake door / ads / pre-sale]
Traffic:     [source; must be the target segment; not friends]
Sample:      stop after [N] visitors from that source or [date]
Metric:      [sign-ups / clicks on paid tier / pre-orders]
Threshold:   pass if >= ___ ; fail if < ___  (set NOW, and why)
Result:
Decision:    continue / change pitch once / re-segment / kill

What makes a smoke test trustworthy:

• Traffic from the segment. A Hacker News spike from people who will never buy is noise.
• Enough of it. A handful of visitors can't distinguish 2% from 10%; decide the sample size before you look.
• One change at a time. If you change the headline and the audience together, you learn nothing.
• A real cost to the visitor. An email is cheap; a click on a paid plan is dearer; a card number is dearest.

Pre-sales and letters of intent

Money is the strongest evidence, and asking for it is the most avoided task in the whole playbook. Ask anyway, at the end of a good second conversation or on the landing page.

PRE-SALE ASK (end of a second conversation)
 
"Based on what you told me, I'm building [outcome] for teams
like yours, first version by [date]. I'm offering [N] early
teams [price/discount] for the first [period], paid now,
fully refundable if it doesn't ship by [date] or doesn't do
[core job]. Would you like one of the slots?"
 
If yes: send the payment link today.
If "not yet": ask "What would you need to see first?"
   - the answer is your spec or your objection list.

LETTER OF INTENT (non-binding; B2B)
 
[Company] intends to pilot [product] for [use case] starting
[date], subject to [conditions, e.g. passing a security
review]. If the pilot achieves [measurable outcome], we
expect to purchase at approximately [price] per [unit].
This letter is not a binding commitment.
Name, role, signature, date

An LOI is weaker than cash but much stronger than a verbal "yes", especially when signed by the person who owns the budget. Pre-sales and LOIs also give you the first users for launch.

Concierge MVP

Deliver the outcome by hand, openly, for a few customers: run their migrations review yourself, send the report by email, do their bookkeeping in a spreadsheet. You learn the real workflow, what "done" means to them, and whether they come back, before writing product code. Paul Graham's Do Things that Don't Scale (opens in a new tab) (2013) makes the case for this kind of manual, unscalable effort at the start.

Riskiest assumption testing

Test the assumption most likely to kill the idea first, not the one that is easiest or most fun to test.

Assumptions mapping (David J. Bland; used in Testing Business Ideas by Bland and Alexander Osterwalder, 2019) plots each assumption on two axes: importance (how badly the idea fails if it's wrong) and evidence (how much you already have). Bland color-codes assumptions as desirability (do they want it?), viability (should we do it: will it make money?) and feasibility (can we do it?). He credits the underlying 2×2 to Jeff Gothelf, Josh Seiden and Giff Constable, from whom he learned it.

ASSUMPTIONS MAP
 
                 little evidence         lots of evidence
               +-----------------------+-----------------------+
 important     | TEST FIRST            | Monitor; plan around  |
               | (riskiest)            |                       |
               +-----------------------+-----------------------+
 unimportant   | Park; revisit later   | Ignore                |
               |                       |                       |
               +-----------------------+-----------------------+
 
Tag each assumption: D = desirability  V = viability
                     F = feasibility
Typical riskiest for a technical founder: D and V, not F.

Strategyzer's test card and learning card keep each experiment honest:

TEST CARD                          LEARNING CARD
We believe that ____________       We believed that ___________
To verify that, we will ____       We observed ________________
And measure ________________       From that we learned _______
We are right if ____________       Therefore, we will _________

Validated learning and build–measure–learn

Eric Ries's The Lean Startup (2011) gives the vocabulary for this phase:

Plan the loop backwards: decide what you need to learn, then what you'd measure to learn it, then the least you have to build to get that measurement. Building first and looking for lessons afterward is how six-month MVPs happen.

Later: the Sean Ellis test

Once real users have used a real product (Phase 5), Sean Ellis's survey asks how they would feel if they could no longer use it. His widely used heuristic: if at least 40% answer "very disappointed", you are likely close to product–market fit. It does not apply in this phase: people can't be disappointed to lose something they've never used. It is covered in launch and iterate, and Marc Andreessen's framing of product–market fit is on the a16z sheet.

False positives and false negatives

Ethics of fake doors and smoke tests

Smoke tests are legitimate if they don't deceive people to their cost. Rules to follow:

Friction points in validation

Kill, pivot or continue

At the end of the timebox, apply the criteria you wrote in the ideation one-pager. These are heuristics for a solo B2B or prosumer product, not laws; adjust them in advance for your market.

Exit criteria and handoff to Phase 2

PHASE 1 EXIT CHECKLIST
[ ] 10-20 interviews with strangers in the segment
[ ] Notes synthesised; unprompted count and grid done
[ ] Pre-set thresholds met (or decision to kill/re-segment)
[ ] Current workaround and its cost documented
[ ] Commitments logged (time / reputation / money)
[ ] At least one reachable channel tested
[ ] Smoke test run against a pre-set threshold (if used)
[ ] Validated problem brief written (template below)
[ ] 5+ interviewees agreed to test the prototype in Phase 2

Handoff artifact: the validated problem brief. Product design starts from section 7 (the must-have outcomes) and uses the interviewees who agreed to test the prototype.

VALIDATED PROBLEM BRIEF                  Idea: _____ Date: ____
Evidence base: [N] interviews, [smoke tests], [dates]
 
1. WHO (the segment the evidence supports - narrower than
   the one you started with, usually)
   - Role / company / context:
   - Early adopter trait:
   - Who pays (if different from who uses):
 
2. THE PROBLEM, IN THEIR WORDS
   - Summary (1-2 sentences):
   - Quotes (3-5, attributed by role, verbatim):
     "..." - [role, company size]
   - Raised unprompted by: [x] of [N]
 
3. FREQUENCY AND TRIGGER
   - How often it happens:
   - What triggers it (the moment of pain):
 
4. CURRENT WORKAROUND AND ITS COST
   - What they do today:
   - Cost per occurrence (time / money / risk):
   - What they tried and abandoned, and why:
 
5. WILLINGNESS-TO-PAY EVIDENCE
   - Current spend on the problem:
   - Commitments: pre-orders [n], LOIs [n], pilots [n],
     intros [n] - list them
   - Price signals (what they compared it to):
 
6. REACHABLE CHANNEL
   - Channel that worked, with numbers (e.g. 30 messages ->
     11 replies -> 7 calls):
   - Communities / directories where they gather:
 
7. THE 1-3 MUST-HAVE OUTCOMES
   (outcomes, not features: "a risky migration is caught
   before it merges", not "a dashboard")
   1.
   2.
   3.
 
8. ANXIETIES AND HABITS TO OVERCOME (JTBD)
   -
 
9. WHAT WE LEARNED THAT CHANGED THE HYPOTHESIS
   -
 
10. OPEN QUESTIONS FOR DESIGN (and who will answer them)
   -
 
Prototype testers lined up: [names, 5+]

References

• Rob Fitzpatrick, The Mom Test: How to talk to customers and learn if your business is a good idea when everyone is lying to you (2013) and momtestbook.com (opens in a new tab): the three rules, bad data, commitment and advancement
• Steve Blank, The Four Steps to the Epiphany (2005): customer development and its four steps; Blank and Bob Dorf, The Startup Owner's Manual (K&S Ranch, 2012) for the expanded method
• Steve Blank, What's A Startup? First Principles (opens in a new tab) (2010): the "search for a repeatable and scalable business model" definition
• Clayton M. Christensen, Taddy Hall, Karen Dillon and David S. Duncan, Competing Against Luck (Harper Business, 2016): Jobs to Be Done
• Christensen Institute: Jobs to Be Done (opens in a new tab): the theory and the milkshake example
• Bob Moesta with Greg Engle, Demand-Side Sales 101 (Lioncrest, 2020) and the Re-Wired Group (opens in a new tab): switch interviews and the forces of progress
• David J. Bland and Alexander Osterwalder, Testing Business Ideas (opens in a new tab) (Wiley, 2019): experiment library, test and learning cards
• David J. Bland, Assumptions Mapping (opens in a new tab) (Precoil): importance × evidence; desirable, viable, feasible
• Eric Ries, The Lean Startup (Crown Business, 2011): validated learning, build–measure–learn, leap-of-faith assumptions, pivot or persevere
• Joel Gascoigne, Idea to Paying Customers in 7 Weeks: How We Did It (opens in a new tab) (Buffer, 2011): the two-page MVP and pricing-page test
• Paul Graham, Do Things that Don't Scale (opens in a new tab) (July 2013): manual, concierge-style effort early on
• Sean Ellis, PMF survey (opens in a new tab): the "very disappointed" question and 40% heuristic, for Phase 5