Some questions about a startup idea can be answered with a conversation or a cheap web page. Others need a working product. A lot of wasted effort comes from mixing the two up.
Joel Gascoigne started coding Buffer before he'd checked that anyone wanted it. He says so himself, in a February 2011 post about getting from idea to paying customers. Partway in he stopped and put up a two-page site instead. Then he wedged a pricing page between the two pages, so a visitor had to click a plan before leaving an email address for a launch notice. A small number of people clicked paid plans, and that was enough to send him back to the code. Seven weeks of evenings and weekends later he launched, on 30 November, and by his account the first paying customer turned up within four days.
Figma went the other way. Its launch post talks about three years of silence. Dylan Field has said that when Figma raised its first round, it had no user metrics and the product was still in stealth.
People cite both stories as proof of opposite rules: test first, or build first and let the market judge the finished thing. Both founders ran companies that survived, so neither story tells you much about the odds. Put side by side, though, they show two founders stuck on different questions. This article is about working out which question you're stuck on, and what the cheapest test of it would be.
Two risks under one label
Rob Fitzpatrick's The Mom Test splits risk in two, at least as summarised by study guides. Customer risk asks whether people want this and will pay. Product risk asks whether it can be built. When most of the risk sits in the product, those summaries say, no amount of customer conversation settles it. His examples include video games and a farm fertility monitor.
Buffer was mostly customer risk. Queuing tweets wasn't technically scary; the open question was whether anyone cared. Figma's early browser-based design tool is a simplified example of the opposite. People could picture wanting it, and nobody knew whether a browser could do the job. That contrast is mine, not something any of the sources says. Real ideas usually carry some of each, so the practical question is which one is the bigger unknown right now.
Not all evidence weighs the same
Here is what common signals can and can't tell you.
| What you got | What it tells you | What it doesn't |
|---|---|---|
| A compliment | Goodwill, maybe real interest | Behaviour, or intent to buy |
| An interview about what they do today | This person has the problem and how they cope | That the market is big, or that they'd switch to you |
| A signup or waitlist | Your message landed in that channel | That they'd use it or pay |
| A pricing-page click | Which plan caught their eye | That anyone would pay. Nobody did |
| A signed pilot or letter of intent | Commitment, as strong as its terms (scope, date, price). Some letters are non-binding | That it holds up beyond the pilot |
| A payment | Willingness to pay for this offer, under these conditions | Renewal, or whether a discount, a friendship or a one-off need did it |
Repeat usage belongs on that list too. It's strong evidence for a product people are meant to use often, and you need a product before you can see it.
Fitzpatrick's own test, again via summaries, is whether a conversation ends in commitment or advancement: the other person gives up time, reputation or money, or agrees to a next step. A compliment costs nothing to give. Treat it as a weak signal until behaviour backs it up.
Don't read the table as a ladder. A payment for a service you deliver by hand says a lot about willingness to pay and very little about whether people will put up with your software. The strongest evidence is whichever kind matches the claim you're testing.
What you can test without a product
Interviews about past behaviour tell you whether the problem is real and how people cope. They're poor at predicting what someone will do with something new, and they're easy to steer without meaning to. A landing or pricing page tests whether a message pulls attention in one channel, and it tests willingness to pay only if money changes hands. A manual or concierge service shows what the job really involves and gives you honest data on price, but it hides everything about scale. A demo video or clickable prototype helps when an idea is hard to explain and says nothing about retention. A scoped paid pilot, a pre-order (with explicit delivery and refund terms) or a letter of intent gets you commitment from a specific buyer, though early buyers often differ from later ones. A technical spike tells you whether the hard part is possible and nothing about demand.
Look at Buffer with that in mind. The pricing-page click wasn't a payment, and the product had little technical risk. Gascoigne is open that his early "validated" was a handful of emails and some feedback. The better evidence, a paying customer, arrived after he'd built and launched. So the sequence wasn't "validate, then build." He ran a cheap test for customer risk, then built the smallest thing that could produce stronger evidence.
Food on the Table shows the manual route. In accounts of the Lean Startup method, an early customer got weekly personal service in place of software. Manuel Rosso's talk on concierge MVPs frames that service as a way to learn rather than to scale, and advises writing code only when it makes learning cheaper. The takeaway is narrow. Manual delivery shows what the job involves and whether people accept a high-touch version. It can't tell you whether software will do the job.
One rule covers every test before a product exists: say what doesn't exist yet. A landing page should describe what customers will get and when, and if you take money, the refund terms should be plain. Buffer's pricing page collected emails for a launch notice, which is a long way from pretending the product was live.
When building is the experiment
Some questions don't yield to conversation. Two cases show how, and both come with the usual survivor caveat.
Dropbox is awkward to describe and obvious once you see it. Drew Houston's own slide deck lists a three-minute screencast on Hacker News in April 2007 that brought "lots of immediate, high-quality feedback", then a private-beta video in March 2008 after which the waitlist jumped from 5,000 to 75,000 in a day. In a 2011 interview he said they'd expected 15,000 at most, and that they used a closed beta because they didn't want to risk a buggy product on all 75,000. One secondary account says a working prototype sat behind the first video. I couldn't confirm that from Houston himself. What the evidence supports is narrower: a demo helped communicate a hard-to-explain product and showed interest among early adopters. It doesn't show willingness to pay, retention, or that the audience generalised. A waitlist is interest, not commitment.
Figma is the cleaner case. The open question was whether a professional design tool could run in a browser at acceptable speed. Field says people who had tried WebGL before warned them off. According to Sequoia's profile, the closed beta came in December 2015 and the first dollar was charged in summer 2017. Interviews couldn't settle feasibility. That took technical experiments and a working prototype, whatever customer learning went on alongside. It was also affordable only because of a founding pair with deep graphics skills and investors willing to fund years in stealth. Few solo founders have both.
What the two cases share, and this is my inference, is that building was justified by one specific unanswered question. If your riskiest assumption can't be tested credibly without a working prototype, build, and build only the part that tests it.
How much validation is enough?
There's no credible number. The nearest research is a line of randomised trials by Camuffo, Gambardella, Spina and colleagues. In the 2020 trial (Management Science, 116 Italian startups in the paper's panel sample), some founders were trained to treat their idea as a theory, form hypotheses and test them, while a control group followed its instincts. The authors report that the treated group pivoted more and earned more revenue.
The 2024 replication in the Strategic Management Journal pooled four randomised trials covering 759 firms. Its abstract reports a positive effect on idea termination and a nonlinear effect on radical pivots: treated firms ran a few pivots rather than none or many. The authors read this as more efficient search plus more "methodic doubt". So the 2020 headline, more pivoting, didn't hold up as stated. The clearer result is more decisive stopping and more deliberate changes.
Both are training programmes for early-stage founders, and neither compares interview-first with MVP-first. They support something narrower: tests built to be able to say no seem to help founders decide. That reading is mine, not a conclusion the authors draw about MVPs.
In practice four things move the answer: what a wrong build would cost, how easily you could undo it, how reachable your customers are, and how much is still unknown. A weekend project you can bin needs very little evidence. Six months of work for buyers you can't easily reach needs a lot more.
A way to decide (my synthesis, not an established method)
Five questions, in order. Skip any you can already answer with evidence.
- Which assumption is riskiest? That the problem exists, that a specific group cares, that your solution works, that it's feasible, or that they'll pay.
- What's the cheapest test that could come back negative? A conversation, a manual service, a prototype, a technical spike, a pilot, a payment.
- What result would change your mind? Write it down before you run the test. If no result would make you stop or change course, you aren't testing anything.
- What's still unknown afterwards? Three paid pilots say nothing about what the fourth customer costs to find.
- What's the next smallest step? Keep learning, build a limited version, run a paid pilot, change the customer, or stop.
A rough rule for interview versus experiment: when several people independently describe the same recent problem and workaround, consider moving from broad discovery to a more specific test. More interviews can still help if you're exploring a different segment or an unresolved objection. Otherwise the next useful question may be whether someone will commit to a concrete step.
An experiment for a solo founder: the scoped pilot ask
This suits a B2B or prosumer idea.
- Hypothesis: People in one narrow segment who already spend time or money working around this problem will agree to a specific, paid pilot.
- Test: Talk to people who've shown the problem in past behaviour. Describe exactly what they'd get, by when, and what isn't built. Ask for a concrete step: a start date, a price, or a deposit with clear refund terms.
- Before you start: Write down how many yeses would make you proceed and how many would make you rethink. Choose a threshold that fits your market, your access to buyers and the cost of being wrong. Don't borrow a universal number.
- Record: For each person, their current workaround, what it costs them, what they said (paid, agreed in writing, asked to wait, declined) and why.
- Limits: You'll mostly reach people like you, a pilot price isn't a product price, and delivering by hand hides software costs.
- Next decision: A yes counts when it's a defined step, such as an accepted scope, timeline and price, not verbal encouragement. Enough of those means deliver by hand and note which steps repeat, since those are what to automate. Repeated "interesting, not now" means urgency or the offer hasn't been shown. Ask why: wrong buyer, unclear offer, budget timing, missing trust, or a problem that isn't pressing. Then narrow the segment, change the offer, or stop.
Back to Buffer and Figma
The two stop looking like opposites once you ask what each founder didn't know. Gascoigne didn't know whether people cared, and a small page could probe that. Field didn't know whether it was possible, and that took building. Both are survivors telling their own stories, so hold the conclusions loosely.
Before your next coffee chat or your next hour in the editor, write down the assumption most likely to sink you, and pick the cheapest test that could prove it wrong.
What's your counterexample? Have you built first and been right to, or run a validation test that said yes while the product still failed? What did you measure, and what would you do differently?
