AI Product Kill / Pivot Criteria: Writing Continue, Pivot, and Stop Conditions Before the Experiment

"Just one more feature and maybe users will pay" is the most expensive sentence in product validation. Without pre-written ending criteria, any failure can be explained away as insufficient sample size, a model that isn't strong enough, or marketing that hasn't started yet. The more code accumulates, the more the team tends to keep investing, and the experiment shifts from searching for facts to maintaining hope.

Kill criteria are not pessimistic forecasts, and Pivot is not switching to a trending direction. They are a pre-commitment decision contract: within a fixed time and sample, what evidence supports continuing, what evidence shows the problem exists but the current solution is wrong, and what results require stopping investment. The criteria must be signed before seeing the experiment results.

Set Thresholds for Each of the Five Evidence Chains

An AI product must pass the following dimensions simultaneously:

1. Demand: Target users have a real, recently occurring problem with observable alternative behavior;
2. Commitment: Users are willing to provide data, time, approval resources, or payment;
3. Value delivery: The product completes the task under clear quality gates and produces measurable improvement;
4. Unit economics: The price can cover models, infrastructure, support, and manual review;
5. Risk: Authorization, privacy, severe errors, and rollback are within acceptable boundaries.

A composite average score cannot mask a fatal item. For example, if payment conversion is high but the product generates medical advice errors that humans cannot catch in time, the risk gate fails and this delivery mode must stop. Conversely, if model accuracy is high but users never return, you cannot declare continuation based on technical metrics alone.

For interview and payment evidence collection, refer to User Interviews and Willingness to Pay Validation. Value delivery should be based on the completion rate and quality gates in the Time to Value Baseline, rather than cherry-picking the best-looking metric on the fly.

Write Clear Triggers for Each Outcome

Taking a six-week pilot for "generating ready-to-publish product compliance descriptions for small e-commerce sellers" as an example, the team plans to contact 30 merchants matching the persona, complete interviews with at least 12, and accept up to 60 authorized cases. Before the experiment, the following can be frozen:

• Continue: At least 6 merchants enter the pilot, 4 pay at 1,000 RMB, first-pass acceptance rate is no lower than 90%, severe compliance errors are 0, per-fully-delivered cost is no more than 80 RMB, and 3 submit a second batch of tasks within four weeks;
• Pivot: The problem occurrence rate is above 60%, but the main reasons for rejection concentrate on a changeable factor, such as target customers lacking approval budget, delivery format mismatch, or a particular input channel being constrained;
• Stop: Fewer than 4 merchants can provide recent cases after 12 qualified interviews, or no one pays after two rounds of quotes, or a severe risk emerges that cannot be controlled by scoping and manual review;
• Extend once: Only when a pre-recorded execution failure causes insufficient valid samples, such as the payment link being unavailable for three days; the extension duration and supplementary sample cap must also be pre-limited.

A "high problem occurrence rate" alone cannot trigger a Pivot. A pivot must have a concentrated, verifiable blocking cause, plus a new experiment that changes only one hypothesis. If demand is weak, changing the audience just restarts research rather than preserving the original conclusion.

Make a Decision with Real Results

After six weeks, 14 of the 30 merchants completed qualified interviews, 9 presented recent cases, 5 entered the pilot, and 3 paid. A total of 45 documents were processed, 39 passed first-pass acceptance, with a first-pass acceptance rate of 39 / 45 = 86.7%; there were no severe compliance errors, but the per-fully-delivered cost was 92 RMB; 2 merchants submitted a second batch.

These results neither meet the "continue" thresholds for payment count, acceptance rate, cost, and reuse, nor trigger the direct stop lines of weak demand or zero payment. If rejection records show that 6 merchants all declined because they needed integration with their existing product management system rather than because the problem was unimportant, you can propose a "delivery channel pivot": keeping the target audience, problem, and pricing unchanged, only changing PDF delivery to a constrained CSV import prototype, and running a two-week supplementary experiment.

The supplementary experiment also needs thresholds, such as 4 merchants trialing, 3 completing the import, first-pass acceptance rate reaching 90%, and manual time dropping below 20 minutes per document. If these do not hold, stop; you cannot simultaneously change pricing, industry, and model. Change one key hypothesis at a time, so you can know where new evidence comes from.

Decision Logs Must Preserve Failures and Dissent

Each review records: frozen date, owner, experiment version, target sample, original denominator, results per threshold, anomalies, refunds, risk events, decision, and next budget. People involved in delivery tend to overestimate the value of continuing. At least one person not involved in daily execution should recompute the metrics and randomly replay failed cases.

Deleting unqualified results to "clean the data" is forbidden. Persona mismatches can be excluded, but the persona rules must be fixed before recruitment; dropouts, refunds, and tasks that cannot be completed all count as results. If a manager approves overriding a criterion, they should write down the new evidence, the maximum additional budget, and the final date, rather than verbally saying "the strategic value is large."

The manual ledger for a concierge pilot can follow AI Product Concierge Pilot; whether a candidate change is worth advancing to the next round can be located via AI Feature Evidence Scorecard to identify the most missing evidence. But tools cannot replace the act of stopping: once a Kill is triggered, close recruitment and payment entries, notify pilot users, refund or complete delivery as agreed, set a data deletion time, and archive the reusable research conclusions.

Distinguish Five Types of Pivot to Avoid a Full Reset

Common pivots include: target audience change, problem scope change, delivery mechanism change, channel change, and charging unit change. Each time, only declare one main type of change, and list the hypotheses that remain unchanged. For example, moving from individual designers to design agencies is an audience change; changing from "generating design drafts" to "checking brand consistency" is a problem change; switching from a web page to an API is a delivery mechanism change.

If you simultaneously change users, problem, channel, and pricing, the old experiment cannot provide a valid prior, and the new direction should establish new hypotheses, samples, and stop criteria. Labeling it as "new research" is more honest than disguising it as a continuous pivot, and makes it easier later to judge which evidence still holds.

Six Final Questions for the Review Meeting

• Was each threshold written down before the first user entered the experiment?
• Can the raw records reproduce the numerator, denominator, costs, and refunds?
• Is there a single-item risk gate, or was it offset by a composite score?
• Does the Pivot change only one main hypothesis, with a new deadline?
• Does the extension match the pre-defined execution-failure exception?
• After Kill is triggered, are there owners for customer notification, refunds, data deletion, and lessons archival?

Clear stop criteria do not kill good products. They stop the team from spending limited time on unfalsifiable directions, and let products genuinely worth continuing hold stronger evidence. The ultimate goal is not to stop frequently, but to let every continuation answer: "What piece of real-world evidence justifies investing the next round of resources?"

Further Reading

• User Interviews and Willingness to Pay Validation
• AI Product Concierge Pilot
• AI Feature Evidence Scorecard
• AI Product Cold Start and PMF