Growth Experiment Pre-Launch Readiness Checklist

Missing metric events, inconsistent assignment, or insufficient traffic can make an experiment’s results unusable. Before launch, review the hypothesis, sample size, instrumentation, and rollout plan with the people responsible for the test.

Pre-Launch Checks

Hypothesis and design

• A written hypothesis exists that specifies: who is affected, what changes, why it should work, what the expected effect is
• The experiment scope matches the written hypothesis, including any changes tested together
• The primary metric is defined and is a real, actionable business metric — not a vanity metric or pure feature-usage metric
• The minimum detectable effect (MDE) is defined — you know the smallest lift that would be worth shipping
• A stopping date is set in advance and shared with the team
• There is a documented decision rule: what outcome means "ship," "discard," or "iterate"

Statistics and sample size

• Required sample size has been calculated using baseline conversion rate, MDE, the agreed statistical power, and the significance threshold for the selected test
• Your expected traffic to the experiment surface can reach the required sample size by the stopping date
• If power is borderline, you have explicitly decided to either extend the test window or accept a larger MDE
• Audience allocation is confirmed (typically 50/50 for two variants; rationale documented for any other split)

Audience and assignment

• Assignment logic is confirmed: the same user identifier is used for both assignment and outcome tracking
• Assignment is sticky — the same user always receives the same variant across sessions
• Any exclusion rules (bots, internal users, users in other active experiments) are implemented and tested
• There are no other active experiments running on the same audience that could create interference
• The experiment targets the right stage of the funnel — users who have not already passed the conversion point

Instrumentation and logging

• An assignment event fires when a user first enters the experiment, before any outcome interaction
• The primary metric event is instrumented and confirmed firing in staging or in a test account in production
• Secondary guardrail metrics are instrumented (error rates, load times, downstream funnel events)
• Exposure logs are deduplicated — if the assignment condition fires multiple times per session, only the first fires
• A sample ratio mismatch (SRM) check is scheduled for 24-48 hours after launch

Rollout plan

• The experiment is using a feature flag with percentage-based rollout configured
• Initial rollout percentage is defined (for new surfaces or high-risk changes, consider starting at 5-10% before expanding)
• There is a documented kill procedure: who can turn the flag off, and how quickly
• Monitoring and alerting is in place for the primary metric and for error rates during the first 24 hours
• The team knows who is responsible for reading results on the stopping date

Communication

• The experiment is logged in your team's shared experiment tracker with: hypothesis, primary metric, stopping date, owner
• Relevant stakeholders (product, engineering, design, data) are aware the experiment is running
• The team will not deploy other changes to the same surface during the experiment window without an explicit decision to pause and restart the test

What to Do If a Box Stays Unchecked

Most unchecked boxes are a signal that the experiment is not ready. Common situations:

"We do not know the baseline conversion rate" — pull it from your analytics before launch. If you cannot find it, your analytics are not instrumented well enough to run a reliable experiment.

"We do not have enough traffic" — reconsider the test surface, use a larger MDE if only larger effects would change the decision, extend the planned collection period, or wait for sufficient traffic.

"We are not sure if assignment is sticky" — test it explicitly. Log in as a test user, clear your session, return, and confirm you see the same variant.

"There is another experiment running on this audience" — either delay until the other experiment concludes or implement audience exclusion rules so the two populations do not overlap.

Resolve failed checks before launch, or document why a check does not apply to the experiment.

Discuss experiment infrastructure with LinkIT.