An A/B test duration calculator can turn a vague testing plan into a practical estimate. By combining baseline conversion rate, expected improvement, confidence level, statistical power, eligible traffic, and traffic allocation, you can estimate the sample size and time required before launching an experiment. This guide explains the inputs, shows how the calculation works, and provides rules for avoiding premature conclusions.
Overview
A test duration estimate answers two related questions: how many visitors or users does each variant need, and how long will it take to collect that sample? The answer depends less on the calendar than on the size of the effect you are trying to detect.
Large improvements are easier to detect than small improvements. A page that increases conversion from 5% to 7% generally needs fewer observations than one expected to increase conversion from 5% to 5.2%. Lower baseline conversion rates also tend to require more traffic because conversions are less frequent.
A useful calculator should estimate:
- Required sample size per variant: the number of eligible visitors or users needed in the control and treatment groups.
- Total sample size: the combined number of observations across all variants.
- Estimated duration: total required sample divided by eligible daily traffic, adjusted for traffic allocation.
- Expected conversions: a reasonableness check for whether the test can produce enough conversion events to support a decision.
The result is an estimate, not a guarantee. Real traffic can vary by weekday, season, device, channel, and campaign activity. Measurement errors can also make a test appear more or less conclusive than it really is.
How to estimate A/B test duration
Start by defining one primary conversion event. For a landing page, this might be a form submission, purchase, or qualified signup. Avoid calculating duration from several interchangeable goals and then choosing the most favorable result after the test begins.
- Record the baseline conversion rate. Use a recent, relevant period and the same audience definition planned for the experiment. If the baseline is 5%, enter 0.05.
- Set the minimum detectable effect. This is the smallest improvement worth acting on. A 20% relative lift from a 5% baseline produces a target rate of 6%. The absolute difference is one percentage point.
- Choose confidence and power. A common planning assumption is 95% confidence and 80% power, but the correct choice depends on the cost of false positives, false negatives, and the decision being made.
- Estimate eligible traffic. Use visitors or users who can actually enter the experiment, not total website sessions if many sessions are excluded.
- Apply the traffic split. With a 50/50 test, each variant receives about half of the eligible traffic. Unequal allocation changes the required duration and usually leaves the smaller group as the constraint.
- Convert sample size into days. If the calculator requires 10,000 total observations and the experiment receives 800 eligible visitors per day, the basic estimate is 12.5 days.
For a two-variant test, a planning formula commonly estimates the sample per variant using the baseline rate, target rate, confidence threshold, and power. In simplified terms, the required sample grows as the difference between the two conversion rates becomes smaller. A calculator handles the statistical values, but the quality of the output still depends on the inputs.
Do not treat the first day that reaches the calculated sample as an automatic stopping signal. Plan for complete business cycles where possible, and define the stopping rule before launch. Repeatedly checking significance and stopping immediately when a result crosses a threshold can increase the risk of a misleading conclusion.
Inputs and assumptions
Baseline conversion rate
Use a stable baseline for the exact conversion event and audience being tested. If traffic sources behave differently, calculate the rate for the traffic that will actually be randomized. A sitewide average may be unsuitable for a paid landing-page experiment.
Minimum detectable effect
The minimum detectable effect, or MDE, should reflect a meaningful business decision rather than an optimistic forecast. Distinguish between relative and absolute lift. If conversion rises from 4% to 5%, that is a one-percentage-point absolute increase and a 25% relative increase. Entering the wrong interpretation can materially change the duration estimate.
Confidence and power
Confidence is used to control how readily the analysis labels a difference as statistically significant under the chosen method. Power describes the chance of detecting the planned effect if it exists. Higher confidence or higher power generally requires a larger sample.
Traffic and unit of analysis
Choose the unit that matches the experiment. Visitor-based tests should not switch to session counts halfway through. User-based tests require consistent assignment so one person does not appear in different variants. Estimate traffic after exclusions such as bots, internal users, duplicate records, or visitors who cannot reach the tested step.
Primary metric and guardrails
Conversion rate alone may hide harmful changes elsewhere in the funnel. Track relevant guardrails such as revenue per visitor, refund rate, lead quality, page errors, or progression to the next step. A test can improve a button click while reducing completed purchases. For broader context, compare the result with your landing page KPI checklist and review the funnel in GA4 funnel analysis.
Tracking quality
Before trusting a duration estimate, verify that the experiment assignment and conversion event are recorded consistently. Check event definitions, duplicate conversions, attribution changes, and missing traffic. The GA4 tracking audit checklist can help identify measurement problems, while a documented website tracking plan clarifies the event, owner, and reporting logic.
Worked examples
Example 1: A moderate landing-page improvement
Suppose a landing page converts 5% of eligible visitors. The team considers a 20% relative improvement meaningful, so the target rate is 6%. Using a two-sided 95% confidence assumption, 80% power, and an even split, a normal-approximation calculation produces roughly 8,100 observations per variant. The total requirement is about 16,200 observations.
If the page receives 1,000 eligible visitors per day, the mathematical estimate is approximately 16 days. A practical plan might run for at least two full weeks and then continue until the required sample and predefined review conditions are met. If weekday and weekend behavior differ, extending across complete weekly cycles can make the comparison more representative.
Example 2: A small improvement at a low baseline
Now assume a checkout step converts at 2%. The team wants to detect a 10% relative improvement, from 2% to 2.2%. With the same confidence and power assumptions, the rough requirement is about 80,000 observations per variant, or 160,000 total.
At 300 eligible visitors per day, that would take more than 500 days under a simple traffic estimate. This result is not a reason to run an extremely long test by default. It is a signal to revisit the decision: choose a larger, commercially meaningful effect, improve the experiment’s traffic source, test a stronger hypothesis, or use a different research method before committing to the experiment.
Example 3: Unequal allocation
If a test sends 70% of eligible traffic to control and 30% to treatment, the smaller treatment group limits the schedule. Even if total traffic is unchanged, the treatment reaches its required sample more slowly than it would in a 50/50 design. Enter the actual allocation into the calculator rather than assuming an even split.
When to recalculate and what to do next
Recalculate before launch whenever the baseline rate, traffic forecast, audience, allocation, primary conversion, or minimum detectable effect changes. Recalculate after launch only for a defined planning reason, such as a sustained change in eligible traffic or a tracking correction. Do not change the target simply because early results are disappointing or because a promising result has appeared.
Review the estimate when:
- paid campaigns, SEO rankings, or seasonal demand materially change the audience mix;
- the conversion event is redefined or the funnel step changes;
- traffic is lower or more volatile than the launch forecast;
- one variant has a technical issue, uneven exposure, or assignment failure;
- the test expands from two variants to several variants;
- the business decision changes from detecting a lift to protecting revenue or lead quality.
Before starting, write down the hypothesis, primary metric, MDE, confidence, power, allocation, planned minimum duration, and stopping rule. During the test, monitor data quality and guardrail metrics without turning every interim result into a final verdict. At the end, report the absolute and relative change, uncertainty around the estimate, sample size, test dates, exclusions, and any tracking limitations.
Use the calculator as a planning tool, not as a substitute for judgment. Revisit it whenever the underlying inputs change, and connect the result to a documented measurement plan. That approach makes A/B testing more repeatable and keeps conversion rate optimization focused on decisions that matter.
