Report
What stores A/B test, and how long they let tests run
We looked at thousands of A/B tests that hundreds of online stores ran and finished on Replo: what they tested, how long each test ran and how they split traffic.
The short answer
Half of the tests ended within a week (median 7 days), and 19% ended within a day. Only 33% ran the two weeks most guides ask for. Stores tested landing pages most (67% of tests), then product pages (29%), and 83% of tests pitted just two versions.
- of finished tests ended within a week
- 51%
- ran two weeks or longer
- 33%
- tested a landing page
- 67%
- tested just two versions
- 83%
Half of A/B tests ended within a week
The median finished test ran 7 days. 19% ended within a day, and 33% ran the two weeks most guides ask for.
- Under 1 day · 46% of stores19%
- 1 to 6 days · 46% of stores32%
- 7 to 13 days · 36% of stores16%
- 14 to 27 days · 35% of stores14%
- 28 to 55 days · 27% of stores10%
- 56 days or more · 23% of stores9%
Base: every finished test, Nov 2023 to Nov 2026, from the day each test started to the day the store ended it. Each band holds tests from at least 20 stores.
Over the last 12 months it was 55% within a week, across hundreds of stores. A week is rarely enough: at a 3% conversion rate, catching a 20% lift takes about 14,000 visitors on each version. The sample size calculator gives the number for your store.
Two in three tests were on a landing page
81% of stores tested a landing page, a page built for one campaign or offer. Product pages came next. Collection pages were rare, but those tests ran more than twice as long.
- Landing pages · 81% of stores67%
- Product pages · 38% of stores29%
- Collection pages · 6% of stores3%
Base: every finished test, sorted by the address of the pages in the test. Under 1% tested another page type, from too few stores to show.
| Page type | Share of tests | Share of stores | Median run | Ran 2 weeks or more |
|---|---|---|---|---|
| Landing pages | 67% | 81% | 6.3 days | 31% |
| Product pages | 29% | 38% | 7.7 days | 35% |
| Collection pages | 3% | 6% | 15.4 days | 54% |
A small group of stores ran two-thirds of all tests
Most stores tried a test or two: 40% finished exactly one. The 14% that finished ten or more ran 67% of every test in the study.
- 1 test · 40% of stores6%
- 2 to 3 tests · 27% of stores10%
- 4 to 9 tests · 19% of stores17%
- 10 or more tests · 14% of stores67%
Base: every finished test. Each store counts once, in the group for its own number of finished tests.
83% of tests had two versions, and 90% of those split traffic 50/50. 50% of the tests stores set up were never started.
What to test first
Start where most stores start, then give the test the time it needs to answer.
Start with a landing page
81% of stores tested a landing page, more than any other page type.
Plan the visitors before launching
At a 3% conversion rate, catching a 20% lift takes about 14,000 visitors a version.
Run two full weeks
Only 33% of tests did. Two weeks counts every weekday and weekend twice.
Test two versions, split evenly
90% of two-version tests split 50/50, so each version fills fastest.
Need ideas? See A/B test examples for stores. When a test ends, check it with the statistical significance calculator, or see how A/B testing works in Replo.
Run a test that finishes
Ask Replo for a new version of any page. It splits visitors on one link and shows each version's orders.
How we counted
- Page A/B tests Shopify stores ran with Replo's page builder, started from November 2023 and ended by November 10, 2026.
- A finished test has a start date, an end date and at least two versions. Tests still running aren't counted.
- Run length is the time from the day the store started the test to the day it ended it.
- Page type comes from the page addresses in the test: /products/, /collections/ or /pages/ for landing pages.
- Internal, demo and test accounts are left out. Every figure rests on 20+ stores, and no store is named.
- Conversion results stay in each store's report, so this study doesn't show lift or which version won.
Cite this study
Replo, "What stores A/B test, and how long they let tests run," replo.app/reports/ab-test-results
Link to this page when you quote a number.
More studies: where brands send their ad traffic and how long brands keep Meta ads running.