Skip to main content
Blog

Conversion optimization

A/B testing examples for Shopify

Eight ecommerce A/B testing ideas for your product page and homepage, each shown on two real store pages that made opposite choices.

Start for free

A/B testing examples for Shopify are page changes you show to half your visitors, to see which version sells more. The best ecommerce A/B testing ideas change one thing near the top of a page. Test the hero, the button, the headline, the first product photo, the reviews or the offer. A/B testing in Replo runs these tests on the pages Replo builds for your store.

Greg Linden ran one of the best-known ecommerce A/B tests at Amazon. He built a prototype that suggested items to add, right on the shopping cart page. A marketing senior vice president argued it would distract shoppers from checking out, so Linden was told to stop. He tested it on live traffic anyway, the suggestions won by a wide margin, and Amazon launched them (Greg Linden's blog).

Each idea below appears on two real store pages that made opposite choices, so you can picture your version B. They show ideas to test on your own page, not test results. We don't know whether either brand ran the test.

What is an A/B test on Shopify?

An A/B test shows two versions of the same page to random halves of your visitors at the same time. Version A is the page you have now. Version B changes one thing. After enough visits, you keep the version that sold more.

Diagram: one link sends 50% of visitors to version A, the original page, and 50% to version B, the same page with one change to the headline. Both feed into orders, where you keep the version that sells.
Both versions share one link and the same visitors, so the one change is the only difference.

Change one thing at a time. If version B has a new headline and a new photo, you won't know which one did the work.

Which Shopify pages should you A/B test first?

Start where your paid traffic lands. Stores have finished thousands of tests on Replo. Two in three ran on a landing page, and most of the rest on a product page (what stores A/B test).

  • Ad landing pages: you pay for every visit, so a small lift pays back fast. See how to A/B test landing pages.
  • Product pages: the photo, the offer and the button decide the add to cart. See what makes a Shopify product page sell.
  • The homepage's first screen: it sets the story for shoppers who come from search.
  • The cart: Amazon's test shows the cart can sell more, too. Shopify limits checkout changes outside Shopify Plus, so test the steps before checkout.

1. Should the hero show a person or the product?

Two iPhones. Left: Liquid Death's homepage with a man holding a can over “Peanut butter cup flavored sparkling water” and a Watch video button. Right: LMNT's homepage with a drink packet splashing into iced tea, “Introducing Lemonade + Iced Tea” and a Get Yours button.
Liquid Death leads with a person and the can; LMNT fills the screen with the product and the drink it makes.

The first screen decides whether a visitor scrolls. Liquid Death shows a person holding the can, and LMNT shows its drink packet mid-splash. Test which one keeps more visitors on the page. Measure clicks on the first button or the add-to-cart rate.

A tall can of Liquid Death mountain water standing on a dark bar, with lit bottles on the shelves behind it.
The can in a real place, like this bar, is the scene a person-led hero tests against a clean packshot. Photo: Wikimedia Commons, CC0, by Викидим

2. Should the button name the offer or say Shop now?

Two iPhones. Left: AG1's homepage with a full-width green “Get 55% Off” button under the offer. Right: Ooni's homepage with a small yellow “Shop now” button over a kitchen photo.
AG1 repeats the offer on a full-width button; Ooni keeps a small Shop now button.

AG1's button says what you get: "Get 55% Off." Ooni's button says "Shop now" and lets the photo sell the oven. Test the words first. Then test size and color, one at a time, so you know which change moved the number.

An Ooni pizza oven with a tall steel chimney on a steel table in a sunny backyard.
An Ooni oven in a real backyard. When the photo carries the pitch, a plain Shop now button may be all the page needs. Photo: Wikimedia Commons, CC BY 2.0, by Dale Cruse

3. Should the headline sell a benefit or name the product?

Two iPhones. Left: Catalyst Pet's product page headline “One box. Everything your cat's routine needs.” over a cat photo. Right: Our Place's product page with a pot photo and the headline “Ceramic Nonstick Perfect Pot”.
Same spot on the page: Catalyst Pet says why to buy, Our Place says what it is.

Catalyst Pet's product page opens with a benefit: "One box. Everything your cat's routine needs." Our Place leads with the product's name. A name helps shoppers who already know what they want. Shoppers from a cold ad often need the reason first.

4. Which product photo should come first?

quip's first two product photos side by side: the whitening toothpaste tube alone on white, and the tube on marble beside the words “Gentle, everyday whitening. Fluoride-free, and peroxide-free.”
quip's gallery already has both: a clean packshot first, the benefit photo second.

A/B testing images is the cheapest test on this list. quip's product page shows the tube on white first and the benefit photo second. Swap the order for half your visitors. You build nothing new, since both photos already exist.

5. Should the description use bullets or one claim with proof?

Two iPhones. Left: Our Place's “Why You'll Love It” with three bold-led bullets: Thermakind nonstick, just the right size, lightweight cast aluminum body. Right: LMNT's “More Salt, Not Less.” paragraph above a chart from a sodium study.
Our Place scans in three bold lines; LMNT makes one claim and shows the chart behind it.

Our Place sums up the pot in three bullets, each led by a bold phrase. LMNT makes one claim, "More Salt, Not Less," and backs it with a chart from the research. Test the format before you rewrite the words.

6. Should reviews sit above the headline?

Two iPhones. Left: Catalyst Pet's listicle with “10,228 reviews and counting” above the headline “The highest performing natural cat litter on the planet”. Right: quip's homepage leading with “20% off sitewide” and a Shop button.
Catalyst Pet puts the review count before anything else; quip leads with the sale.

Catalyst Pet's listicle puts its review count above the headline, so shoppers see proof before the pitch. quip's homepage leads with the sale instead. Test where the proof sits, not only whether the page has it.

7. Should the offer be one bundle or a pack picker?

Two iPhones. Left: Catalyst Pet's bundle box with “Bundle price: $41.49, you save $31.25” and the items listed. Right: quip's product page with pack sizes 1 to 4 and a refill plan at $12.
Catalyst Pet shows one bundle and the saving; quip lets shoppers pick a pack size and a refill plan.

Catalyst Pet shows one bundle and how much you save. quip lets shoppers pick one to four tubes and a refill plan. Judge this test on revenue per visitor, not conversion rate. A picker can sell fewer orders but bigger ones. Keep the prices the same in both versions, so the test measures the layout of the offer.

8. Should the ad open a product page or a listicle?

Two iPhones, both Catalyst Pet. Left: the product page with “One box. Everything your cat's routine needs.” Right: the 5 reasons listicle with a review count, a headline and a Shop now button.
One brand, one product, two whole pages an ad could open.

Catalyst Pet sells the same litter from a product page and from a listicle that gives five reasons to switch. Split one ad's clicks between two whole pages like these. It's the biggest test on this list, and often the one with the biggest gap. Tushy ran it on its ad traffic: the landing page converted at 5.5% and the product page at 2.5%. Replo case studies shows more tests like it, with each brand's result.

How many visitors does a Shopify A/B test need?

Each version needs enough visitors to show the lift you care about, over at least two full weeks. That way weekdays and weekends both count. Half the tests stores finished on Replo ended within a week, which is too short for most stores.

Replo's A/B test sample size calculator: a page converting at 3% with 1,000 visitors a day, looking for a 20% lift, needs 13,914 visitors per version, about 28 days.
At 3% conversion and 1,000 visitors a day, catching a 20% lift takes about four weeks.

Put your own numbers into the A/B test sample size calculator before you start. When the test ends, check the result with the statistical significance calculator.

How do you run these tests in Replo?

A/B testing in Replo runs on the pages Replo builds and hosts on your domain.

  1. Ask for the test in chat. Type something like "Test a benefit headline on my product page." Replo copies the page, makes the change and sets the copy beside the original.
  2. Launch the test. Replo splits visitors on the page's one link before the page loads. Nothing flickers, and the URL stays the same.
  3. Wait for the result. Replo calls a winner only at a 95% chance to beat the original. Every version first needs 100 visitors, 10 conversions and a full week.
  4. End the test and keep the winner. Every visitor gets the winning version within minutes, with no republish.
Replo's A/B test results: a 25% off headline leads the original 8.5% to 7.5% on add to cart, an 81% chance to win, so the test keeps running until it reaches 95%.
The new headline is ahead, but at 81% Replo keeps the test running instead of calling it early.

For pages inside your Shopify theme, Replo Theme Builder has its own A/B testing.

What mistakes ruin A/B test results?

  • Stopping early. A version that leads after three days often loses by day fourteen. Wait for the sample size you planned.
  • Changing two things at once. You'll get a winner but not the reason. Test one change, keep the winner, then test the next one.
  • Reading only the average. A change can win on desktop and lose on phones. Check results by device before you ship.
  • Running two tests on one page. The results mix together. Run them one after the other.

Start for free and set up your first test this week.

Frequently asked questions

How long should each Shopify A/B test run?
Run most tests for at least two full weeks, so weekday and weekend shoppers both count. Then keep going until each version reaches the sample size you planned. Many stores settle on two to four weeks.
Which Shopify apps make split testing easiest?
Replo makes it easiest on the pages it builds. It splits visitors on one link before the page loads, so nothing flickers and you write no code. Shopify's Rollouts tests theme changes on Grow plans and up, and pricing apps still work alongside Replo for price tests.
Can I A/B test the checkout on Shopify Plus?
Mostly, yes. Shopify Plus lets you customize checkout much further than other plans, so you have more to test there. On other plans Shopify limits checkout changes, so test the product page, the cart and the offer that lead into checkout.
What is a good conversion rate lift to aim for?
Aim for a lift your traffic can confirm. A page converting at 3% with 1,000 visitors a day needs about four weeks to confirm a 20% lift. Smaller lifts need far more visitors, so smaller stores should test bolder changes first.

Start A/B tests

Replo splits your traffic between two versions of a page and tells you which one sells more.