+44 7476 930739 support@ayexperts.com We reply the same working day
★★★★★ 4.9 on Google Agency partner? White-label →
CRO · Guide

A/B Testing for Online Stores: A Beginner's Guide

How to run fair, useful A/B tests on an online store: what to test first, how to set up a test properly, and how to avoid the mistakes that produce misleading results.

YL Yasir LatifFounder, AY Experts Updated 3 min read
In this guide

Key takeaways

  • A/B testing compares two versions of a page with real shoppers.
  • Start with research: test changes that fix real problems.
  • Tests need enough traffic and time to give reliable answers.
  • Low-traffic stores should prioritise fixes over formal tests.

What is A/B testing?

An A/B test shows two versions of a page, or one element on it, to different groups of visitors at the same time, then measures which version leads to more sales or another goal. Because both versions run side by side, outside factors such as seasons or ad campaigns affect both equally.

A/B testing for ecommerce replaces opinions with evidence. Instead of debating whether a new product page layout is better, you find out with real shoppers.

Step 1: Research before you test

The best test ideas come from finding real problems, not copying what other stores do:

  • Analytics: where do visitors drop off between product page, basket and checkout?
  • Heatmaps and session recordings: what do people click, miss or struggle with?
  • Customer feedback: surveys, reviews, returns reasons and support emails.
  • Usability checks: watch a few people try to buy something on your store.

Step 2: Decide what to test first

AreaExample test ideas
Product pagesDelivery information near the price; more lifestyle photos; reviews higher up
Category pagesFilters visible by default; different default sort order
BasketFree delivery progress bar; clearer returns message
CheckoutExpress payment buttons first; fewer form fields
OffersBundles vs single-item discounts

Prioritise ideas by likely impact, confidence from your research and effort to build. Our guides to product pages and cart abandonment are full of starting points.

Step 3: Set up a fair test

  1. Write a hypothesis: "Because shoppers miss our delivery information, showing it beside the price will increase add-to-basket rate."
  2. Choose one main metric, such as conversion rate or revenue per visitor.
  3. Work out the sample size you need with a test calculator before you start.
  4. Split traffic randomly and run both versions at the same time.
  5. Run for full weeks, usually at least two, to cover weekday and weekend behaviour.
  6. Don't stop early because one version looks ahead after a few days.

Test bold changes. Tiny tweaks like button colours rarely produce measurable differences on smaller stores. Changes that fix a real problem are more likely to show a clear result.

What if your store has low traffic?

Formal A/B tests need a lot of orders to detect a difference. If your store doesn't have that volume yet:

  • Fix obvious problems directly, such as broken pages, slow checkout or missing delivery information.
  • Test bigger changes that could make a large difference.
  • Use earlier-stage metrics, such as add-to-basket rate, which happen more often than purchases.
  • Compare before and after carefully, accounting for seasonality and promotions.
Need a conversion plan?Our CRO audit finds the biggest opportunities and prioritises them for you.
Get a free proposal

Reading results

  • Look at your main metric first, then check revenue and average order value.
  • Check results on mobile and desktop separately.
  • Treat inconclusive results as useful: they tell you the change didn't matter much.
  • Record every test, its hypothesis and the outcome, so you build knowledge over time.

Keep your tools privacy-friendly. Testing and recording tools usually use cookies, so make sure your cookie consent covers them and you mask personal details in recordings.

Common mistakes

  • Stopping a test as soon as one version looks better.
  • Testing several changes at once and not knowing which worked.
  • Running tests during big sales or launches that skew behaviour.
  • Testing small details on a store with too little traffic.
  • Not checking that both versions work properly on every device.

A/B testing FAQs

How long should an A/B test run?
Until it reaches the sample size you calculated in advance, and for full weeks, usually at least two, to cover different shopping patterns.
What tools can I use?
Dedicated testing tools, some Shopify apps and server-side testing platforms all work. Choose one that fits your platform and doesn't slow your pages down.
Can A/B testing hurt SEO?
Not if done properly. Run tests only as long as needed, don't show search engines different content from users, and use canonical tags if you test separate URLs.
What should I test first?
The pages with the most traffic and the biggest drop-offs, usually product pages and checkout, based on what your research shows.

The bottom line

A/B testing for ecommerce works when it starts with research, tests meaningful changes, runs long enough and is measured honestly. On smaller stores, fix obvious problems first and test bigger ideas. Over time, a steady testing habit compounds into significantly more sales.

Keep reading

Related articles

All articles
Free proposal

Want a testing roadmap? Get your free proposal.

Tell us a little about your goals and we'll send a tailored plan with timelines and a fixed price. No hard sell, no obligation.

  1. 1
    Instant confirmationYou'll get an email the moment your enquiry lands.
  2. 2
    Reply the same working dayA senior specialist reviews your site and competitors.
  3. 3
    Free proposal & strategy callA clear plan, timeline and fixed price — yours to keep.
No obligation NDA on request Same-day reply 4.9★ Google
Prefer to talk? Call or WhatsApp +44 7476 930739 Email support@ayexperts.com Or book a call in our calendar →
Step 1 of 3About you
Who are you?

What best describes you?

This helps us send the right person and the right proposal.

🔒 Your details are safe with us. We never share or sell your data.