Short answer: A/B testing shows two versions of a page to different visitors at once and keeps the one that does better on a goal you set in advance. Change one thing at a time, wait until you have enough data, and set the test up so Google doesn’t see two different sites.
What A/B testing is
Google defines A/B testing as testing two or more variations of a change, such as different fonts on a button to see if more people click it.
Nielsen Norman Group (NN/g) calls it a quantitative research method, sometimes also known as split testing. You compare the original design, called the control, with a variation, called the variant.
Ideally, the variant differs from the original in only one element, such as a button, an image or a description.
During the test, the traffic is split so that each visitor sees only one version. Then you compare which one gets more of the action you care about.
Each version can have its own URL, with visitors redirected to it, or the variation can be inserted on the same URL with JavaScript, as Google explains.
A/B testing and multivariate testing
Google describes multivariate testing as testing more than one type of change at a time, to see the impact of each change and how they work together.
Multivariate tests need even more data to give reliable results. If you change several things in one A/B test, you won’t know which change made the difference.
When A/B testing is worth it
A/B testing needs traffic. NN/g says you often need thousands of users to reach a statistically significant difference, which makes it a bad fit for pages with little traffic.
It also needs a goal a computer can count. According to NN/g, it works for projects with one clear, all-important goal, measured by counting simple user actions, such as sending a form.
And it only works for designs that are fully built. You can’t A/B test a sketch.
The biggest limitation, NN/g says, is that you don’t know why you get the measured results. A test tells you which version won, not what people thought.
In our view, most small business pages should start elsewhere. Watching a few people use the site shows why they get stuck, as our guide to usability testing explains. For clicks and scrolls, see our website heatmaps guide.
How to set up a test
Start with a hypothesis based on user research and tied to one clear goal, for example that a clearer button label will get more clicks.
Change one design element, not the whole page. A button label is a common place to start, and our guide to calls to action covers how to write one.
Choose your metrics before you start: a primary metric, which shows whether the variant changes behavior, plus guardrail metrics, which show whether that change helps the business.
Track the goal the same way on both versions. Our guide to setting up Google Analytics 4 shows how to mark your leads as key events.
Work out how long to run the test. NN/g says the sample size depends on three numbers: your current conversion rate, the smallest change you want to detect and the statistical significance you want, usually 95%.
Even with enough traffic, NN/g recommends running the test for at least one to two weeks, to cover normal changes in behavior.
Check the tool first with an A/A test, where both versions are identical. Two identical designs should give an inconclusive result; if they don’t, something in the setup is wrong.
Run the test without hurting your Google rankings
Google says small changes, like the size, color or placement of a button or the text of a call to action, often have little or no effect on rankings.
Don’t show Googlebot something different from what visitors see. Google calls this cloaking and says it is against its spam policies, test or not, and that it can get a site demoted or removed from the results.
If cookies control your test, Google notes that Googlebot generally doesn’t support them, so it sees the version shown to browsers that don’t accept cookies.
If each version has its own URL, put rel="canonical" on the test URLs pointing to the original. Google recommends this over a noindex tag. Our guide to the canonical tag explains how the tag works.
If you redirect visitors to a variant, use a temporary 302 redirect, not a permanent 301, so Google keeps the original URL in its index. Google adds that JavaScript-based redirects are also fine, and our guide to 301 redirects covers the difference.
Run the test only as long as necessary. When it ends, update the page and remove the test URLs, scripts and markup as soon as possible. Google warns that it may treat a test running for an unnecessarily long time as an attempt to deceive search engines.
How to read the result
If the variant clearly beats the original with statistical significance, in many cases it makes sense to use it. If the test is inconclusive or the original wins, keep the original and consider testing another variation.
Don’t stop early. Watching a test in real time and drawing conclusions too soon gives unreliable results.
Expect most variants to lose. NN/g says only one in every seven A/B tests is a winning test.
A result can also be statistically significant without being worth the change, so NN/g advises combining it with qualitative research and business context.
A/B testing after Google Optimize
Google’s free testing tool is gone. Google Optimize and Optimize 360 stopped working on September 30, 2023, and every test still running that day ended.
Google says it is investing in third-party A/B testing integrations for Google Analytics, and it has made its APIs public so any testing tool can connect.
For your Google Ads campaigns, our guide to Google Ads experiments shows how to test a change before you apply it.
Run your first A/B test
- Pick a page with enough traffic and one goal you can track, such as a sent form or a click on the call button.
- Write a hypothesis from what you saw in user tests or heatmaps, and change one element.
- Choose the main metric and the guardrails, and work out the sample size before you start.
- Set it up so Googlebot sees what visitors see, with rel="canonical" on test URLs and a 302 for any redirect.
- Run it at least one to two weeks and until the sample size is reached. Keep the winner and remove the test.
These steps follow NN/g and Google.
Before you start the test
- The page gets enough traffic to reach the sample size.
- The variant changes one element, and you wrote down why.
- The goal is tracked the same way on both versions.
- Googlebot sees what visitors see, and test URLs point to the original with rel="canonical".
- Any redirect to a variant is a 302, not a 301.
- You know when the test ends and who will take it down.
RememberTest one change at a time on a page with enough traffic, and decide the goal before you start. Wait for the result, then keep the winner and remove the test. Show Google what visitors see.
Our web design service builds every plan in WordPress with Bricks, with the tracking set up before launch. You don’t rent the site from us: it stays in your name and keeps running if you leave.
Questions about A/B testing
What is A/B testing?
A/B testing shows different versions of a page to different visitors at the same time and measures which one works better for a goal you set in advance. Google gives the example of trying different fonts on a button to see if more people click it.
How long should an A/B test run?
Until you have enough data. Google says the time depends on your conversion rates and your traffic, and that a good testing tool tells you when the result is reliable. NN/g recommends at least one to two weeks, even with enough traffic, to cover changes in behavior.
Does A/B testing hurt SEO?
Not if you follow Google’s advice. Show Googlebot what visitors see, add rel="canonical" to the test URLs, use a 302 redirect and end the test as soon as you can. Google says small changes, such as a button’s color, often have little or no effect on rankings.
Can a small business website use A/B testing?
Only on pages with enough traffic. NN/g says a reliable result often needs thousands of users, so A/B testing is a bad fit for pages with little traffic. In our view, a small site learns more from watching five people use it than from a test that never ends.
Is Google Optimize still available?
No. Google shut down Optimize and Optimize 360 on September 30, 2023. To test pages now, you need another A/B testing tool, and Google has made its APIs public so those tools can connect to Google Analytics.
Sources
Read on November 7, 2026. From Google Search Central: A/B Testing Best Practices for Search. From Google Analytics Help: [Sunset September 2023] Google Optimize. From Nielsen Norman Group: A/B Testing 101. Putting A/B Testing in Its Place.