The sunsetting of Google’s Optimize tool left many SEOs wondering how best to carry out A/B testing in Google Analytics 4 (GA4).
While Google OptimizeA/B testing will be missed, its closure provides an opportunity to explore other GA4-integrated solutions for driving experimentation and growth.
In this article, we’ll add some context to Optimize’s removal and explore three integrated A/B testing tools you can use alongside GA4. We’ll also run through a high-level A/B testing process to help you get set up with your chosen app.
Let’s get started.
Optimize, Google’s freemium split-testing and personalization tool, officially shut down on September 30, 2023 after six years of operation.
Image Source: Google
Despite it being helpful for conversion rate optimization (CRO), Google claims that Optimize didn’t have all the features and services its customers “request and need” for effective experimentation.
The sunset’s silver lining?
Google will instead “invest in solutions that will be more effective for our customers.”
More specifically, it’ll invest in third-party Google Analytics 4 A/B testing integrations.
Further reading: Did you know you can get actionable AI-driven advice inside Semrush? The AI Narratives for GA4 app uses your GA4 data to explain insights, reports, and alerts in plain English so you can easily understand your website’s performance.
Why Google Analytics 4A/B Testing Still Matters
Google Optimize’s pairing with GA4 made it a uniquely powerful CRO tool.
It let users inform their website experiments with a range of insightful, accurate data—and then measure outcomes with the context of key performance indicators (KPIs).
The main advantages of supporting website experimentation with Google Analytics are:
- Data-driven decisions. GA offers comprehensive data on user behavior and conversion rates. A/B testers can use this to make informed choices about which changes are most likely to improve click-through rates, bounce rates, and time on page.
- Audience segmentation. Connecting GA4 to your A/B testing tool allows you to import accurate audience data. It means you can conveniently target different segments with your experiments and compare the responses to find the best way forward.
- Measuring ROI: GA’s metrics help testers assess the impact of their SEOA/B tests and determine if the changes lead to improved search engine rankings and increased organic traffic.
Fortunately, Google’s investment in and endorsement of three third-party A/B testing tools means this all remains possible.
3 Ways To Carry Out A/B Testing In Google Analytics 4
To run A/B tests with GA4 data, you’ll now need to use third-party tools and integrate GA4.
Here are three great A/B testing tools with official Google Analytics 4 integrations.
AB Tasty
AB Tasty is a versatile, enterprise-level A/B testing tool that helps some of the biggest ecommerce brands optimize their websites for usability and search performance.
Image Source: G2
Rather than target one user type, it aims to support marketers, product managers, engineers, and developers to enhance user experiences and optimize conversion rates.
Its low/no-code WYSIWYG (what-you-see-is-what-you-get) editor lets users at all levels design tests and determine the best engagement tactics for their websites and apps.
The editor has a vast library of drag-and-drop widgets, from ads and images to promo banners and countdown timers.
Image Source: G2
This simplicity means you can set up and test experimental features for your website incredibly quickly.
How Does AB Tasty Support A/B Testing in Google Analytics 4?
AB Tasty offers both “pull” and “push” integrations with GA4.
The pull integration lets you receive audience data from Google Analytics, so you can easily target different segments with your tests in the AB Tasty interface.
The push integration lets you send AB Tasty campaign data to your Google Analytics property. Then you can analyze your test results and return on investment (ROI) in GA, with your website performance KPIs right there for context.
AB Tasty has detailed tutorials on both types of GA integration, with full setup processes: AB Tasty and Google Analytics 4—Pull integration and AB Tasty and Google Analytics 4—Push integration.
Three More Reasons to Choose AB Tasty
- Feature management: Feature toggling lets you turn experimental features on and off easily and keep track of which ones are live
- Automation: KPI-triggered automated rollbacks undo website changes that don’t deliver, optimizing your site’s performance sooner
- Artificial intelligence: AI-powered traffic allocation automatically directs visitors to the most effective variations on your site
Cost: AB Tasty uses a bespoke pricing model. Fill out the company’s quote form to learn more.
User review: “AB Tasty is a great tool that enables you to create experiments of all complexity. The editor is great for handling simple changes, and the ability to use custom JavaScript and widgets expanded the capabilities. CSM and technical support are great, and overall the package is very well priced compared to competitors.”—Nick R, CXO Lead at a mid-market company [via G2]
Optimizely
Optimizely boasts A/B testing capabilities as part of its Web Experimentation platform.
It, too, has a low/no-code WYSIWYG visual editor that makes the tool beginner-friendly. It means users can test and make changes to websites and apps without needing a developer’s help.
Image Source: G2
As well as A/B testing, Optimizely supports multivariate testing—a method where multiple variables are modified and tested simultaneously. By simplifying this process, the tool speeds up complex experimentation programs so you can get results sooner.
How Does Optimizely Support A/B Testing in Google Analytics 4?
Optimizely runs on the Google Cloud Platform and integrates with various Google products, including GA4.
The GA4 integration allows you to track your Optimizely Web Experimentation tests in the GA4 interface.
Image Source: Optimizely
You can also send specific variations to GA4 as audiences to keep your data consistent and speed up the testing process.
As an added benefit, you can connect Optimizely with Google Tag Manager (GTM) to add and update your own tags for conversion tracking and site analytics. It’s all part of the same integration.
Optimizely walks you through the connection process in its detailed guide: Optimizely and Google Analytics 4. And there’s information on what the Google Tag Manager integration can do in this separate support article: Optimizely and Google Tag Manager.
Three More Reasons to Choose Optimizely
- Program management: Ideation, collaboration, and reporting capabilities allow team members across departments to get involved with testing programs. It’ll help you launch tests faster and build a company-wide experimentation culture.
- Stats Engine: Anadvanced data analysis system, developed in conjunction with Stanford University statisticians, promises more accurate insights and faster, more confident optimization decisions
- Optimizely One: Web Experimentation is just one part of Optimizely’s all-in-one marketing operating system. The platform has fully integrated tools for app testing, content management, ecommerce customer experience optimization, and more.
Cost: Optimizely’s personalized pricing packages are available on request.*
*Optimizely currently (at the time of writing) offers a limited-time platform and migration services discount to new customers switching from Google Optimize.
User review: “The point-and-click web editor makes setting up quick copy A/B tests easy for even non-technical users to do, while the ability to add custom CSS and Javascript allow you to do just about anything you want. The ability to share links to preview individual A/B tests before they’re published makes updating stakeholders easy.”—Devin P, Web Developer at a mid-market company [via G2]
VWO
VWO promotes itself as the “#1 A/B testing platform in the world” and promises to let users run tests in minutes without any help from IT.
Image Source: G2
Like its main competitors, the tool has a WYSIWYG visual editor, increasing accessibility for beginners. Although you can add and edit web page assets using HTML, too.
VWO’s intuitive, easy-to-read testing dashboard shows a range of real-time experiment data and experience optimization metrics.
Image Source: G2
Plans also come with heat maps and site surveys, letting you measure your changes’ impact on user behavior.
Heatmaps are visual representations of data that measure how users interact with your website. Like when they click elements, select certain options, and generally move around each page.
In VWO, they look like this:
Image Source: VWO
Based on your heatmap reports in VWO, you can identify the best on-page locations for your most important links and CTA buttons.
How Does VWO Support A/B Testing in Google Analytics 4?
VWO’s GA4 integration allows you to create GA4 audiences in the VWO interface and import your audiences from GA4 back to your VWO account.
When you connect the two tools, VWO aims to detect the Google tag on your website automatically.
If you installed GA4 in a more complex fashion (e.g., direct or via Google Tag Manager), you can manually specify the details to ensure VWO pushes its data to the correct tag.
Either way,VWO explains the GA4 integration process in this step-by-step guide: Integrating VWO with Google Analytics 4.
Three More Reasons To Choose VWO
- Google Cloud Storage (GSC) integration: Automatically export campaign data to GCS daily, eliminating the need to download CSV (comma-separated values) files from VWO’s dashboard
- Session recordings: Experience your website through visitors’ eyes to spot common drop-off points and user experience (UX) glitches. You can filter sessions by browser, device, or location to analyze users with specific traits.
- VWO Rollouts: Implement website changes in minutes and push them out to specific audience segments based on a wide range of parameters (e.g., user language, new/returning users, search keywords)
Cost: VWO bases pricing for its testing software on Monthly Tracked Users (MTU). A free Starter account is available for up to 50,000 users, with paid plans starting at $200/month (billed annually) and rising to $10,326/month for large enterprises with 5 million MTUs.*
*VWO currently (at the time of writing) offers migration discounts to customers switching from Google Optimize.
User review: “The tool has a great algorithm as the results are very accurate and the improvement sustains itself when the winning variant is deployed at 100%. I also liked the overall feel of the app. It is easy to understand and even the dashboards are designed in a manner that the user can easily digest the data and make decisions.”—Shringarika J, Data and QC Officer at a Mid-Market Company [via G2]
A/B Testing in 7 Steps: The Easy Way to Get Started
New to A/B testing or need a refresher after some time away? Whichever experiment tool you choose, use these steps as your roadmap.
Step 1. Identify Improvement Opportunities
First, measure your website’s current performance with data from:
Semrush’s ImpactHero tool can also shine a light on underperforming and high-performing content on your website.
To see what’s working and what’s not, follow four simple steps:
- Enter your domain
- Install the code using Google Tag Manager
- Wait for the tool to collect some data
- Explore ImpactHero’s insights
Focus your A/B testing efforts on underperforming content. And check ImpactHero’s recommendations for improvements to help you achieve your content and SEO goals.
Looking at Google Analytics data for the first time?
It’s easy to feel intimidated.
Semrush’s AI Narratives for GA4 app helps you make sense of the metrics. It uses artificial intelligence to turn GA data into actionable insights you can use to inspire experiments and improve your website.
Just import your data and get conversational insights to your inbox daily, weekly, or monthly. Like this:
Step 2. Find Your Variable
The data from your analytics tools should provide plenty of inspiration for variables to test.
When starting, it’s best to focus on one variable at a time. Then it’ll be easier to spot the reasons for changes in performance.
Here’s an example:
If all your top-performing blog posts feature numbers in their titles, you could run A/B tests on older, poorly performing articles by rewriting the titles to include numbers.
Then track page views in Google Analytics to see if the change improves engagement.
In this instance, your variable is the title format.
Or, to improve your conversion rates for an email list sign-up form, you could experiment with different call-to-action (CTA) messaging on your landing page to find the most persuasive wording.
Here, the CTA is the variable.
Step 3. Decide on a Test Hypothesis
Your test hypothesis is the idea you want to prove or disprove with your A/B test.
Continuing our earlier example, your hypothesis could be:
“Blog post headlines containing numbers are more compelling than blog post headlines that don’t contain numbers.”
And how will proving or disproving your hypothesis benefit your business? It could be that:
“Incorporating more listicles (articles that typically include numbers in their headlines) in our content plan will grow our website traffic and increase engagement.”
Even if you eventually disprove your hypothesis, you can always tweak it based on what you learn during the testing process. Then start a new, more informed experiment that’s even more likely to help you optimize your site.
Step 4. Set Your Goals and Test Period or Sample Size
Most tests involve measuring several metrics. If you choose your most important metric before you start, you’ll know exactly how to gauge effectiveness.
If you’re testing two new features alongside each other, measuring the same metric from each audience segment (i.e., your primary metric or KPI) will allow you to compare performance directly.
Image source: Goldinlocks
For simultaneous tests, choose a test period that fits your objectives.
The longer you run your test, the more data you’ll collect and the more accurate your findings will be.
On the other hand, the faster you get results, the sooner you can optimize your website. So aim for a balance that suits your objectives and circumstances.
One month is a sensible starting point if you’re unsure.
When testing tweaks to existing content or functionality, ensure you have a benchmark (i.e., how it performs before your changes) to measure the new version against.
Some A/B tests require you to measure performance over time. For instance, you could check a listicle’s views and engagement metrics in the 30 days before changing the headline and compare them to the 30 days after.
Others require sample sizes. Like when testing email subject lines—you could compare open rates for the 500 emails before tweaking with rates for the 500 emails afterward.
Step 5. Create Variations Based on Your Hypothesis
Now configure your “control” and “challenger” versions.
- Control version: The unaltered version of whatever you want to test
- Challenger version: The altered version that will compete with your control
For example, if you run an A/B test to increase newsletter sign-ups, your existing CTA could be your control version.
Like this:
“Sign up for our newsletter today to receive weekly insights”
The new iteration, with different phrasing, would be the challenger.
Like this:
“Receive weekly insights straight to your inbox. Sign up today.”
Comparing how these perform against one another is how you’ll determine the most effective version.
Step 6. Run Your Test
Use your chosen A/B testing tool to run your test and collect data.
Remember that if your objective is to improve search performance by altering a page, it’ll take time for the new version to rank. A solution here is to set up a 302 redirect to your challenger—this is a temporary way to divert users from one page to another.
When you have enough data (e.g., 500 emails from before and after your changes) or your test period ends, compare the control and challenger versions’ results to see which performed better.
Step 7. Analyze the Results and Plot Your Next Steps
Use the test data to guide your next steps.
If your challenger outperformed your control, keep the change and apply what you’ve learned to other parts of your website or digital marketing strategy.
Then keep tracking the impact of your changes to ensure you stay on the right track. While test results are always a strong indicator, there’s never a guarantee that your changes will scale up effectively.
If your test disproves your hypothesis, you’ve still gained some insight into what works and what doesn’t. Consider testing the control against other variations to learn more or move on to different test elements.
Supercharge GA4 with Artificial Intelligence
Whether you’re using it to support your A/B testing or mining website performance metrics for SEO opportunities, Google Analytics 4 is a must-have for digital marketers.
Squeeze more out of this amazing tool and save valuable time with AI Narratives for GA4—it’s a Semrush app that uses artificial intelligence to pull helpful insights from your website data.
