A/B Testing User Guide - Reports

With A/B Tests, also known as experiments or split tests, you grow your business and maximize your success by comparing different versions of your website or app to detect the most successful version that increases your sales, revenue, conversions, pageviews, and more.

Filter

Getting started

What is A/B testing? What are examples of A/B tests? Create, view, track and analyse A/B tests I do not have any experience with A/B testing yet, will I still be able to use it?When do I typically need A/B tests?What is Estimated Unique Visitors in A/B testing?

Create and manage

Create, view, track and analyse A/B tests Where can I create, edit or manage my A/B test experiments in Matomo?Change the traffic allocation between variations Schedule an experiment to start and finish at a certain date and time Change the confidence threshold / statistical significance needed to decide a winning variation Define on which webpages an experiment should be activated Create an unlimited number of A/B test Use an unlimited number of variations within an A/B test Create experiments via the Matomo HTTP Reporting API Monitor changes to A/B tests experiments

Run A/B tests

Embed an experiment into my project Perform redirects to compare entirely different pages or layouts We are already using an A/B Testing framework, can we keep using it?Run an experiment only on a subset of my users Run experiments on the server, for example using PHP, Python, Java or any other language I want to run an experiment that needs changes on the server and in JavaScript, is it possible?Run experiments on an Android or iOS mobile app Run an experiment in a marketing campaign such as a banner ad, search marketing ad or an email marketing campaign How is chosen which variation a user gets to see? Is a request to the Matomo server needed?How do we ensure a user always sees the same variation when visiting the website multiple times?Test a specific variation or share a URL for a specific variation Exclude employees or partners views and clicks from affecting my A/B tests results Run A/B test using Elementor to reduce flickering

Analyse A/B tests

Decide which variation within my experiment is the most successful Know which of my variation was the most or least successful For how long should I run an A/B test experiment?Segment Matomo reports by A/B test experiments and variations Apply other segments (such as Country or Marketing campaign) to the A/B tests reports Fetch experiment reports via the Matomo HTTP Reporting API

On-Premise FAQs

Where can I get A/B Testing?Who develops & maintains the A/B Testing plugin?What about the quality of this plugin?Does A/B Testing work as a standalone product?Which Matomo version is required for this plugin?Is any of the tracked data shared with another party?Do I get access to the raw experiments data that was tracked?Under what license is the JavaScript A/B testing framework released?Does the A/B testing framework add a lot of data to the Matomo JavaScript tracker?Enable HyperLogLog to show Estimated Unique Visitors in A/B testing Change the accuracy rate of HyperLogLog in A/B testing