How to run an A/B experiment with Matomo Tag Manager

A/B testing helps you compare different versions of a web page to see which one performs better based on a defined goal, such as clicks or conversions.

When you create an A/B experiment in Matomo, it generates a JavaScript snippet that determines which variation to show when a visitor views a specific web page. The experiment code snippet is typically embedded in the website code and executed as early and synchronously as possible, before any variation-specific content is rendered and before the Matomo tracker sends the initial page view.

Note: Timing is critical. If the experiment loads too late, the visitor may briefly see the default content before it switches to a variation. The visual flicker can disrupt the user experience and affect accuracy of the test results.

To avoid flicker, the experiment code is typically embedded directly in the website’s code before the page view is tracked. Read more about running A/B tests using JavaScript. However, if you’re using Matomo Tag Manager (MTM) to manage tracking scripts, you can still run experiments reliably if the experiment is triggered early enough. With proper configuration, MTM can manage experiments without hardcoding them into your site.

This guide focuses on running your A/B experiment (test) using Matomo Tag Manager for standard websites and Single Page Applications (SPAs).

Before you start

To create and analyse experiments, the A/B Testing premium feature must be available in your Matomo setup.

  • Matomo Cloud: A/B Testing is included in your Cloud plan and ready to use.
  • Matomo On-Premise: Install and activate the A/B Testing plugin available from the Matomo Marketplace.

Create a new A/B Test in Matomo

Follow the step-by-step instructions in the guide on how to create an A/B experiment. It explains how to define the experiment name, goals, variations, and target conditions.

After saving the new test configuration, Matomo generates the JavaScript snippet for you to run the A/B experiment.

  1. In the new A/B Test, go to Embed code and copy the snippet to a text editor if you need to modify the code.
    • If the test shows different content on the same page, define the variation in the code snippet by adding JavaScript inside the activate() function of the variation.
    • If each variation is shown on a separate page with its own URL, you do not need to define any variation changes. It will use the correct page URL configured for each variation.
  2. You will need to use this code in the next step to run the experiment using Matomo Tag Manager.

Configure Matomo Tag Manager to load the A/B Test

When using Matomo Tag Manager, the experiment code must execute first and synchronously, so the browser pauses other script execution until any UI changes, redirects, or JavaScript logic defined in the variation runs without delay. This ensures the visitor sees the correct variation immediately and the correct variation is reported in the matomo.php tracking request.

  1. Go to Matomo > Tag Manager and open the container to edit.
  2. Navigate to Tags and click Create New Tag.
  3. Choose Custom HTML as the tag type.
  4. Paste the experiment code snippet into the Custom HTML field.
  5. Set the Position to Head Start, so the script is injected first in the <head> tag.
    add ab test to custom html tag in tag manager
  6. In the section, Configure when the tag should do this, assign a Pageview trigger to run the experiment when the page loads. If you use a DOM Ready or Window Loaded trigger, a fire delay should be defined on the Matomo tracking tag.
  7. Click on Advanced settings to set the tag’s Priority to a low number such as 1. The default is 999, so using 1 ensures your experiment runs before other tags. Matomo Tag Manager uses priority values to decide the order of execution. Lower numbers run first.
  8. Make sure no other tags (like your Matomo tracker or page modification scripts) run before this experiment tag.
  9. Click Create New Tag to save.

Run an A/B experiment on a Single Page Application (SPA)

A/B test logic must be re-executed when a virtual pageview occurs in a Single Page Application (SPA), because the initial experiment code runs only once on first load.

  1. To re-trigger A/B test logic on each virtual pageview, follow the steps explained above to configure Matomo Tag Manager.
  2. The Custom HTML tag that contains the A/B experiment snippet must be linked to a Pageview trigger and History Change trigger. This ensures the experiment logic runs when the page first loads and every time the URL changes without a page reload.
  3. Set the tag’s Priority to a very low number (e.g. 1), so it executes before other tags.

This approach ensures the correct variation is applied and tracked every time a visitor views a new page in your SPA. Read more on tracking Single Page Applications in Matomo.

Validate the configuration

  1. Enable the Tag Manager Preview / Debug mode and open the browser developer tools to view the Network tab.
  2. Go to the web page containing the A/B experiment.
  3. In the browser dev tools > Network tab, the A/B experiment JavaScript should execute before matomo.php (the tracking request).
  4. There should be no visible flicker on the page.
  5. Ensure that no tracking requests, reload logic, or scripts run before the experiment tag. This will prevent race conditions and ensures the variation is applied and tracked correctly.
  6. Check the Matomo Visit Log report to verify your test entry on the experiment page was correctly recorded.
    view ab test in visit log
  7. When tested successfully, publish the container to your live environment.

Next steps

Once your A/B test has collected enough data, you can use the results to identify which variant performs better based on your chosen goal, such as clicks, form submissions, or purchases. From here, you can decide whether to implement the winning variant as the default experience or run additional tests to refine specific elements.

  • Explore the Matomo A/B Testing guides to learn how to analyse your test results and optimise performance.
  • For technical implementation details and troubleshooting tips, refer to the developer documentation on the A/B Testing JavaScript Framework, which explains how the experiment code works and answers common questions.
Previous FAQ: How do I use the Matomo Tag Manager with SharePoint Classic sites?