For how long should I run an A/B test experiment?

The time it takes for an A/B test experiment to run depends on the experiment, your business cycles, the improvement rate you expect and many other factors such as conversion rates. Once an experiment is running, Matomo (Piwik) will let you know the minimum number of visitors that need to participate in your experiment in order to reach a statistically significant result that will be repeatable and not due to random chance. The amount of needed visitors may change while the experiment is running.

We recommend to let experiments run for several business cycles, for example several full days or even better let the experiment run for several full weeks, as user behaviour changes at night and on the weekends.