How do I upgrade a large Matomo instance without losing any tracking data? (no Tracking API downtime)

If you have a large Matomo (Piwik) instance, upgrading to the latest Matomo version may take a long time (if one of the newest Matomo releases include database schema changes).

To upgrade your large Matomo instance, follow the manual three step procedure. This includes putting Matomo UI and Tracker in maintenance mode, and run the core:update command in the console. While Matomo Tracker is in maintenance mode, no data is being collected in the database. Therefore you may lose data for the few hours when Matomo was in maintenance.

This FAQ explains how to import the missing tracking requests in Matomo.

You have two options.

Option 1

During maintenance, visitors on your websites and mobile apps are still issuing the tracking requests. These piwik.php? tracking requests are stored in your Matomo server access logs files. Once the upgrade to the latest Matomo version is successful, you can create a new log file that contains all these piwik.php requests (for the time window that Matomo was in maintenance mode and not tracking).

Then you can replay this log file in Matomo which will backfill all your data: learn more in this FAQ.

Option 2

The second option requires the use of the QueuedTracking plugin with MySQL (or Redis). An advantage of this method is that it will be able to recover 100% of all tracking requests, while Option 1 (using server access logs) will not be able to recover the POST requests.

Caveat: This method is not recommended when upgrading between major releases; use Option 1 if you are going from Matomo 3 to 4 or going from Matomo 4 to 5.

Steps to follow:

1. Install and setup QueuedTracking plugin
2. Enable writing requests into the queue
3. Disable the setting “Process during tracking request”. Check you have not setup a cronjob running queuedtracking:process. (We want to make sure the requests in the queue are not processed).
4. Enable partial maintenance mode by disabling the User Interface (set maintenance_mode=1), but it is important to not disable Matomo Tracking (leave the default record_statistics=1).
5. Start Matomo upgrade
6. Finish Matomo upgrade
7. Disable maintenance mode
8. Process all requests from the queue into DB to backfill all your data (via the ./console queuedtracking:process console command)
9. Disable QueuedTracking plugin again
10. Matomo normal behavior resumes

If you have any feedback or suggestion, please leave a comment below.