Flatlined Data, Hollow Dots, and Report Invalidations

MATOMO ON-PREMISE ONLY

If your Visits Overview graph has data on all of the days, but some of the data points are hollow and say “invalidated period” you can skip ahead: go to the section below Troubleshoot Report Data. But if one or more days with zero visits, you will first need to troubleshoot your Raw data. If that checks out okay you can proceed to Troubleshooting Raw Data.

Remember, Matomo works in two stages: Raw Data and Report Data.

1. Raw Data – These are visible in your Matomo interface at Visitors >> Visits Log. They appear nearly instantly when your app or webpage is viewed by a visitor.
2. Report Data – These are all of the other views in your Matomo interface. They will show zeroes until the backend of your Matomo server runs the core:archive. That is the PHP code of Matomo which turns raw data into report data.

This distinction can help determine why certain data points may not appear or why graphs seem incomplete. Read more on What is the difference between raw data and report data? and How Matomo processes and stores data.

Troubleshoot Raw data

If your pages are not sending tracking http requests, then there will be reduced or even zero raw data on certain days.

• To check this in your interface, log in to your Matomo interface and navigate to Visitors > Visits Log. Compare multiple time periods and identify any anomalies. Most websites follow a weekly traffic pattern, repeating every seven days, so check for irregularities in that pattern.

• Better, if you have database access, retrieve the exact daily visit count by auditing a few days of the matomo_log table. This table stores raw tracking data within seconds of Matomo receiving a request:
  SELECT count(*) FROM matomo_log_visit WHERE visit_last_action_time >= '2024-02-09' AND visit_last_action_time <= '2024-02-13' AND idsite=5;

• You can also inspect raw data using the Matomo API. In the Visitors > Visits Log, click the export icon to extract the records.

• Alternatively, you can check the raw data in the API by navigating to Matomo settings Platform > API > Live.getLastVisitsDetails > JSON

Remedy: if any of the above methods show a lack of raw data coming in, you can troubleshoot by checking in the interface for tracking failures: Matomo settings > Diagnostic > Tracking failures.

Also check the browser when you load the website you are monitoring. Open the browser’s Developer Tools and look for errors in the Console.

And while you are in the developer tools of your favorite browser, reload the page you think you are monitoring and look for successful tracking requests in the Browser Developer Tools >> Network Tab. At least one request should contain the parameter rec=1 and idSite for the site number – these indicate that a “pageview” was successfully tracked.

Troubleshoot Report data

If your raw data looks fine, but it seems to not get fully processed into reports data, you can check to see if the invalidations are being processed correctly. An invalidation is just a label that Matomo places on the report of a day, week, month, or year (types 1, 2, 3, 4 respectively in the database tables we will describe below). An invalidation is simply a label that says “this period/site/segment is at least partially uncalculated; during the next execution of core:archive please re-do all of the calculations for it”.

1. SQL query for the database: SELECT * FROM matomo_archive_invalidations;. It’s normal to see hundreds or thousands of rows since this table lists reports that are not yet properly constructed.

2. Inspect the column ts_started: SELECT * FROM matomo_archive_invalidations WHERE ts_started IS NOT NULL ;. There will often be no rows with a timestamp older than 24 hours (started but never completed). If you do see older times, it can mean a crashed archiving attempt, or it can mean a healthy archiving process is taking a long time. A process can successfully run for several days sometimes: the PHP processes of Matomo are robust and reliable usually and can do that without a problem: check for PHP errors or slow queries in the SQL database for more information about what is happening. Strive to have your Matomo set in a way so that it usually finishes all of its core:archive processes before the next scheduled execution of core:archive.

3. Invalidate days with low values or hollow data points in your report graphs: Run the following command to refresh the affected data. This process typically completes in seconds, though it may take a few minutes for large datasets.
   php console core:invalidate-report-data --dates=2024-12-21,2024-12-21 sites=99
   Adjust the site ID and date range based on your specific needs.

4. Allow core:archive to run at its normal scheduled time, or run it manually with php core:archive

5. Inspect the Matomo interface to see the graphs.
   Expected: You should see few or no hollow points except for incomplete periods (including the future). There will be normal amounts of data when you visually inspect the graph.

6. Inspect the invalidations table.
   Expected: It will have few or zero rows.

Optional

The hollow dots on a graph are generated from the following table (you may want to repeat Step 3 to remediate these hollow dots).

SELECT * FROM matomo_archive_numeric_2024_12 WHERE name = "done" AND value = 4 AND idsite =99 ORDER BY period;

Adjust matomo_archive_numeric_2024_12 to match the year and month of interest and set the idsite to an appropriate value.