What marketers need to know

There is a misconception that digital analytics reports are inaccurate. In reality, they are highly accurate in their own way, just not precise. The issue lies in users who don’t know what the analytics data means or how it is gathered. To make matters worse, different tools measure things differently but call them by the same name. 

In this article, we’ll take a closer look at nuances in data measurement and how various analytics software are in action.

Looking at nuances in data measurement  

Digital analytics tools were never intended to work as accounting systems or sales registers. They were made to collect and quantify interactional user data into easily usable insights and reports. Over the years, these tools’ data collection methods have evolved. In turn, the way specific data points are measured also changed. 

Let’s say you changed your tape measure from imperial (measuring in inches) to metric (measuring in centimeters). The length of a desk might be reported as 39.4 in one and 100 in the other. The length of the desk didn’t change, but how you measured it has. 

Try switching between different analytic tools. Often, you’ll see that your numbers may be different, but trend lines remain similar. Each tool counts things slightly differently; the same issue frequently applies when upgrading software.

At one point, unique users were counted by combining the total number of unique IP addresses that accessed a website in a given period. Eventually, organizations started using firewalls/proxy servers, requiring all internal users to access the internet with a single IP address. How unique IP addresses were counted didn’t change, but the count of unique users dropped dramatically.

Counting of unique users evolved into using a combination of IP address, OS and browser (type and version), then the addition of a persistent cookie to better estimate unique users. Once again, no matter how you count unique users if the user cleared their cookies and cache or switched computers (office vs. home vs. phone), no analytics tool will have provided an exact number. Nowadays, tools take other factors into account when counting unique users..

Dig deeper: Data analytics: Your stack’s past and limitations

How to think of your analytics data

Your analytics software is imperfect because of many factors beyond its control. Users might be blocking cookies or other tracking methods. Internet blips might prevent data from reaching the data collection server. The best way to think of your analytics data is by viewing it as a poll of user activity.

Everyone is familiar with polls at election times. A typical U.S. presidential election poll surveys approximately 10,000 people (or less) out of 150+ million eligible voters (0.006% of voters). This is why when news broadcasters report on the poll results, you hear something along the lines of “This data is accurate within 4 percentage points 4 out of 5 times.” This equates to it being off by more than 4 percentage points 20% of the time.

When it comes to your digital analytics tools, most analytics professionals estimate the loss of data to be no more than 10% and most likely around 5%. How does this translate into data accuracy?

If your site received 10,000 sessions in a reporting period but for various reasons, you could only capture data on 9,000 sessions, your data would be accurate within a margin of error of less than 1%, 99 times out 100. 

In other words, 99 times out of 100, your data is accurate and 1 out of 100 times, it is off by more than 1%. Simply put, your data is accurate, but it is not perfect (precise) and will not match your sales records.

Such data is more than accurate enough to determine which marketing efforts — SEO, paid ads, sponsored posts, social media marketing, email marketing, etc. — are working and even which ones drive traffic versus drive sales.

Dig deeper: Don’t apply wishful thinking to your data

Analytics in action

While analytics data may be accurate, even being off a small percentage in precision can call your analysis into question. This is especially true when the difference between two data sources changes. 

The key is to monitor the data and, where possible, compare it. If there is a sudden change in accuracy, you need to investigate. For example, was your website recently changed? Was this change properly tagged to capture the data?

A client once added a pop-up to their Shopify account after an order was placed but before the thank you page was generated. Their analytics tool records sales only when the user receives the thank you page. 

With the pop-up in place, the order still went through, but many users didn’t click through the messaging. As a result, a large percentage of sales were suddenly not being captured as no thank you page was generated. There wouldn’t have been an issue if the pop-up appeared after the thank you page.

Below is an example of monitoring sales and orders between Shopify and Google Analytics 4 (GA4). We can see how much data is being lost because of various factors. Using Shopify’s analytics as a record of true sales and comparing it to data collected via GA4, we see the following:

The daily variations in total revenue and orders varied from virtually 0% to nearly 13%. Overall, in these 24 days, GA4 reported 5.6% less revenue and 5.7% fewer orders. This data is accurate, especially when applied to marketing efforts to see what drove the user to the site to make the purchases. 

Should this company use GA4 to report sales? 100% no! That’s what accounting software is for.

If your organization demands even more accurate data, there are methods to push data directly to most analytics tools (server side). This avoids issues with user browsers and cookies. 

While sales data may be more accurate, other soft measurement aspects of user interaction may drop (e.g., scroll tracking). This is a complex and time-consuming method to implement for most organizations. 

You must ask yourself, “is this extra effort necessary just to capture another 2-5% of sales revenue in my analytics reports?”

Understanding your analytics data

Everyone needs to have faith in their analytics data. The key is ensuring your analytics software is installed and configured correctly. Understand that it can’t capture everything. 

Your analytics software simply takes a poll with a sample size of over 90%. This makes the results highly accurate (on target), if not 100% precise (actual numbers).