advertisement
advertisement

Shareaholic’s Data Viz Of Declining Reddit Traffic Misleads

A report and chart released earlier this week suggest there’s been a dip in referral traffic from Reddit to publishers. Here’s why you shouldn’t bank your 2014 marketing strategy on it.

Shareaholic’s Data Viz Of Declining Reddit Traffic Misleads

Earlier this week, the content discovery tool Shareaholic published a report claiming that the web traffic Reddit refers to other sites is rapidly declining. A chart showing the dip accompanied the report. Shortly thereafter, the report appeared on several news outlets, including The Atlantic and Business Insider, which recommended that publishers and search engine optimization experts start looking at web sites other than Reddit for referral traffic.

advertisement

Normally I’d ignore a report like this because I don’t care much about SEO, but the analysis in Shareaholic’s report was so bad–and the chart so misleading–that I had to write a response.

It takes three points to make a trend

Shareaholic’s claim rests on their comparison of two cherry-picked data points: Reddit referral traffic at December 2012 and December 2013. (That’s odd, given that Shareaholic has been running since 2008. Why didn’t the company show its data since 2008? Or at least 2010?) Shareaholic compares referral traffic between those two points and notes that it’s gone down by 33%, then speculates that Reddit’s referral traffic is crashing.

One of the things we learn in high school math class is that it takes at least three points to make a trend, not two. In fact, if Shareaholic had posted this report in June 2013, it could’ve told the opposite story: Reddit’s referral traffic is skyrocketing (up 66%)! That’s a sign right there that the methods Shareaholic is using are flawed.

Reddit referral traffic trend, Dec ’12 to Jun ’13. Image: Randy Olson

Their visualization doesn’t even show us if there’s a trend

If we want to talk about trends, we need to consider all of the data. Shareaholic shows us a simple line plot connecting the dots, but doesn’t even try to show us what trend is actually going on. Can you easily tell if there’s really a significant downward trend going on there? That’s the sign of a poor data visualization.

Reddit referral traffic trend, Dec ’12 to Dec ’13. Image via Shareaholic

The trend isn’t even significant

When we take a closer look at the data Shareaholic presented, we can draw a trend line through those points and notice that there isn’t much of a trend going on at all: The line is almost flat, going down just 0.007% every month.

Of course, this trend line assumes that Reddit referral traffic will stay in constant decline, which we know isn’t the case. Just look at what happened in June 2013. There’s a lot more to Reddit’s referral traffic trend than meets the eye, and Shareaholic’s oversimplified analysis doesn’t even come close to explaining it.

advertisement
Reddit referral traffic trend, Dec ’12 to Dec ’13. Image: Randy Olson

Even worse, if we drop the December 2013 point entirely and plot the trend again, the line becomes even flatter (-0.004% per month). If a trend relies so heavily on one data point, then it isn’t much of a trend at all. There simply isn’t enough data here for Shareaholic to make any kind of claim about Reddit’s traffic referral trends–especially the audacious claims made in the news.

Reddit referral traffic trend, Dec ’12 to Nov ’13. Image: Randy Olson

When it comes to sorting out the signal from the noise, this is pure noise that Shareaholic is trying to make look like a signal.

Fractions can be misleading

It’s even strange that Shareaholic presented this data in terms of fraction of traffic referred from Reddit. For example, they say: “During December 2012, sites saw 0.33% of their overall traffic come from Reddit.” Why not report the raw traffic numbers instead of a fraction?

Fractions can mislead when total numbers aren’t reported

It could very well be that even though the fraction of traffic from Reddit went down, the total amount of traffic from Reddit went up.

To drive this point home: Even though 50% seems less than 75%, if we’re comparing 75% of 200 visitors (= 150 visitors) vs. 50% of 1,000 visitors (= 500 visitors), clearly the 50% represents more traffic overall. In other words, even if there was a declining fraction of traffic from Reddit, it could simply be because the websites Shareaholic tracks are receiving more traffic overall.

With that in mind, we can’t even be confident that the Shareaholic referral traffic data actually represents a decline in overall referral traffic from Reddit, as the company so boldly claimed (and publishers rehashed).

advertisement

Shareaholic doesn’t seem to care if its analysis is correct

Before publishing this article, I wrote to the Shareaholic team to explain the faults in its analysis. The company’s response:

Ultimately, our goal is to provide data [that] marketers can act upon, not data that they’d sit around in a room pondering about for hours on end.

It seems that Shareaholic doesn’t even care if its analysis is correct, as long as it offers “data that marketers can act upon.” Are you sure you want your marketers acting upon data that’s as unreliable as this?

About the author

Randy Olson is a Computer Science graduate research assistant at Michigan State University specializing in artificial intelligence, artificial life, and evolutionary computation. He runs a research blog where he writes about Python, scientific computing, evolution, and AI.

More