Posted by: Jim Adcock | February 18, 2011

Spam Referrers

Spam has become a part of our daily lives. We get it in our inboxes, we get pop-up advertisements when we visit websites.

As bloggers we get spam comments – comments that aren’t really from people, saying generic things that might make sense in many contexts, but that are full of links to spammy web pages, or to pages whose owners have paid for high SEO rankings without caring about the unscrupulous tactics used to get those rankings.

WordPress has done an excellent job with the Akismet comment system, which spots almost all spam comments and holds them for moderation. Yay! This keeps our blogs from being polluted with nonsense and prevents the polluters from being rewarded with backlinks.

(An aside for people new to SEO, aka Seach Engine Optimization – search engines use the number of links pointing to a page as a way to determine a page’s value – someone linking to a page obviously values the information on the page, in essence voting it up. The more links pointing to a page, the more the information on the page is considered essential by the internet community at large, right? Except that companies specializing in SEO know this, and help their clients by gaming the system… planting links all over the place pointing to their client’s web page so that the search engines will rank that page higher.)

And now a new form of spam rears its ugly head – spam referrers.

If you aren’t checking your blog stats (and you should be), you may have missed out on this variation. If you have, then you’ve almost assuredly seen them.

A composite image of a small number of some of the spammy referrers I have had

A referer (a real one) is a web page containing a link to your blog or one of your posts. When someone clicks on that link, you get a page view and the referring page gets logged and displayed on your stats page. The idea of a spam referer is that you see these referrers on your stats, and say to yourself, “There is a link to my blog on this page? And a lot of people are coming from that page to my blog? They must be saying the nicest things about my blog there, maybe I gotta check it out!” And you go there and find an advertisement for something or other that you don’t actually give a damn about, or a come-on for a get-rich-quick scheme, or anatomy enlargement, or perhaps the page loads malware on your system… but gee, that’s funny, there isn’t a link to my blog on this page. What happened????

You have played into the hands of the spammy referer. You’ve been advertised to, or scammed, or infected with a virus. They get you curious and you click on the link, and they win.

If I thought it was needless to say, I’d not say it… but it needs to be said: don’t click the link.

Now, I’m sure a lot of these spammy referrers are like search spiders, automated programs that search out blog posts and send a page request to the web server with the phony referer attached as part of the request. Which means that any click you get from one is likely to not have been actually seen by a person. The WordPress support staff clued me into another possibility – that the phony referer might be coming from a real person visiting your site legitimately – but that person’s browser had been hacked to put phony referrers in the logs of any web host that the unsuspecting user browsed.

WordPress wants you to report any spam referrers you find, but don’t post the link in the WordPress support forum. Send an e-mail to WordPress support listing the referring link.

Not sure how to spot a spammy referer? Read this on the support forum.

The problem has gotten a lot worse recently.

Daily count of spammy referrers. On the left, the occasional bump. On the right, even discounting the big spikes, you can see that the activity is much more frequent.

So far this year, 8.5% of the page views on my blogs are from spammy referrers. From 2009-2010, spammy referrers were only about 1.6%. Worse, my overall traffic is up this year – I’m averaging more than twice the number of hits per day this year than last. That means that phony “page views” are up a whopping 1449%, from .43 per day to 6.23 per day. Unless something is done, I foresee 37 per day before the end of the year.

Cumulative count of spammy referrers. See how the accumulation is accelerating?

So what harm is there from bad spammy referrers?

Well, first of all, there is the risk of a blogger clicking on a phony link and rewarding the spammers, or becoming infected with malware.

There is hit count inflation. The page views count is supposed to mean something, the number of people who you have informed, illuminated, or entertained with your writing. The object is not to get a bigger number just for the sake of the bigger number, it is what that number is supposed to represent that is what is important.

There is also the impact your ability to get feedback on your work and your promotion of it. If your posts are getting twice the hits that you used to get, does that mean you are doing the right things, or does it mean your blog is being inundated by fake page views?

Then there is the overhead for WordPress.com (or your web host if you are self-hosting). These spammy referrers are a request to your web host to serve up data (in this case the web pages containing the blog entries). Let’s say (for the sake of explanation) that WordPress.com hosts all of the affected blogs, and has just enough computers to serve the current traffic. Let’s also say that no blogger is going to abandon their blog, but no blogger is going to increase their traffic either; essentially, legitimate traffic is going to stay exactly the same. If, by the end of the year one third of my traffic is phony page requests as I project, and if I were an average blog (between the Scalzis of the world and the blogger who only gets read by their family), then in order to handle the increase in traffic from phony page requests, WordPress would need to buy half again as many computers as it already has. Just to serve phony page requests that no human is going to look at! How long do you think blogging would be free with that kind of cost growth?

Jim Adcock is a SharePoint Administrator, and blogs about SharePoint at his main blog, Working It Out. He is also Vice President of Launch Pad Job Club, an organization in Austin, Texas, whose mission is help people who have lost their jobs to get the skills they need to land their next job, and to help them cope with the interim between jobs. Consequently, Jim also blogs about career management. He also serves as Secretary on the Austin Software Process Improvement Network Board of Directors. He also wants to know why everyone keeps asking him about sleep…

Advertisements

Responses

  1. Thanks for this updated write-up on spam referrers. Way annoying to say the least!

    • Definitely. Just not clicking on the links is the only thing I know of to discourage the spammers, but unfortunately, it will probably only make them do it more to make up for the clicks they don’t get. Sheesh.

  2. Thanks so much for this. I recognized the temptations and the dangers right away, although I am old and computer illiterate. However, I still cannot figure how to recognize the spam url’s, or more importantly, the legitimate ones, because I got my hand spanked when I tried to click on the above link (read this on the support forum). Anyway, I have been to the support forums, once and read that I can learn how to detect them from practice. Seems like sink-or-swim, to me. Hate sinking. Could you send me the url, instead? (But not to the info I mentioned above, about trial and error! Hate error!) Thanks.

    • It looks like, as of today, that WP has decided to remove all URL shorteners (bit.ly, ow.ly, to.ly, alturl.com, is.gd, tinyurl.com, and so on) from everyone’s referrers lists. I am disappointed that they decided to implement this without listing those referrers under “Other Sources” so that I can accurately (stats nerd that I am) how much traffic I am getting from fake people, but at least it will discourage the spammers from using that technique. This is a good thing, because they had gotten quite aggressive – 16 to 18 hits per blog from these guys, when the blogs in question get fewer legitimate page views that that.

      As far as being able to identify spam URLs in your referrer list, for me it helps to have thematic blogs, so that I know that the “stock portfolio” URL is phony – none of my posts are investment-related, so it makes it obvious.

      Other than the URL shorteners, though, the spammers haven’t done too much to hide who they are. If the URL hints at sex, quick profits, some sort of product, or something funny, it is likely phony. If it looks like a blog post but the URL isn’t constructed like a blog post, it’s probably spam.

      If the page purporting to link to you is a blog (or is pretending to be a blog) and you don’t remember posting a comment on it, and you didn’t get a “Like” or a comment on your blog form the other blog’s owner, chances are it is spam.

      Beyond that, if the URL doesn’t make clear what the page is about, it is probably spam. Blog URLs come from the title of the blog entry, and most commercial URLs similarly have a page name that indicates the purpose of the page. If you were in charge of the other site, and it wouldn’t make sense for you to have a link to your blog on a page named, for example, “/how-would-world-looks-if-the-internet-disappear/” (bad grammar aside) or “/about/”, or “/in/48/0/”, then it is probably spam.

      If you are still unsure, you can always send the suspect link to support @ wordpress.com. I’ve sent quite a few suspect links, and when it was a legitimate source, the very helpful support tech let me know when they replied. It may also help to single out the suspicious ones you are not sure about and indicate that you aren’t sure about them.

  3. Thank you for posting this. My blog’s quite diddy and fairly specialist, but I’m still getting a disproportionate amount of these spam referrals.

    I have take your advice and e-mailed support with a short list of offenders; I’ve also suggested that they consider blocking referrals from sites hiding behind a tiny-style URL (I can’t for the life of me imagine why a genuine referring site would use that sort of thing).

    Zalamanda

    • It looks like they have indeed removed URL shorteners from the referral lists. So far it hasn’t reduced the suspicious pageview count, but that should follow shortly, as spammers will change their tactics.

      It looks to me like the spammers were/are using a program to crawl the new posts on the tag search pages and sending a click (or three) for each tag used on a post. We’ll see what new tactic they follow up with in the neverending battle for truth, justice, and a spam-free blog.

  4. […] numbers include removal of spam clicks from the counts.  With the spam clicks added, I passed the two-year total last week. The spammers […]

  5. […] my total to 125. Over 45,000 page views, bringing my total to 67,701 (not including the over 1000 spam clicks). A lot of great stuff happened to make those numbers happen. My series on SharePoint Governance […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: