looking for a few good ideas

  amongst the irregular verbiage

The Spam Experiment - Results

My friend Matt wrote and asked me about the results of this. I hadn't written it up because I kind of blew the last few weeks of the trial, but I wrote out my feelings anyway, and I hope Matt won't mind if I repeat them here:

Just to recap: For the last couple of months I had been responding to "opt out" links on the incoming Spam email that I received, to see if it really did reduce the amount of spam.

This project started when I decided to use Outlook's Rules Wizard to filter out some of the more objectionable spam that arrived in my mail box. I selected specific words and phrases, and spam mail that matched these in the SMTP header would automatically be directed to my junk mail folder.

Up until this point, I was not considering "opting out" of the spam lists. But I think this was when the idea of testing it out occurred to me.

My rules were simple: For any given distinguishable spam vendor, I would respond to opt-out once, then direct all further mail from that sender to a "junk" folder for later collation of results. Any mail that went directly to the junk folder would not be responded to, even if it was from a new vendor whose email contents just happened to comply with my existing filters.

In general, I found there were two kinds of opt out mechanisms: an email address to reply to with "Unsubscribe" or such-like in the header; and a web page URL to click on.

I tried following both, but very soon I determined that the email options all but unuseable, because mail would bounce back with a "recipient mailbox is full" or "invalid address" response. So after a week or so, I stopped attempting to respond to email opt-out links, and followed only web page links.

During the successive weeks, I noticed a number of facts:

  • Filtering purely by sender's email address is largely ineffective. I observed identical spams on successive days, each one identified as a unique spam vendor, with some gobbledegook computer-generated email address.
  • 50% of the spam had no way of contacting the vendor to opt out.
  • 50% of the opt-out-able spam used email address links, which were unuseable (see above)
  • 50% of the web page links were either 404 errors , invalid links, or went to some other home page with no opt-out information available.
  • The remainder did actually provide a valid link to a page where you could enter your email address and submit for opt-out. So, basically, 12.5% of the spam allowed me to see that I had opted out, in that I would get a "you have been been removed, please allow 72 hours for removal from our database etc" or similar.

Feeling incredibly stupid, every morning for two months I dedicatedly typed my email address into spam opt-out pages.

Then I kind of lost control of the trial. In mid-February Lisa and I started working at WonkaTech with our laptops, and I started collecting my email on a different machine. Without my spam filters set up on the laptop, I soon go frustrated with the still considerable amount of spam I recieve, and I would just deleted it from my laptop inbox without saving it or adding to my collect-and-track trial.

This impacted my results at *exactly* the point that I felt I should have been seeing some decrease in the incoming spam levels. I should have tried to reconcile the two instances of Outlook - after all, i was transferring important mail from the laptop to the desktop, why not spam too? I guess I was just too tired to think about it much. So I guess the trial officially stopped at around the end of February.

I was actually collating the results of the trial a few days before Matt wrote.  The results:

  • I started in December with 80 spams a week.
  • January looked much the same as December, maybe dropping to about 45-60 spams/week.
  • Last week of Jan was 37 spams, not bad - an almost noticeable reduction but no orders of magnitude.
  • The next couple of weeks were averaging 40, 50.
  • Then we started working at Wonka and the measured rate really dropped off - I just started automatically deleting them and not trying to follow up the opt out links.

Considering that only 12% of the spam offered me a "successful" opt-out path, I could reasonable expect to only see a 12% reduction in the amount of junk mail - and this kind of matches what I observed.

So, next time someone says, "Don't follow those opt-out of spam links - you'll just end up getting even more spam!" I can say with some authority, "Strictly speaking that turns out not to be the case."

However, practically speaking, it is way too much effort for too little gain to try and opt-out. The relief I felt at just hitting the delete key instead of examining each piece of crap mail for an opt-out option was pretty big.

So now I just "add to junk sender list" and hit that delete key. Maybe in another couple of months I'll report on how much spam I get in my inbox vs. how much ends up in my trash mail folder unseen each week....