USC Annenberg Online Journalism ReviewUSC





Bloggers Declare War on Comment Spam, but Can They Win?

0

Spammers find a way to game Google search results by posting links in comments sections of popular blogs. Now the makers of Movable Type and bloggers are banding together to try to keep real-time interactivity alive in the blogosphere. Here's a look at the battle so far.

Blogger Jeremy Zawodny wrote about "Guerrilla Tactics Against Blog Comment Spammers." Blogdex honcho Cameron Marlow called it the "Comment Spam Arms Race." And blogger/consultant Adam Kalsey said with resolve, "We now intend to fight back."

Just what the heck is "comment spam" and how did it get so many prominent bloggers up in arms? Basically, spammers have been using blogs to help boost their standings in Google searches by posting massive numbers of comments that include links to their pornography sites, scams and get-rich-quick sites. If your site is linked by a top-ranked site or blog, then Google will often raise your site's ranking -- at least that's the thinking of spammers.

In fall of 2003, spammers flooded blog comments, using automated schemes to infiltrate comments on high-ranked blogs such as Dan Gillmor's eJournal, David Weinberger's Joho the Blog and Rafat Ali's PaidContent.org, among others. What followed were a series of measures, counter-measures, tactical warfare and the usual gnashing of rhetorical teeth in the blogosphere. All three of these bloggers had to turn off comments for a period of time and are only tentatively trying out new solutions to keep the comments alive.

Now, Google's Blogger software puts links in comments through a redirected URL, taking away any PageRank boost. And Six Apart's latest TypePad service and Movable Type software include multiple ways to stem comment spam, including Jay Allen's MT-Blacklist and the TypeKey registration system.

For mainstream news outlets just getting into the blogging game, this battle is instructive. While most big-media blogs contain no comments and only run a select list of reader e-mails, others might want to retain the interactivity of comments and the real-time feedback that flows from them. PaidContent's Ali considers the comments part of his site's "ecosystem," and wants people to be able to comment on his more opinionated blog posts.

"For a trade site like mine, I'm trying to create an ecosystem that combines news, comments and networking," Ali told me. "Comments form a huge part of it, and I don't want to miss out on that. If people see you as the source for news, community and deals, that's very important."

The problem with comments, open forums, e-mail discussion lists and other user-generated online content is that it takes work -- usually a moderator or trust system -- to make sure the comments stay relevant, clean and spam-free. In fact, spammers have made old Usenet groups unreadable, have flooded most e-mail inboxes, and have brought their multi-level marketing messages to every open chat room, bulletin board and online guest book they can get their digital paws on.

The following are some highlights and lowlights in the War Against Comment Spam, including spammer strategies and blogger counter-strategies; the words of war-mongering bloggers and defeatists who've thrown in the towel; and links that can help you win your own personal war with comment spam before you go AWOL from your senses.

Manifestos in the Run-Up to War

Prominent blogger and Yahoo technical guru Jeremy Zawodny published his "Guerrilla Tactics" against blog spammers last October. His idea was farcical, asking bloggers to beat the spammers at their own game by creating mock posts about "Cheap Viagra, Vicodin, Penis Enlargement" et al, and then linking like mad to them, driving their page ranks above the spammer sites. But Zawodny's rhetoric is unflinching: "Bloggers, collectively, have so much more Google Juice than these scum spammers that we ought to consider using it as a weapon in this battle."

While his tactic never caught on, Web consultant Adam Kalsey asked his comrades-in-blogging to take more direct aim at the spammers in his "Comment Spam Manifesto," posted last November. Kalsey tells people to track down spammers' ISPs and hosting companies, and even to start a posse to help out others in need.

Kalsey minces no words: "Usenet news succumbed to spam long ago. E-mail was next...What you [spammers] failed to understand is that bloggers are smarter, better connected, and more technologically savvy than the average email user. We control the medium that you are now attempting to exploit. You've picked a fight with us, and it's a fight you cannot win."

Waving the White Flag

While the hawks were calling for all-out battle, other bloggers felt that the fight against spammers wasn't worth the trouble -- and was totally unwinnable. These pessimists felt that spammers were more determined to make money than bloggers were to rid themselves of spam comments.

Programmer Mark Pilgrim, who pens the popular comment-free blog Dive into Mark, wrote about the nascent war efforts last November: "Spammers are smart and determined, and people are numerous and stupid, and spam pays. You can't make it not pay. Going after ISPs won't help; they'll auto-register somewhere else...Going after them in court won't help; they're already living under friendly governments....They will keep coming and coming and coming until you give up, go home, cry uncle, take Prozac."

More recently, the godfather of blogging Dave Winer, former CEO of UserLand, told me that comments are not an intrinsic part of a Weblog and have basically failed after a brief honeymoon period in the early history of blogs. "I think a blog is a publication, and publications have proven that letters to the editor are useful," Winer said. "But blogs with comments are not letters to the editor. Letters to the editor are edited, they're selected, and that selection process is a very important aspect of it."

Instead, Winer thinks commenters should simply run their own blog if they want to comment. While he thinks that all the war tactics by bloggers will ultimately fail, he says that Google itself could solve the problem by adjusting PageRank so that it doesn't weight links from comments as heavily as links within blog posts or on other pages.

The White Knight

As the war was heating up last fall, in stepped Jay Allen, a developer who created MT-Blacklist. The software works by blocking any comments that include known spammers' URLs, which are listed in a community-generated database. If a blogger saw someone attempting to add a spam comment, he could add it to the database. Allen's MT-Blacklist, which works with Six Apart's Movable Type blogging software, won a contest Six Apart held for best plug-in, and the Blog Herald wanted to nominate him for "Blogger of the Year."

Allen told me that the beauty of MT-Blacklist is that it shifts the burden to the spammer, who will have to keep registering more and more domains. "While there are an unlimited number of domains, each one costs the spammer both time and money to register, while it takes me approximately 10 seconds to ban a spammed URL from ever appearing on my site," Allen said. "By so massively shifting the burden onto the spammer, we have done what the Usenet and e-mail spam fighters have never been able to do."

The downside? Like most blacklists, there are always some legitimate sites that end up getting blocked.

Spammer Strikes

In fact, Dan Gillmor detailed one case of legitimate sites getting blocked as a byproduct of comment spam blocking. What the spammer did was devilish, including links to Gillmor's blog itself and the San Jose Mercury News site among its spam site links in a comment. That meant that Gillmor had to take the extra step of checking just what links were blocked to make sure there were no "good" sites in there.

"In this case, they put weblog.siliconvalley.com and mercurynews.com URLs into spams containing their porn and casino links (or whatever it was), and the comment blacklist software picked our URLs up and offered to ban them," he explained on his blog. "We missed this, and the result was that I couldn't post comments here briefly."

(Ironically, Gillmor told me he probably deleted the above post accidentally from his blog while deleting spam.)

Another insidious strategem by spammers is to make a comment that includes a link to another blog that looks innocuous. In actuality, the other blog is crawling with comment spam itself, thereby creating a vicious cycle of self-promotion among unsuspecting blogs. Kalsey struck a familiar tone against lax bloggers who didn't clean up spam in their old comments: "If you continue your apathy and allow comment spam links to linger on your site you are helping the spammers...you are against us."

Just as embarrassing are posts on David Weinberger's Joho the Blog about comment spam. Weinberger temporarily turned off his comments, then triumphantly turned them back on in January with a post titled, "Comments are back." Last week, a spammer appended a comment to that exact post, touting free color business card printing. Ouch.

While it hasn't been as widespread as comment spam, some bloggers are worried that spammers will also target Trackbacks, basically links on blogs that show which other bloggers are writing about them. But rather than talk up the problem, bloggers are hoping the less they talk about it, the less the enemy will target that area of their blog.

Blogger Parries

Over the past year, blog software vendors and bloggers themselves have tried a variety of solutions to pry spam from their comments. Here are some of the more effective strategies employed so far -- along with possible side effects:

1. Turn all comments -- or at least old comments -- off. Not everyone agrees with Winer's view on comments not being essential for blogs. But if they're causing more trouble than they're worth, it makes sense to simply turn them off. Popular group blog BoingBoing turned off comments, and readers started their own discussion group on Tribe.net. Another option is removing or freezing comments on old blog postings, where spammers like to strike. There's even a program called MTCloseComments to do just that within Movable Type blogs.

2. Don't allow links in comments.
This would make it impossible for spammers to score any links on your site. However, turning off links in comments means that any person commenting on your blog would not be able to include links to stories they mention, their own blog postings or other important sources of information online. If links are the currency of the Web, then this idea might bankrupt your discussion.

3. Try a blacklisting service.
If you use Movable Type, the new version 3.1 includes the MT-Blacklist plug-in. That means comments that include links to known spam sites will be automatically blocked. Of course, no blacklist is perfect, and the universe of sites to promote via spam is seemingly endless. But this method will at least keep some repeat offenders off your comments.

4. Redirect all links from your blog comments.
If comment links all are redirected through a special URL, then the offensive URL will get no boost in search engine results. Google's free Blogger software includes such a redirect via the link: http://blogger.com/redirect/?r=[URL to be linked]

5. Include registration steps or a comment preview page.
Any extra steps for comments takes away automated spamming techniques. Some bloggers have tried a "captcha" where people need to retype letters and numbers in an image before writing their comments. Others force every commenter to preview their comments before posting them.

And Six Apart created the TypeKey registration system so that commenters would have a universal log-in for every blog they want to comment on. Plus, Six Apart is providing code so that other software developers can implement TypeKey into their applications, making it a more universal service. Movable Type also has "comment throttling," limiting the number of posts someone (or some robot) can make in a certain time period.

Six Apart co-founder Ben Trott told me that people running high-traffic blogs will start requiring registration for comments. "This can be of benefit," he said, "since it has the dual effect of reducing comment spam and improving the general quality of the conversation because it tends to form a community of visitors around a Weblog."

The downside of registration is that spammers could simply register first and then spam. Plus, registration sets up a barrier for people who want to join the discussion. Usually, there's an e-mail verification for first-time commenters. But most bloggers would rather put a small burden on commenters than bearing the huge burden of fighting comment spam themselves.

6. Spell out the law.
One Irish blogger, Antoin O Lachtnain, was so upset after spending hours removing comment spam that he decided to post a special disclaimer for anyone who wanted to comment on his blog:

"Relevant comments are very welcome, whether you agree or disagree with what I have to say. However, advertising of goods or services is not permitted on this forum without payment of a fee. The fee per advertisement is 500 Euros, which is payable immediately by bank draft. If you post an ad but do not pay the charge immediately you have corrupted data on this Web site without my permission. As such, you are guilty of criminal damage under the Criminal Damage Act, 1991 and subject to a prison sentence of up to 10 years and a fine of up to 12,700 Euros...Please note that posting on this forum will have no effect whatsoever on the PageRank of any links that you post."

Unfortunately, on the very blog post where Antoin first discusses his new disclaimer, all nine comments are from spammers promoting pornography, credit reports, online poker and video on demand.

One of the best resources for fighting comment spam is Elise Bauer's "Learning Movable Type" blog, which has a special page called "Concerning Spam." Bauer has many more technical ideas for fighting spam on Movable Type blogs.

End Game

So, in the end, will comments on blogs survive the onslaught of spammers? Duncan Riley, editor of the Blog Herald in Australia, thinks the dynamic will likely shift in the blogosphere, with mid-tier bloggers abandoning comments altogether.

"Blogging will survive comment spam, however the blogosphere as we know it today will not be the same as a result of it," he told me via e-mail. "The ability to comment on blogs has been a large part of the phenomenal success blogging has had over the last few years. Expect a correction in blog numbers as the industry suffers from burnout, suppliers fold, are bought out or merge, and as the novelty for some wears off from newer distractions, less time and intolerance of comment spam."

 

0
0
Related Links
Antoin: The best and worst about weblogs
Blog Herald: Will Jay Allen be blogger of the year
Blogger
Blogger Help: Keeping Comments Clean
BoingBoing
Dan Gillmor's eJournal
Dave Winer's Scripting.com
Dive into Mark: Weblog spam
Joho the Blog
Joho: Comments are back
Kalsey: Comment Spam Manifesto
Kalsey: New Comment Spam Technique
Learning Movable Type
Learning Movable Type: Concerning Spam
MT-Blacklist
MTCloseComments
Marlow: Comment Spam Arms Race
PaidContent.org
TypeKey
TypePad
Zawodny: Gureilla Tactics Against Blog Comment Spammers
0

Jay Allen, MT-Blacklist developer

0
0

Dave Winer, blogger and former CEO of UserLand

0
0
0