Old-school community journalism shows: It's a wonderful 'Light'

Video highlights

Christmas speech by Dave Mitchell

David Mitchell gives bleak report at Christmas dinner. (Windows Media File, 540k)

“Journalists are supposed to have a bias,” Mitchell said. (Windows Media File, 1 MB )

Efforts to save the Light. (Windows Media File, 2.8 MB )

Dave Mitchell

Mitchell has had some dark days along the way. (Windows Media File, 940k)

Park rangers call West Marin residents “fruitcakes.” (Windows Media File, 4.5 MB )

Singing Sheriff's Calls song

The Light’s weekly rundown of the Sheriff’s calls is leavened with Mitchell’s droll wit. Here, Stu Art Chapman performs a song comprised of the best of the Sheriff’s calls. (Windows Media File, 1.2 MB )

(All video and photos by the author. For more information about why the author chose this story, listen to this audio account, in Ogg format.)

I was trying to ask legendary editor/publisher David Mitchell the Big Important Question: how to save the soul of American journalism, but the wind shifted direction and the stench made it impossible to talk.

It’s understandable. Mitchell’s second-story offices of the Point Reyes Light (weekly circulation: 4,100 and rising) are in a converted creamery only a country block from the cattle yards that still supply the good people of Marin County with their curds and whey.

I’d been trying to interview Mitchell about the state of journalism today – because not a day goes by without another thumbsucker being inflicted on us, wailing and gnashing about how the whole profession is headed straight for Satan’s jaws.

I take the thick fragrance of manure that chokes out my question to be a rather literal form of cosmic commentary on the whole subject.

Nonetheless.

In the fall of 2004, Mitchell’s one true lifelong love, his newspaper, the Point Reyes Light, teetered on the brink of extinction. The San Francisco Weekly did a story – “Can the Light Stay Afloat?” – about how Mitchell had been steadily draining his meager inheritance to run the paper at a loss, a story that was then picked up nationally – it even made Romenesko.

Then, in a scene straight out of the 1946 weepy Christmas classic, It’s a Wonderful Life, the good townsfolk of Point Reyes, Bolinas, Tomales, Stinson Beach et al., came together and basically passed the hat and bailed him out.

“Almost every day when people wrote in to renew their paper, someone would say, ‘Here’s an extra twenty bucks.’ We got lots of people who just gave us $100. Even people who didn’t hardly have two nickels to rub together stepped up and gave us $5.

“And then sixteen of them wrote in and said ‘Raise your cover price. Take it up to a dollar. We need the Light because we don’t have any city governments out here. And the Light is providing us with a forum for us to work out civic issues.’ ”

Somewhere, Tribune and Cox and Gannett execs are weeping at the thought of having such a devoted readership – one that not only keeps subscribing, but forms fan clubs devoted to your paper and then demands that you raise your price.

At this point, with the fall circulation reports looming, most circulation managers would settle for just not hemorrhaging readers. Heck, some circulation managers would settle for not having to wear an orange prison jumpsuit for the next couple years.

In 1979, the Light won the Pulitzer Prize for community service for its investigative stories on the violent and paranoid Synanon cult. In 1984, Paul Michael Glaser, still fresh from “Starsky & Hutch,” played Mitchell in a made-for-TV movie (with the embarrassing, Skinemax-like title, “Attack on Fear”) based on the book that Mitchell wrote about the experience.

The Light has been famous ever since for its uncompromising integrity. And to this day, the paper has a reputation for being the training ground for strong, accomplished reporters and editors. [Full disclosure: My wife, Janine Warner, worked at the Light from 1991-92. While working on this story, I stayed at Mitchell's house and even took a dip in his real-live Marin County hot tub.]

One more thing: Much of what follows seems to be about weekly newspapers, one of the oldest forms of journalism. Hardly appropriate for the Online Journalism Review, with its readership of the bleeding-edge, newest of the new, budding media moguls. But the parallels between this small print outlet and online news are really rather marked. Mitchell and the Light teach a lesson applicable across the spectrum: Readers can find a news outlet so appealing that they will fight to keep it alive.

Let your readers help you

Running a news site on a budget that won’t buy much more than a roast beef sandwich and a couple turns on the PlayStation is an increasingly necessary skill. The Light’s success is a relevant example.

Big papers can just throw dozens of reporters on a story (the New York Times famously called it “flooding the zone,” while the Los Angeles Times more cynically referring to “unleashing the flying monkeys”). But if a small paper commits too much to a single story, the expense can drag it under.

“They [the big papers] would send a couple reporters out into the field, and they would do research for several months, come back and give you a blockbuster story, a main bar that skipped over six pages and three sidebars and all the rest,” Mitchell said. “You’re asking a lot of your readers to go through all that volume of stuff.

“Because we can only cover a little of the story each week, we keep the story in front of our readers week after week. Because of space limitations, we’re also forced to do that.

“So in the long run, this is why issues in weekly communities don’t quickly come and go – the way they do in so many big-city papers.” Even a small operation like the Light can integrate the Web into its operation. And it’s not by putting all its content on the Web and hoping that readers will go there – it’s by using the Web and its readers in a way that allows it to report on the community better than ever.

A recent hot scandal (well, by the Light’s standards) showed how. The story involved a run-in between tourists and park rangers in which pepper spray was deployed.

“One reader – a bus driver – got a hold of me and said if you go to the New York City Citizens’ Review Board, they’ve done a study on the use of pepper spray, and the dangers, when it’s appropriate,” Mitchell said. “They not only tell you what’s safe and what’s good and what will work and when not to do it, they tell you what the law is and what the police training is.

“Another person came to us and ‘Hey, you don’t realize it, but there’s a Web site for the law enforcement ranger association. You oughta check that out.’ ” Mitchell said. “We went to the Web site and we found that rangers who worked out here … were writing in [and] held Marin County residents in absolute contempt.

“The things they would post on their own Web site – they considered us about like Osama bin Laden, or at least we loved him, if not being part of Al Qaeda.”

The Light’s smart, dedicated blending of its oldest resource (its readership) and the newest (the Internet) allowed it to take what would have been just a small story, at most a funny Sheriff’s Call gone wrong, and turn it into a cause that is having a transformative effect on the community. The story has woken readers up to a festering problem in their midst, and they’re starting to take steps to demand real changes. This is almost a textbook example of real beneficial watchdog journalism.

When you stumble onto something that sparks some real reader reaction, then your project starts to become a thing, where people start talking about your coverage and you keep covering it because everybody’s talking about it. Even if you don’t manage to catch lightning in a jar the first time around, Mitchell points out that there have been many instances where the paper started out looking into one story and wound up going in a completely unexpected direction.

“Researching a news story should be done with the same approach as empirical science,” Mitchell said.

“You set out to prove or disprove that hypothesis, and that gives you the starting point. There’s been many times that we started out, this was our working hypothesis and discovered, no, when we really got into the story, it was something else altogether.

“But at least that gave us a way of asking questions. And then we keep hammering at it.”

Live in the community to report on the community

One of Mitchell’s requirements is that his reporters live in the community – which can be a bit of a hardship because it’s so expensive in West Marin. But he insists – because so many stories will come from just listening to people in the check-out aisle, or at the post office. Many journalists for metro papers are covering communities that they don’t live in – and wouldn’t be caught in after dark.

Bob Cauthorn, former vice president of digital media at the San Francisco Chronicle, pointed out that most of the disconnect between newspapers and their audience has happened because “the practice of modern journalism, at anything from a mid-size market up, takes place over the telephone.”

“If you have your ass on the street where it belongs, you don’t need a focus group,” Cauthorn said. “Simple as that.”

This has important implications for the online world, where the name of the game is to try to find a niche in which you can prosper. As broadband penetration spreads, as more and more cities start creating wireless Web zones to attract businesses, national and international news will arrive via the Web.

“National news? Piece of cake. Anywhere, everywhere. I can get Pope coverage pretty much anywhere,” said Mark Potts, one of the founders of the all-local citizen journalist startup, Backfence.com.

Potts and his investors are betting that as local businesses grow more accustomed to the Web, as more people rely on it for information, there will be a crucial gap opening up that they can fill. A site that tells you how to find a good local plumber, what the Little League schedule is, and what the City Council is doing to try to solve the traffic problem could be a real force. Thus, start-up ultra-local sites could find themselves duking it out with weekly newspapers like the Light.

Invest in your coverage

If you’re reading this, you are probably well aware that plunging circulation figures have collided with corporate demands for 20-30 percent profits and produced a very nasty climate. We’ve all seen the cycle at work – revenues are down, so the newsroom staff has to be cut. Resulting in a thinner, watered-down product, a product the public doesn’t like. So numbers go down even further, budgets have to be cut again, thinner product. Rinse, repeat.

Mitchell took the opposite approach. In the 1990s, he noticed that Mexican immigrants were pouring into Marin County and that even tolerant locals were starting to get a little uneasy. So Mitchell cut his own salary to fund an ambitious series of stories about all the other waves of immigration that had washed ashore (in some cases, literally – they were shipwrecked while on their way to somewhere else).

“We still, ten years later, are getting letters to the editor about this. It was significant. It really told where did the old families come from that made this community, why did they come here, how are their lives different from the relatives who stayed in the old country,” Mitchell said.

The Light’s coverage produced the conclusion that the current wave of Mexican immigration was no different from any of the preceding waves. The immigrants were facing the same difficult journey, the same problems assimilating, the same fear and hysteria over a purported “invasion” that would ruin things, and ultimately the same slow process of integrating into the community.

The stories delighted the old guard families who had been in the area for generations and then helped bridge the gap between the established and the newer waves of immigrants. The stories brought people together, taught them about each other. They provided a staging ground for people to begin to talk to each other.

New models for community news?

Cauthorn envisions weeklies embracing a model where they publish their print version to establish which issues are at the forefront. Then the weekly’s Web site becomes the host for the discussion by the community.

“I think this would be really interesting, because then you have a really dynamic model where you’re flowing readers back and forth between print and online,” Cauthorn said. “The kind of thing that we could do be doing in metro markets, but that would require – oh my God – creative thought.

“Newspapers in their glory days – at the height of the power of modern journalism, in the 60s and 70s, when newspapers really made a goddamn difference – their circulation was exploding,” Cauthorn exclaimed. “Trust me, people who were reading about civil rights stories and Vietnam and women’s rights – these people were not reading fluff stories, you know?

“The assumption that if you align yourself with your readers – somehow or another you’re dumbing down – means that you think your readers are dumb. That’s the inescapable result of that logic.

“And it’s wrong!

“Our readers aren’t dumb. Our readers are great.”

Cauthorn finds an important lesson in the readers’ rescue of the Light, one that he hopes the other news publishers will pay attention to. “This tells you in no uncertain terms, with a kind of heat and passion that I wish existed in the normal newsroom, that our public wants us to succeed.

“Our public wants us to survive. Our public wants us to thrive. Our public wants newspapers that matter.

“Our public is leaving us because we are chasing them away with a stick.

“Point Reyes proves it.”

* * *

More from Bob Cauthorn

A complete transcript of David’s interview with Bob Cauthorn about the Light and journalism today.

Online communities: Growing an Internet garden

For the most part, past media offerings have been a one-way experience. While this was OK in the past, future generations of media consumers are growing up with bigger expectations. They want to interact and communicate. Two-way journalism is a way of reestablishing trust with the public by starting an ongoing, evolving conversation – a community.

Preparing the Soil (Software)

While there are many types of web applications that allow for community (blogs, wikis, etc.), I want to concentrate on forums (also called bulletin boards). All too often, these areas of newspaper sites are overlooked.

On the positive side, forums are usually user friendly, offering people an easy way to keep up with conversations developing online. They’re also familiar to readers because newspapers have used them since the very early days.

They do lack some of the more advanced moderation features, though, like the ability for people to rate other people’s comments.

Chris Willis of Hypergene believes moderation is critical “… as evidenced by the growth and continued success of Wikipedia, Criagslist and eBay. Each plays a strong role in its community when needed to make sure the community is allowed to flourish.

“Message boards are notoriously poor at this for several reasons. They, mostly, resemble a glorified e-mail thread where every voice has equal weighting and flame posts can take on life of their own. It is also inherently difficult to follow any thread.”

Newspapers, of course, aren’t the only ones doing community online. If you look beyond newspaper sites, you begin to see that there’s been a lot of evolution in the tools available to build online community. Slashdot has a complicated Karma system that gives moderation duty to people in batches, letting them rate comments in conversations they may or may not have been involved in. Other sites, like Kuro5hin, allow some of their users the ability to vote to hide comments on their own.

If newspapers are going to grow successful communities online, they need to look beyond just dropping an out-of-the-box forum system on their Web site. They should be developing software that takes the best of what forums have to offer while adding features such as moderation of individual posts. By opening the moderation responsibilities to the site as a whole, you have a greater chance of increasing plants and getting rid of weeds.

Protecting Your Garden (Legal Aspects)

Suppose for a moment that your garden is going good when some miscreant decides to plant poppy or marijuana plants in middle of it. Are you responsible for people planting stuff in your garden? What if the plant/post is libelous or illegal in nature? Are you even more responsible? Even if you weed/edit?

Written to overturn an earlier ruling regarding moderation of posts, Section 230 of the Communications Decency Act says:

“No provider or user of an interactive computer service shall be treated as
the publisher or speaker of any information provided by another information content provider.” 47 U.S.C. § 230(c)(1).

According to Jonathan D. Hart of Dow, Lohnes & Albertson, in Washington D.C., “… the dominant view is that a web publisher does not take on responsibility for message board postings merely because it edits those postings. Instead, a publisher would be libel for the content of message board postings only if its edits gave rise to or aggravated the libel.”

Another area to be concerned with is your privacy policy. While some users think anonymity is a right they have in your virtual community, it might be a good idea to remind them that in some extreme cases, records may be handed over to authorities.

As with most things on the Internet, the legal aspects of publishing user content on your site change frequently. It’s important to stay up to date with legal rulings about publishing online.

For example, Hart noted, “The California Supreme Court is currently reviewing a decision of an intermediate appellate court that concluded that a Web publisher can be held liable for third-party content once the publisher has been put on notice of the libelous nature of the speech. In the view of the appellate court, once a Web publisher is informed that a message board posting is libelous, it would become responsible as the publisher of that posting if it failed to take it down.”

Know the plants (Social Networks)

To grow a garden well, you have to know what type of plants the soil will support. The same holds true for online community – you need to know a little bit about why people contribute.

In Chris Willis and Shayne Bowman’s We Media (PDF), they list the reasons people contribute as:

  • To gain status or build reputation in a given community

  • To create connections with others who have similar interests, online and off
  • Sense-making and understanding
  • To inform and be informed
  • To entertain and be entertained
  • To create

While Internet trolls might fall under the fifth reason, they need to be considered on their own when trying to grow your community. If you see a weed and let it go, it’s going to spread. The same applies to Internet trolls. ‘Don’t feed the trolls’ can be replaced with ‘don’t water the weeds!’

Tending the Garden (Managing Community)

If you run one of the popular forum scripts, a lot of the moderation will come down to one or a few people. With the size of newsrooms around the country still shrinking, most publishers don’t want to dedicate one person to babysit a forum and actively moderate it. The problem with that is that any decent garden needs to be tended to constantly. One idea that might work in this case is that of volunteer moderators. Think of them as day laborers who work free for a portion of the harvest (the conversations).

If you spend more than a week looking at your forums at least once a day, you’ll soon notice who the ‘regulars’ are and start to form opinions about the personalities of the people posting. If you’ve followed the forums growth from the beginning, you’ll also know who the old-timers are. If you haven’t been around since the launch of the forums, looking at user registration dates and activity will give you a good picture of who’s who in your virtual community. You could also just start a thread asking for help, or have the members of the forum nominate moderators for you.

Whether you e-mail them separately or post something transparent in a forum thread, the chances are high that you will have people interested in helping manage the community. Knowing that people are subjective by nature, it’s a good idea to try to choose moderators that represent different ends of various spectrums, be they political or otherwise. I would also recommend getting real names and a phone number from these moderators so you can call and talk to them personally. While not absolutely necessary, it’s a good way to weed out people who won’t take the responsibilities seriously.

After you have a group of moderators, there are numerous ways you can handle their duties. One is to give them generic accounts with names like moderator1, moderator2, etc. Doing it this way, there’s less of a chance their warnings will be taken personally. Also, this gives them a means to respond subjectively (their own account) and an account where they can consciously try to be more objective.

The moderators can also use their forum names to do their moderation. This way is a little more personal, but there’s a chance the moderators will be hunted down and constantly pestered about decisions they made in the past. One of the first conversations you have with your volunteers might be about whether to be anonymous.

Another good idea is to set up a separate area of the forum that only moderators can access. Most forum software will allow you to do this. This can be a place where questionable threads or posts can be moved so the moderators can discuss them with you and the other moderators before taking action. It also helps allow the moderators to bond, forming a mini-community within the community.

In addition to volunteer moderators, reporters are another possibility for forum moderation. You may not have the resources to dedicate one person to growing your community, but if each reporter spent a few minutes on the forums each day, the chance of noticing problems goes up. Also, by participating in the forums, they can find new sources, gather story ideas and get a pulse on what the community is talking about.

I would recommend using their real names so that people see their online presence as an extension of their real world status. A page explaining that the reporters are there in a semi-official manner might be a good idea. That is, explain that on the forums the reporters may be subjective as well as objective when talking about issues. Not everyone agrees, though.

Jennifer Scott, Online Editor at nwitimes.com, wrote in an email, “The Times does not encourage staff members to participate in discussion nor is it banned. I think the reporters and editors would have to be very careful in what discussions they participate and it would be wise for them to avoid topics in which they report.”

On June 8 of this year, The Times posted a message saying that all nwitimes.com employees would be identified as such on the forums and that no reporters or editors would be participating in the discussion.

Even if they don’t interact with the community officially, though, they can still be an extra set of eyes to catch small problems on the forums before they become big problems. Get to the weeds before they grow out of control. As Chris Willis put it, “Know who should not be in your community.”

Speaking of anonymity, there’s something to be said for making forum users use their real names. It’s a tough question. If you force people to use their real names, you might get higher quality posts (or, at least posts not as offensive), but less people might sign-up and participate. If you’re moderating, though, giving people a little anonymity and watching their interaction with the community, you might end up with more signal overall.

Chris Willis on anonymity:

“… there is a natural reward for participants to be more forthcoming if security issues can be addressed properly. Greater disclosure can result in greater trust, reputation and more meaningful collaboration.

“From my observations, anonymity is more than a name. People seem helpless to not share intimate or personal details in their conversation/interactions that they would rarely share with coworkers or neighbors.”

I think at this point it’s best to let the members of your community decide if they want to use their real names or not. Some publishers might try to force people to use their real names to participate as a quick fix for Internet trolls, but if you look at sites like Slashdot and K5, you can see that with the right moderation you can maintain a pleasant plant/weed ratio while allowing anonymity.

Photos from MorgueFile

Non-traditional sources cloud Google News results

OJR readers may remember a September 2004 article by contributing editor J.D. Lasica that suggested a political bias in the popular online news portal Google News. Searching on the term “John Kerry,” Lasica cited several stories from “second-tier” online-only news and commentary sites that appeared to have a conservative tilt. Among them were headlines such as “Swift Boat Veterans for Truth Expose John Kerry’s Lies.”

Lasica’s piece got me thinking about ways to measure bias in search results. His observations became the basis for my recently completed master’s thesis, the findings of which may be of interest given the ongoing debate about the quality of Google News’s sources. While my analysis does not indicate an overall conservative or liberal slant, it does confirm Lasica’s suspicion that non-traditional news sources are injecting ideologically biased articles into Google News search results. The data show that articles returned in Google News searches are significantly more likely to have an ideological bias than those returned in searches on Yahoo News. (See below for detailed study results.)

The study examines search results for evidence of bias. By analyzing the content of articles returned in searches on the major-party presidential candidates in the days leading up to the 2004 election, it aims to assess the aggregator’s level of political bias. The study looks at balance within these articles as an indicator of bias, using results from the same searches on Yahoo News as a benchmark.

Notably, almost all of the additional bias in articles returned by Google News searches can be attributed to the site’s use of non-traditional news sources. In other words, if we consider only sources affiliated with old-media companies, the average bias scores for articles on Google News and Yahoo News are virtually identical.

Google News, still in beta three and a half years after its launch, tracks the top stories on some 4,500 English-language news sites, updating its index roughly every 15 minutes. The ability to effectively search this huge collection of timely information has helped make Google News one of the Internet’s most popular news portals, drawing about 5.9 million visitors a month.

But its groundbreaking method of identifying top stories based on how frequently they appear on sites in its index – and doing so entirely without human intervention – put the portal in critics’ crosshairs from the beginning.

That its algorithms are able to automatically determine relative importance of stories and present a front page with top stories in different subject areas has been interpreted by some as an ominous sign that computers will someday make human editors obsolete. At the same time, users have ridiculed bugs that cause the site to occasionally attach a photo to an unrelated article or elevate a relatively minor story to a prominent spot on its front page.

(It should be noted that my research concerns only the site’s search results and is unrelated to Google News’s practice of automatically ranking the top stories on its front page and section fronts.)

Because it uses no human editors, Google News has considered itself immune to bias.

“The algorithms do not understand which sources are right-leaning or left-leaning,” Google News inventor Krishna Bharat told Lasica last year. “They’re apolitical, which is good.”

But choosing which sites to index is perhaps as subjective an editorial decision as selecting the stories to play on the front page of a newspaper or website.

Google News does not share the list of sites it crawls, a practice that has resulted in a lot of speculation about its criteria for inclusion and the notion that there might be some ideological imbalance in its list of sources.

In an attempt to shed some light on the question, one blogger has written a script that grabs the news portal’s front page regularly and logs all the sources that it finds. The count stood at 2,256 as of Wednesday night, indicating that about half of the 4,500 sources have been identified.

Along with the mainstream sites in the list are a number of relatively obscure, online-only news sources (some of which are best described as weblogs), including the opinion sites MichNews.com and Useless-Knowledge.com.

Earlier this year, Google News dropped several sites, including the white supremacist journal National Vanguard, from its index after users complained that hate speech was turning up in searches.

It seems the news portal has been making plenty of its own news lately – albeit unwittingly. In March, Agence France-Presse charged in a lawsuit that Google was infringing its copyright by displaying AFP material on Google News pages. Days later, Google announced it would stop using AFP content. Since then, the Associated Press also has expressed “concern” about Google’s use of its material without payment.

And just this month we learned of a patent application filed by Google scientists in 2003, laying claim to methods of “improving the ranking of news articles” based on the “quality” of the articles’ sources – an apparent admission that relevance alone is not a satisfactory measure of an article’s value.

Google’s patent application offers the following variables, among others, as possible measures of a source’s quality: the volume of traffic it receives, the amount of content it produces, the speed at which it responds to breaking news, the size of its editorial staff and the number of bureaus it maintains. Any of these factors would appear to favor traditional media outlets.

If this is an admission that non-traditional sources are of lower quality, how does that square with Google News’s stated goal of increasing the diversity of viewpoints presented on its pages?

Google News currently does not distinguish opinion from fact in its search results (though it now attempts to identify press releases and satire). Hence, editorials and other opinion pieces frequently appear alongside straight news stories in search results. It is not clear that average users can make the distinction between the two, especially given the many online-only sources that peddle a confusing mixture of fact and opinion.

Ranking news stories based on some measure of quality may be a step in the right direction, but to maintain its credibility, Google News needs transparency – both in its selection criteria and its list of sources.


Key findings of the study

I was intrigued by the notion that a site without human editors might still be biased, and I wanted to test it scientifically. To do this, I analyzed the content of articles returned in searches on “George W. Bush” and “John Kerry” in the weeks leading up to the 2004 election. [More complete results and a detailed description of the research process are available in the full study (PDF).]

I wrote a crawler script to retrieve the results from Google News and Yahoo News for the search terms “George W. Bush” and “John Kerry” at four-hour intervals. The program run was during the two weeks preceding the Nov. 2 presidential election, resulting in a total of 80 “snapshots.” Each snapshot contained four sets of search results: “George W. Bush” on Google News, “George W. Bush” on Yahoo News, “John Kerry” on Google News and “John Kerry” on Yahoo News. The program also downloaded the full text of the top articles returned in each result list.

For each of five snapshots, chosen randomly, the first five articles from each of the four result lists were analyzed, ensuring an equal number of Bush and Kerry results and an equal number of Google News and Yahoo News results. This resulted in a sample of 100 articles, which then were examined sentence-by-sentence. Overall, 1,587 sentences were coded in one of five ways:

  • Favorable to Bush
  • Unfavorable to Bush
  • Favorable to Kerry
  • Unfavorable to Kerry
  • Neutral

Using the values for each sentence, two scores are calculated for each article, measuring the degree of the article’s overall favorability to each candidate. These favorability scores could take values of –1 (completely unfavorable) to 1 (completely favorable), with 0 being neutral. For instance, a Kerry favorability score of –0.3 for an article would indicate that, on balance, 30% the content of an article is unfavorable to John Kerry.

Two charts – one for Google News and the other for Yahoo News – provide a basic summary of the data. They show the two candidates’ favorability scores for each article, plotted against each other. This facilitates comparison of the overall favorability of the two portals’ search results.

Favorability plots by news portal

Google

Yahoo

Each data point represents an article, and its placement on the chart represents its favorability to the two candidates:

  • Upper left quadrant: Article is favorable to Kerry and unfavorable to Bush
  • Upper right quadrant: Article is favorable to both
  • Lower right quadrant: Article is favorable to Bush and unfavorable to Kerry
  • Lower left quadrant: Article is unfavorable to both

In other words, articles in the upper right and lower left are more balanced than those in the upper left and lower right. Articles closer to the center are more neutral. The circular boundary is a density ellipse drawn to make it easier to see patterns in the data.

To determine the direction of bias in a particular story, we compare favorability scores for Bush and Kerry. Where they are similar, the article is more balanced. Each article is assigned a balance score, which is the difference between the two favorability scores. A balance score greater than 0 would indicate bias toward Kerry while a negative score shows bias toward Bush. Both Google News and Yahoo News have average article balance scores that are very close to 0, indicating balanced search results. In other words, both the Google News and Yahoo News searches returned articles that were, on the whole, equally favorable to both George W. Bush and John Kerry. This is what we would expect to see of balanced search results at a time when public opinion was pretty evenly divided between the two candidates.

Balance scores

However, the spread of articles’ balance scores reveals an important difference: Articles returned by Google News tend to be significantly more biased in one direction or the other than articles from Yahoo News.

Besides being coded for favorability, articles were also classified by whether they came from an independent, online-only source (such as Salon.com) or a website affiliated with a traditional news source. A traditional news source is defined as a wire service, newspaper, magazine, TV station, radio station, broadcast network or cable network. (Content from one of these sources that is syndicated on a news aggregator such as Yahoo News is also considered traditional.) Of the articles returned by Google News, 40% were from non-traditional news sources, while only 24% of the Yahoo News results came from non-traditional sources.

When articles from non-traditional sources are omitted from the comparison, there is no significant difference in the spread of the article balance scores between Google News and Yahoo News. This indicates that virtually all of the difference in bias between articles returned by Google News and those returned by Yahoo News is attributable to Google’s use of non-traditional news sources.