A journalist's guide to the scientific method – and why it's important

Why should journalists care about the scientific method? I suggested in my post last week that journalism students should take a lab science class to learn about the scientific method. Here’s why I think that’s so important to journalists today.

The scientific method provides a standard procedure through which scientists gather, test and share information. Obviously, part of that should sound familiar because gathering and sharing information is what journalists do, too.

But there are substantial differences between the scientific method and journalism reporting. And while I believe that those differences did not affect journalism’s viability when newspapers had an information monopoly in their communities, our lack of standards for testing information is hurting us in today’s more competitive information market.

Before I go any further, let’s introduce the scientific method, for those readers who aren’t familiar with it. Here’s a good overview of the scientific method:

1. Find a topic or question worth exploring

2. Do some initial, background research to learn about your topic or question. Read what’s been written before.

3. Come up with a hypothesis. This is your best guess of what happened/is happening/will happen, based upon what you already know.

4. Test your hypothesis. You do this by collecting data, either through controlled experimentation or observation.

5. Look at and analyze your data.

6. Based on your analysis, either accept or reject your hypothesis.

7. Publish your information, including all relevant details on how you collected and analyzed your data.

The scientific method evolved over centuries as scientists looked for the best ways to test their theories about why things are they way they are in nature. Ultimately, science emerged from philosophy as scientists settled upon an empirical approach toward testing, rather than replying on “what made sense” to their ideas of logic.

But empirical analysis of information is just one part of the scientific method. Publication and open disclosure play essential roles, as well. Ultimately, the scientific method works because it not only provides a way to test data empirically, but to test others’ tests, as well. That couldn’t happen unless scientists shared their results, and told others how they obtained them.

Anyone who’s done A/B testing on a website design before has used empirical data. But only when you share your results do they become part of public knowledge, and not a mere trade secret.

The scientific method should matter to journalists because it represents humanity’s best method to date for observing and describing the world around us. Frankly, without the scientific method for expanding technical knowledge, the Enlightenment and Industrial Revolution never would have happened.

Journalists are supposed to observe and describe the world around us, too. But our methods don’t begin to approach the rigor of the scientific method. We simply don’t have the same commonly accepted procedures for our work that the science community uses. We do have a code of ethics, which provides guidance for those who choose to follow it.

The SPJ code’s first two lines strike me as most relevant here:

“Test the accuracy of information from all sources and exercise care to avoid inadvertent error. Deliberate distortion is never permissible.”

“Diligently seek out subjects of news stories to give them the opportunity to respond to allegations of wrongdoing.”

Unfortunately, neither the SPJ code nor any other widely accepted procedure in journalism that I know of provides us any instruction on how to test the accuracy of information we receive from sources. And the second half of the SPJ’s statement suggests that the error we most should seek to avoid is distortion.

Ensuring that someone is quoted accurately is something very different than ensuring that what they say is true. Looking through the rest of the SPJ code, one sees a document focused on ensuring that a diversity of sources are included and that they are portrayed fairly and in appropriate context. Those are all noble goals, but don’t raise journalism much above the level of really good stenography.

Of course, to test, one must have a hypothesis to test. But we teach our reporters not to take a point of view in a news story, whether it be our own or one of our sources’. Instead, we are to present multiple points of view, even contradictory ones, and allow our readers to make whatever judgments they see fit.

That method worked well when journalists controlled the flow of information in a community, and we could silence voices by not including them. But that’s not the world we live in today. Communities don’t have a single printing press anymore – they have as many “printing presses” as there are Internet-connected people in that community. A code of ethics designed to promote the flow of accurately quoted information no longer serves a society that’s drowning in information and needs a way to separate the important from the trivial, and the truth from the lies.

Today’s journalism ethics are the ethics of a profession serving yesterday’s information-starved communities. Today, we need a journalistic method that serves communities seeking truth and relevance within the abundance of information surrounding them.

Scientists found a method that allowed them to accurately and truthfully observe and report upon the world. We need to find a method that works for us, as well. It doesn’t need to be the same method that science uses – we’re reporting to different audiences with different daily needs. But we need something that works better than the he-said, she-said, you-fall-asleep stenography too many of us are peddling now.

And peer review will have to play an important role within that, as it does for the scientific method. Journalists double-checking other journalists, as scientists do, will help encourage better journalism and ultimately encourage more public trust in our work. We don’t acknowledge, much less test, each others’ work often enough, and our reputation suffers for that.

Social networking is replacing journalism as the primary method for many sources to deliver information to communities. If journalism is to survive, we must transition from being a medium of information to becoming arbiters of that information. That’s what the public needs from us now.

But to do that, with the accuracy, honesty and truthfulness to which our profession should aspire, we need our own version of a scientific method to guide us in those judgments.

Advice for this year's incoming journalism students

I’m making this an August tradition, so here is my advice for this year’s incoming journalism students. These tips are given to encourage new students to look beyond the typical j-school curriculum, to help them develop the skills that they will need in their future careers, but that j-schools too often fail to teach.

Take at least one laboratory science class

I’m not talking about those general, survey courses that university science departments typically offer non-majors. Cajole your way into an actual lab science class – a physics, chemistry or biology class with a weekly lab session. Learn about the scientific method – how scientists develop a hypothesis and test it using observational data. Then compare and, if applicable, contrast that experience with what you are learning about reporting in your journalism classes.

Much of what passes for “objectivity” in news reporting would be laughed at as naivete in the much more rigorously tested world of science. Learn what objectivity really means by spending some time in a science department. Think about how you might apply what you observe and learn there in your reporting.

Learn how to run a business

Get involved in a student organization where you need to handle cash – raising income from sales and budgeting expenses. Watch how others make decisions about how to get money and how to spend it. Be attentive, and work hard, so that you can move into a position where you have budget responsibilities.

Journalism schools frequently bring guest speakers onto campus. Ask every one of them about the business side of their publication and organization. Learn about the various business models in publishing (and not just in journalism publishing).

Ignore anyone who tries to lecture you about “the wall” or the impropriety of editorial employees knowing too much about the business side of the news industry. Those individuals’ time is past in the journalism field, and while they might be able to teach you some editorial skills, the skills they can teach you are incomplete for what you will need as a journalist.

Get an internship or a part-time job before you graduate. When you do, seek out the people running the business side wherever you work and ask them about their jobs. Ask them what they see happening in the industry and how they are reacting to it. Then ask them how they wish they could react, and what’s holding them back from doing that.

Network

A journalist is only as good as his or her network. Without sources, you are a novelist. (Not that there’s anything wrong with that.) Use your non-journalism classes to build your personal network. Use your student organizations and jobs to build your personal network. Invite guest speakers, professors and fellow students into your network. Publish on Facebook, Twitter, LinkedIn and Google Plus. Blog. Video blog. Make meeting and conversing with people your addiction.

But whenever you do meet and converse with people, online or off, do so with the knowledge that anything you do, say and write may be retold or republished to anyone else. Conduct yourself with intent, with dignity, intelligence and professionalism, at all times. This might be the most difficult of all the tasks I’ve recommended to you. But this is what will earn you the personal respect that you will need to encourage others in your network to invite into opportunities in the future. Fail to network responsibly, and all the smarts in the world won’t help you succeed.

Good luck, best wishes and don’t ever think for a moment – even after graduation – that your education is complete.

ProPublica's outreach a welcome step toward "open-source" journalism

A couple of outreach efforts by ProPublica this week caught my eye as examples of how the Web can make journalism more open and effective — and reminders that both journalists and the public need much more of this.

The first was a post on the ProPublica website Monday offering a “step by step guide” and searchable database for anyone tracing the influence of a nonprofit organization called ALEC that has proven highly effective in developing “model bills” for state legislatures.

The second was a conference call Tuesday that drew about 140 people to hear about using ProPublica-built data and a news application for reporting on education access issues in local schools and districts.

ProPublica published a national story based on the data, examining the relationship of poverty to educational access, along with a Facebook-integrated app for looking up and comparing schools and districts.

These two efforts are moves in the right direction not just for ProPublica but for journalism and the public. By sharing data and making it easy to use, ProPublica produces more value from its deep investments of time and expertise. ProPublica can also benefit from the insights and experiences of others who share or report on the data.

During the conference call, reporter Sharona Coutts, news application developer Al Shaw and computer-assisted reporting director Jennifer LaFleur heard questions, comments and suggestions. Reporters, whose affiliations included both traditional and startup news organizations, also poked and prodded at some of the findings.

As anyone who’s worked with databases knows, data analysis tends to prompt as many questions as it answers. The ProPublica team explained what they’d done to clean up and amplify two major sets of federal data and encouraged reporters to add their knowledge and mash up the new data with other sources. ProPublica also emailed followup links later to those on the call.

This kind of nitty-gritty, story-specific journalism discussion has generally occurred mainly among a limited subset of journalists through specialized skills organizations (such as Investigative Reporters and Editors), in training seminars or in members-only settings. ProPublica’s model shows the promise of opening up that discussion much more broadly — not just among journalists, but for public view of how journalism is done.

Richard Tofel, ProPublica’s general manager, told me that transparency and public engagement have been part of the core discussion at ProPublica since its launch in 2008. In the past year ProPublica has accelerated its social media push, growing Twitter followers by more than five times (55,883 as of this morning) and Facebook friends by more than three times (20,280).

ProPublica has as much competitive DNA as any news organization. Yet Tofel and Editor in Chief Paul Steiger note that their decisions to share databases and expertise don’t have to pass muster with corporate owners or stockholders.

Last year, a ProPublica collaboration with several other news organizations on a project called “Dollars for Docs,” showing pharmaceutical company payments to physicians, expanded its impact after the initial series by sharing and inviting further use of ProPublica’s data. Eventually, dozens of print, online and broadcast outlets drew on the database to produce stories. ProPublica’s “tools and data” page shows other examples.

Given ProPublica’s mission to “make change,” Tofel said, anything that extends the organization’s reach is worth trying.

“That tends to drive us toward open source and it tends to drive us toward sharing,” Tofel said, “and it tends to drive us toward wanting people to follow up on our stuff.”

ProPublica benefits from such followup as its work is credited broadly and its databases and stories are linked off other sites. Social media efforts like the #muckreads feature launched recently (Tweet stories using the #muckreads hashtag and ProPublica considers and aggregates on its site), along with news apps and story links, can help boost traffic to the ProPublica site, now at about 300,000 monthly unique visitors and 1 million monthly page views.

The Web, of course, offers many resources for learning about journalism. Poynter has greatly expanded its online training and knowledge-sharing, through blogs and the News University curriculum, and numerous journalism/media blogs publish spot reports, opinion pieces and guidance that fuel shared learning. Foundation and university-led institutes and websites keep up a steady stream of conversation about ideas and practices. And professional organizations play varying roles in learning for members, with IRE standing out as a leader.

ProPublica adds a new dimension as a news organization sharing its resources directly.

The Web and social media channels also are rich in open discussion and knowledge sharing about some aspects of news and information online — data analysis and visualization, use of social media, new tools and technology. Tech culture is intersecting more and more with journalism, and journalism can gain much more from that influence than new gadgets for old ideas.

Journalism researchers Nikki Usher and Seth C. Lewis explored this idea in an article on the Nieman Journalism Lab blog examining how open-source themes emerged in the learning lab portion of the Knight-Mozilla News Technology Partnership. (I wrote here earlier about the partnership, known as “Mojo.”)

“What can open source teach journalism,” Usher and Lewis asked, “and journalism open source?”

Their findings outline ways the authors think some of the ideas of open-source software align, or don’t, with journalism: transparency, iteration, standards and collaboration. The Mojo experiment should be a good test of cross-pollination.

I’d like to hear about and share other examples of open sharing of resources that enable public-affairs news and information. Please post examples in comments here or email me using the link above. I’ll report back here.