Duke University's new Reporter's Lab for investigative tools

When Sarah Cohen looks back at the exhaustive work she and other Washington Post journalists poured into a Pulitzer-winning investigation on child deaths, she sees not just accomplishment but opportunity — to make such work easier, and to enable more of it.

Cohen now is Knight Professor of the Practice at Duke University and director of the university’s new Reporter’s Lab, which aims to be a central resource for developing and sharing technology to improve and simplify the hands-on work of public-affairs reporting. The lab plans to make its software and other resources available to anyone who wants them.

In a recent chat, Cohen told me the project aims to bring technology innovation to in-depth reporting, which she thinks has been left behind even as digital tools have transformed how news is organized and consumed.

Here’s how she put it in a May 16 blog post introducing the Reporter’s Lab:

“For professional and pro-am journalists who specialize in public affairs, the technological revolution passed them by sometime in the early millennium,” continuing that the lab aims “to do for modern reporting what photocopiers did in the 1970s, and e-mail, the Web, spreadsheets and databases did in the 1990s. It will go beyond the hype to test, create, commission or apply new methods to make the hard work of original reporting easier or more effective.”

Cohen’s work also attacks the central question facing accountability reporting, especially the highly valued variety that requires significant time and labor: How do we continue to afford it?

“What I’m looking at is how do we reduce the cost of original reporting without losing anything,” Cohen said

The Reporter’s Lab, which is part of Duke’s DeWitt Wallace Center for Public Policy and related to the center’s computational journalism initiative, has several related goals focused on acting as a central resource for innovation and advances in the core work of in-depth journalism.

“I spent about a year interviewing reporters, editors, technologists, online folks, academic researchers, actual computer scientists, about 100 of them altogether, about what’s needed versus what’s possible using today’s technology,” Cohen said.

She also drew on her own deep experience: 20 years of reporting and editing mostly focused on computer-assisted journalism, including 10 years at the Post.

In a summary describing the lab, formally known as the Duke Project for the Advancement of Public Affairs Reporting, Cohen wrote that analysis of 15 boxes of handwritten forms and other documents for “The District’s Lost Children” series (for which she shared the 2002 investigative Pulitzer) took six months.

“Electronic tools that would have made those documents searchable, extracted the little precise information that was not censored and grouped the recommendations might have cut that effort by a third,” she wrote. “If the analysis were easier more reporters in other cities might have tackled similar projects.”

Part of the lab’s work will be in building, adapting or testing tools for depth reporting. For instance, the lab created a tool called TimeFlow (for reporters to use organizing material on long-running stories), which has been downloaded 1,500 times.

The lab recently hired Charlie Szymanski, app developer and visualization pro. Szymanski worked previously at the National Journal and the Sarasota Herald-Tribune, where his portfolio includes a Knight-Batten Innovation Award winner and impressive work for the paper’s series on fraud in real-estate “flipping.”

Cohen plans to work with Investigative Reporters and Editors to develop a “test kitchen” approach involving the lab’s tools and technology from other sources.

“I think there’s a great need for this,” said Mark Horvit, IRE executive director. There’s no lack of new tools being used and touted, he said, but even the open ongoing exchange at the NICAR-L listserv for computer-assisted journalism sometimes becomes overwhelming with the range of recommendations.

“It’s very hard to keep up with all of that — it’s very hard to know what tools are best for your needs,” he said.

Additionally, as Cohen notes in her project summary, tools for web-scraping, indexing material or doing other key tasks in simpler ways often are too expensive or technologically daunting to be used effectively or broadly by many reporters — especially on deadline.

Cohen plans to build testing and training into the lab and noted that thanks to Duke’s support, “everything that’s being done here is open source and free.”

ProPublica's outreach a welcome step toward "open-source" journalism

A couple of outreach efforts by ProPublica this week caught my eye as examples of how the Web can make journalism more open and effective — and reminders that both journalists and the public need much more of this.

The first was a post on the ProPublica website Monday offering a “step by step guide” and searchable database for anyone tracing the influence of a nonprofit organization called ALEC that has proven highly effective in developing “model bills” for state legislatures.

The second was a conference call Tuesday that drew about 140 people to hear about using ProPublica-built data and a news application for reporting on education access issues in local schools and districts.

ProPublica published a national story based on the data, examining the relationship of poverty to educational access, along with a Facebook-integrated app for looking up and comparing schools and districts.

These two efforts are moves in the right direction not just for ProPublica but for journalism and the public. By sharing data and making it easy to use, ProPublica produces more value from its deep investments of time and expertise. ProPublica can also benefit from the insights and experiences of others who share or report on the data.

During the conference call, reporter Sharona Coutts, news application developer Al Shaw and computer-assisted reporting director Jennifer LaFleur heard questions, comments and suggestions. Reporters, whose affiliations included both traditional and startup news organizations, also poked and prodded at some of the findings.

As anyone who’s worked with databases knows, data analysis tends to prompt as many questions as it answers. The ProPublica team explained what they’d done to clean up and amplify two major sets of federal data and encouraged reporters to add their knowledge and mash up the new data with other sources. ProPublica also emailed followup links later to those on the call.

This kind of nitty-gritty, story-specific journalism discussion has generally occurred mainly among a limited subset of journalists through specialized skills organizations (such as Investigative Reporters and Editors), in training seminars or in members-only settings. ProPublica’s model shows the promise of opening up that discussion much more broadly — not just among journalists, but for public view of how journalism is done.

Richard Tofel, ProPublica’s general manager, told me that transparency and public engagement have been part of the core discussion at ProPublica since its launch in 2008. In the past year ProPublica has accelerated its social media push, growing Twitter followers by more than five times (55,883 as of this morning) and Facebook friends by more than three times (20,280).

ProPublica has as much competitive DNA as any news organization. Yet Tofel and Editor in Chief Paul Steiger note that their decisions to share databases and expertise don’t have to pass muster with corporate owners or stockholders.

Last year, a ProPublica collaboration with several other news organizations on a project called “Dollars for Docs,” showing pharmaceutical company payments to physicians, expanded its impact after the initial series by sharing and inviting further use of ProPublica’s data. Eventually, dozens of print, online and broadcast outlets drew on the database to produce stories. ProPublica’s “tools and data” page shows other examples.

Given ProPublica’s mission to “make change,” Tofel said, anything that extends the organization’s reach is worth trying.

“That tends to drive us toward open source and it tends to drive us toward sharing,” Tofel said, “and it tends to drive us toward wanting people to follow up on our stuff.”

ProPublica benefits from such followup as its work is credited broadly and its databases and stories are linked off other sites. Social media efforts like the #muckreads feature launched recently (Tweet stories using the #muckreads hashtag and ProPublica considers and aggregates on its site), along with news apps and story links, can help boost traffic to the ProPublica site, now at about 300,000 monthly unique visitors and 1 million monthly page views.

The Web, of course, offers many resources for learning about journalism. Poynter has greatly expanded its online training and knowledge-sharing, through blogs and the News University curriculum, and numerous journalism/media blogs publish spot reports, opinion pieces and guidance that fuel shared learning. Foundation and university-led institutes and websites keep up a steady stream of conversation about ideas and practices. And professional organizations play varying roles in learning for members, with IRE standing out as a leader.

ProPublica adds a new dimension as a news organization sharing its resources directly.

The Web and social media channels also are rich in open discussion and knowledge sharing about some aspects of news and information online — data analysis and visualization, use of social media, new tools and technology. Tech culture is intersecting more and more with journalism, and journalism can gain much more from that influence than new gadgets for old ideas.

Journalism researchers Nikki Usher and Seth C. Lewis explored this idea in an article on the Nieman Journalism Lab blog examining how open-source themes emerged in the learning lab portion of the Knight-Mozilla News Technology Partnership. (I wrote here earlier about the partnership, known as “Mojo.”)

“What can open source teach journalism,” Usher and Lewis asked, “and journalism open source?”

Their findings outline ways the authors think some of the ideas of open-source software align, or don’t, with journalism: transparency, iteration, standards and collaboration. The Mojo experiment should be a good test of cross-pollination.

I’d like to hear about and share other examples of open sharing of resources that enable public-affairs news and information. Please post examples in comments here or email me using the link above. I’ll report back here.

What if we are part of the voiceless community?

I hate hypocrites… especially when they’re journalists.

I’ve been a bit disappointed with how some journalists have been writing about Jose Antonio Vargas‘ recent announcement that he is an undocumented immigrant. Many are questioning Vargas’ journalistic credibility because he had to hide his immigration status.

As if journalists – including columnists and editors – have never lied before or broken any laws. (Just think about your college years.)

Jose Antonio VargasLike the communities we cover, newsrooms are filled with sinners and saints… perfectly flawed human beings.

But lies have different degrees, don’t they?

It wasn’t long ago that people had to hide, or lie about, being gay. They had to conceal a part of their true identities to avoid discrimination or to get a job, including one as a reporter.

While they felt forced to hide a part of themselves, something tells me they still made strong journalists and did not lie in their reporting.

If I recall correctly, when the gay marriage issue erupted in San Francisco, The Chronicle pulled a gay photographer off the story because editors assumed a conflict of interest. What Chronicle editors failed to note is that straight people also have opinions about gay marriage that may also pose a conflict of interest.

The bottom line is, as far as I know, Vargas never lied in his stories. And just because he had to hide about being part of a certain community, it doesn’t automatically nullify his journalistic credibility or achievements.

Being a part of a community does not disqualify him as a journalist.

Just like Diane Sawyer, for example, isn’t disqualified as a journalist because she worked in Republican Party politics before, during and after President Nixon’s administration and subsequent resignation. Same as George Stephanopoulos isn’t disqualified after working for President Clinton’s administration.

They are just two of many examples.

What has bothered me the most, really, is how journalists are treating Vargas as “other” … as if his reality is not a common one. As if undocumented immigrants, or illegal aliens or whatever label you use, aren’t part of our communities.

We are all made up of different communities, and these often are the same communities we attempt to cover through our journalism. Some communities we praise, others we tolerate and others go unacknowledged.

I, like many others, believe that a diverse newsroom – comprised of different communities – makes for stronger, more relevant journalism. But the sad reality is that not all communities are seen as equal – or as newsworthy.

Our job is to give voice to the voiceless… but what happens if we are part of the voiceless community?

That’s the position Vargas found himself in. And he, like others from different communities before him, decided to come out and remind people the “other” is really a part of “us.”

About a year ago, I actually wrote a post about this topic, but under advisement from my closest editor I deleted it.

The post was inspired by Harvey Milk‘s powerful message: “You must come out” to give a real face to a community that is under attack. This was around the time of Arizona’s SB1070 bill.

My editor thought my post could be taken out of context and hurt my career.

I don’t know how these words will be taken… and quite frankly, I’ve debated whether or not I should ever publish them… but I hope my editor is not right.

In light of Vargas’ story – one that took more courage to share than my story – I feel that I am obligated to share my experience.

Allow me, however, to frame the reason why I am sharing my story now:

  • I’m not asking for any political action. (Don’t call me an activist.)
  • I’m not trying to ride Vargas’ coattails. (Don’t call me a poser.)

I’m writing this because as journalists we can’t afford to forget that we are part of the “other” … that good journalism is truly inclusive.

I, like everyone else, am part of multiple communities: I am a father, a husband, a renter (former homeowner), college graduate, an educator, a Roman Catholic (but I often disagree with the church) and the son of immigrants from El Salvador.

While my mother entered the country by plane with the right papers, my father entered by crossing the border illegally in the 70s.

He quickly became a U.S. citizen.

But let’s be honest here, the act of an immigrant crossing the border without the right papers in pursuit of a better life often overshadows their accomplishments as legal citizens.

To clarify, I was born a U.S. citizen. But all my success as a person and as a journalist, I owe to my immigrant parents.

My father, like millions of other immigrants, reflects the story of America – whether we want to admit it or not. Coming to this land (by any means necessary) with nothing, working [expletive] hard and making a better life for himself and his family.

For the record, my father graduated at the top of his high school class in El Salvador, which earned him a scholarship to Germany. He worked there, but giving into the request from my mother’s family, he moved to the United States after marrying her.

My father ran several small businesses and was a homeowner for more than 40 years. He lost them in the bad economy, but had relaunched his auto repair shop early last year. He passed away in November and the outpouring of support from the local community was a true testament to his accomplishments. That man helped so many people… I had no idea.

My mother struggled and worked hard in her own immigrant story. She made a small living by cleaning houses and other service jobs, including working at the food court of Cal State University, Northridge. She joined my father as an entrepreneur until they separated.

I can tell you more about their story, but let me just say this: “Their” story is part of “my” story. And “my story” is part of “our” community. And all of that is part of journalism. To shun someone, even a journalist, for owning their story, their community, is bad journalism.

If you invalidate Vargas as a journalist for being an illegal immigrant, you are a journalist in denial thinking that he is not part of your community (the one you are trying to cover).

Again I ask: Our job is to give voice to the voiceless… but what if we are part of the voiceless community?

If you are part of a community that is being attacked or politicized, as a journalist it takes courage to step forward and speak up, not as an activist… but as someone who wants facts to prevail. Not talking points.

I applaud Vargas for his courage. He’s a reminder that the “other” is really within “us.”

Robert Hernandez is a Web Journalism professor at USC Annenberg and co-creator of #wjchat, a weekly chat for Web Journalists held on Twitter. You can contact him by e-mail ([email protected]) or through Twitter (@webjournalist). Yes, he’s a tech/journo geek.