Duke University's new Reporter's Lab for investigative tools

When Sarah Cohen looks back at the exhaustive work she and other Washington Post journalists poured into a Pulitzer-winning investigation on child deaths, she sees not just accomplishment but opportunity — to make such work easier, and to enable more of it.

Cohen now is Knight Professor of the Practice at Duke University and director of the university’s new Reporter’s Lab, which aims to be a central resource for developing and sharing technology to improve and simplify the hands-on work of public-affairs reporting. The lab plans to make its software and other resources available to anyone who wants them.

In a recent chat, Cohen told me the project aims to bring technology innovation to in-depth reporting, which she thinks has been left behind even as digital tools have transformed how news is organized and consumed.

Here’s how she put it in a May 16 blog post introducing the Reporter’s Lab:

“For professional and pro-am journalists who specialize in public affairs, the technological revolution passed them by sometime in the early millennium,” continuing that the lab aims “to do for modern reporting what photocopiers did in the 1970s, and e-mail, the Web, spreadsheets and databases did in the 1990s. It will go beyond the hype to test, create, commission or apply new methods to make the hard work of original reporting easier or more effective.”

Cohen’s work also attacks the central question facing accountability reporting, especially the highly valued variety that requires significant time and labor: How do we continue to afford it?

“What I’m looking at is how do we reduce the cost of original reporting without losing anything,” Cohen said

The Reporter’s Lab, which is part of Duke’s DeWitt Wallace Center for Public Policy and related to the center’s computational journalism initiative, has several related goals focused on acting as a central resource for innovation and advances in the core work of in-depth journalism.

“I spent about a year interviewing reporters, editors, technologists, online folks, academic researchers, actual computer scientists, about 100 of them altogether, about what’s needed versus what’s possible using today’s technology,” Cohen said.

She also drew on her own deep experience: 20 years of reporting and editing mostly focused on computer-assisted journalism, including 10 years at the Post.

In a summary describing the lab, formally known as the Duke Project for the Advancement of Public Affairs Reporting, Cohen wrote that analysis of 15 boxes of handwritten forms and other documents for “The District’s Lost Children” series (for which she shared the 2002 investigative Pulitzer) took six months.

“Electronic tools that would have made those documents searchable, extracted the little precise information that was not censored and grouped the recommendations might have cut that effort by a third,” she wrote. “If the analysis were easier more reporters in other cities might have tackled similar projects.”

Part of the lab’s work will be in building, adapting or testing tools for depth reporting. For instance, the lab created a tool called TimeFlow (for reporters to use organizing material on long-running stories), which has been downloaded 1,500 times.

The lab recently hired Charlie Szymanski, app developer and visualization pro. Szymanski worked previously at the National Journal and the Sarasota Herald-Tribune, where his portfolio includes a Knight-Batten Innovation Award winner and impressive work for the paper’s series on fraud in real-estate “flipping.”

Cohen plans to work with Investigative Reporters and Editors to develop a “test kitchen” approach involving the lab’s tools and technology from other sources.

“I think there’s a great need for this,” said Mark Horvit, IRE executive director. There’s no lack of new tools being used and touted, he said, but even the open ongoing exchange at the NICAR-L listserv for computer-assisted journalism sometimes becomes overwhelming with the range of recommendations.

“It’s very hard to keep up with all of that — it’s very hard to know what tools are best for your needs,” he said.

Additionally, as Cohen notes in her project summary, tools for web-scraping, indexing material or doing other key tasks in simpler ways often are too expensive or technologically daunting to be used effectively or broadly by many reporters — especially on deadline.

Cohen plans to build testing and training into the lab and noted that thanks to Duke’s support, “everything that’s being done here is open source and free.”

ProPublica's outreach a welcome step toward "open-source" journalism

A couple of outreach efforts by ProPublica this week caught my eye as examples of how the Web can make journalism more open and effective — and reminders that both journalists and the public need much more of this.

The first was a post on the ProPublica website Monday offering a “step by step guide” and searchable database for anyone tracing the influence of a nonprofit organization called ALEC that has proven highly effective in developing “model bills” for state legislatures.

The second was a conference call Tuesday that drew about 140 people to hear about using ProPublica-built data and a news application for reporting on education access issues in local schools and districts.

ProPublica published a national story based on the data, examining the relationship of poverty to educational access, along with a Facebook-integrated app for looking up and comparing schools and districts.

These two efforts are moves in the right direction not just for ProPublica but for journalism and the public. By sharing data and making it easy to use, ProPublica produces more value from its deep investments of time and expertise. ProPublica can also benefit from the insights and experiences of others who share or report on the data.

During the conference call, reporter Sharona Coutts, news application developer Al Shaw and computer-assisted reporting director Jennifer LaFleur heard questions, comments and suggestions. Reporters, whose affiliations included both traditional and startup news organizations, also poked and prodded at some of the findings.

As anyone who’s worked with databases knows, data analysis tends to prompt as many questions as it answers. The ProPublica team explained what they’d done to clean up and amplify two major sets of federal data and encouraged reporters to add their knowledge and mash up the new data with other sources. ProPublica also emailed followup links later to those on the call.

This kind of nitty-gritty, story-specific journalism discussion has generally occurred mainly among a limited subset of journalists through specialized skills organizations (such as Investigative Reporters and Editors), in training seminars or in members-only settings. ProPublica’s model shows the promise of opening up that discussion much more broadly — not just among journalists, but for public view of how journalism is done.

Richard Tofel, ProPublica’s general manager, told me that transparency and public engagement have been part of the core discussion at ProPublica since its launch in 2008. In the past year ProPublica has accelerated its social media push, growing Twitter followers by more than five times (55,883 as of this morning) and Facebook friends by more than three times (20,280).

ProPublica has as much competitive DNA as any news organization. Yet Tofel and Editor in Chief Paul Steiger note that their decisions to share databases and expertise don’t have to pass muster with corporate owners or stockholders.

Last year, a ProPublica collaboration with several other news organizations on a project called “Dollars for Docs,” showing pharmaceutical company payments to physicians, expanded its impact after the initial series by sharing and inviting further use of ProPublica’s data. Eventually, dozens of print, online and broadcast outlets drew on the database to produce stories. ProPublica’s “tools and data” page shows other examples.

Given ProPublica’s mission to “make change,” Tofel said, anything that extends the organization’s reach is worth trying.

“That tends to drive us toward open source and it tends to drive us toward sharing,” Tofel said, “and it tends to drive us toward wanting people to follow up on our stuff.”

ProPublica benefits from such followup as its work is credited broadly and its databases and stories are linked off other sites. Social media efforts like the #muckreads feature launched recently (Tweet stories using the #muckreads hashtag and ProPublica considers and aggregates on its site), along with news apps and story links, can help boost traffic to the ProPublica site, now at about 300,000 monthly unique visitors and 1 million monthly page views.

The Web, of course, offers many resources for learning about journalism. Poynter has greatly expanded its online training and knowledge-sharing, through blogs and the News University curriculum, and numerous journalism/media blogs publish spot reports, opinion pieces and guidance that fuel shared learning. Foundation and university-led institutes and websites keep up a steady stream of conversation about ideas and practices. And professional organizations play varying roles in learning for members, with IRE standing out as a leader.

ProPublica adds a new dimension as a news organization sharing its resources directly.

The Web and social media channels also are rich in open discussion and knowledge sharing about some aspects of news and information online — data analysis and visualization, use of social media, new tools and technology. Tech culture is intersecting more and more with journalism, and journalism can gain much more from that influence than new gadgets for old ideas.

Journalism researchers Nikki Usher and Seth C. Lewis explored this idea in an article on the Nieman Journalism Lab blog examining how open-source themes emerged in the learning lab portion of the Knight-Mozilla News Technology Partnership. (I wrote here earlier about the partnership, known as “Mojo.”)

“What can open source teach journalism,” Usher and Lewis asked, “and journalism open source?”

Their findings outline ways the authors think some of the ideas of open-source software align, or don’t, with journalism: transparency, iteration, standards and collaboration. The Mojo experiment should be a good test of cross-pollination.

I’d like to hear about and share other examples of open sharing of resources that enable public-affairs news and information. Please post examples in comments here or email me using the link above. I’ll report back here.

"Mojo" working — on journalism and the Web

If you’re interested in how journalism on the Web might be freed from its often-clunky constructs to flourish in the digital age, you should stop by the website where participants in the new Knight-Mozilla News Technology Partnership (Mojo for short) are mixing it up this month.

A kind of online summer school, the Mojo Learning Lab is running webinars, discussion, reference pointers and coaching for more than 60 people — those who made the first cut in a process that’s part contest, part collaboration and large part public experiment.

While not alone in trying to harness tech innovation for better news and information flow, the Mojo effort has drawn several hundred idea pitches — for better online discourse and storytelling, better tagging and linking of parts of video and other improvements — that touch on both the problems of current formats and the opportunities of evolving Web tools.

The $2.5 million project is a joint effort of the John S. and James L. Knight Foundation and the Mozilla Foundation. It aims to “embed” 15 people and projects in partner organizations over the next three years. This year’s selection process began with a broad call for ides and meetups around the Web and in several countries and will continue with 20 participants being picked for a two-week “hackfest” in Berlin this fall before five finalists are selected.

Mojo is interesting not just for what it wants to do — connect Web innovation with journalism needs — but also for the way it’s trying to do it.

Instead of spinning winning ideas and their authors off with some prize money, the partnership hopes to develop them in working news operations. The 2012 partners: Al Jazeera English, the BBC,the Boston Globe, the Guardian and Zeit Online.

Organizers also promise, and are hammering home to participants, that winning projects must be built using open Web standards so they can be broadly used for greater impact. The program website describes a process that will train participants to turn concepts into code and offer publicly available demos and reference materials.

Anyone can follow along at the Learning Lab and project website, or via Twitter at #MozNewsLab.

Spreading “lessons of the Web”

All that can sound a little abstract, but a lecture the other day by London based “international developer evangelist” Christian Heilmann of Mozilla connected some of the dots. Heilmann’s focus was on programming standards, particularly regarding HTML5, but his illustrations focused on making news and information on the web easier, simpler and more elegant — both for people who create material and for people who use it, no matter the device, browser or screen type.

Mozilla Foundation executive director Mark Surman,who stopped by the lecture to introduce Heilmann, told participants the Mojo partnership aligns with the foundation’s decision to reach beyond its signature Firefox Web browser to other projects “in particular places where we feel like the future of the Web is going to be shaped.”

“Journalism and media is one of these places,” Surman said. Mozilla, he said, has two main interests, “One, that the lessons of the Web and how organizations like Mozilla operate are things that media can tap into,” and second that new tools and services are based on common standards, including HTML5.

Both Knight and Mozilla hope the Mojo project, beyond the software it produces, can act as an accelerant to the frustratingly slow movement of innovation into the core of news culture.

In a recent interview, Jose Zamora, journalism program associate at the Knight Foundation, referred several times to “bridging the gap” between innovation and organizations with significant news capacity and audiences.

He noted that Knight has committed $27.1 million in the past five years to the Knight News Challenge, a contest-based grant program aimed at jump-starting media innovation, funding 76 projects from 12,000 applications.

Zamora said the Mojo partnership is a different approach with similar goals to the news challenge, aiming to pull ideas and skills of programmers, Web designers, artists and other disciplines into thinking about news and information.

“The environment is changing so fast and it’s constantly moving, that we don’t even know exactly what we looking for,” he said. “It’s probably things that we haven’t even imagined.”

To that end, Zamora said, Mojo reached out worldwide and to many disciplines outside journalism to solicit applicants, few of whom came from traditional news backgrounds.

“In the first round it will be more about technology, but it’s about trying to bridge a divide and create a different culture,” Zamora said.

Mojo’s supporting foundations joined forces after Surman met Alberto Ibarguen, president and CEO of the Knight Foundation, and recognized “a kindred spirit” in terms of civic aspirations and interests in finding ways to accelerate technology innovation in journalism and news.

The two organizations share a commitment to information as part of civic life and want to “build stuff, not just talk about stuff,” Surman says. As he and the Knight leadership talked, Surman said, he also attracted by the “subversive idea of really getting inside big media organizations and playing inside.”

Can “open” ideal come true?

Yet Mojo has built some large challenges into its plans. The first is trying to stimulate both competition and collaboration — and open prototyping — at a time when ideas for apps and solutions crowd the marketplace.

“Figuring out the balance between contest and collaboration is both intentional and not easy,” Surman said.

Another tall order is Mojo’s promise to make outcomes open to anyone who wants them, an ideal that has not proved out with some Knight News Challenge projects.

“Nobody’s good at taking iterative inventions that are interwoven with something bigger and pushing them back into the world,” Surman said. Mojo’s paid fellows will be working in news organizations with their own content management systems, he noted.

“We won’t know till we get there what it means to do it in the open usefully in ways that others can pick up and run with it,” he said.

Along the way, Mojo’s champions hope to link like-minded people: news and information experts, programmers, designers, videographers and others who want to build better tools for creating and consuming news — and who’ll do so using open platforms and collaboration.

“We want to create a bit of a school of thought around these changes,” said Phillip Smith, a Toronto-based digital publishing consultant leading Mojo’s operational process.

Smith said he talked to dozens of journalists, journalism educators and newsroom programmers before putting out calls for ideas. The other day he blogged an invitation for journalists to lob their suggestions into the Mojo process. He also has posted at PBS MediaShift about Mojo.

The Mojo participants trend more heavily toward code than newsgathering, but offer a notable mix of interests, backgrounds and information passions.

Chris Keller, who’d worked in print and online roles in newspapers before joining madison.com to work on audience development, is hoping to develop better topical pages for news issues. Corbin Smith, 23-year-old working on his graduate thesis in Toronto, pitched his idea for a “kind of fact-checking and narrative building platform” that would be associated with a user rather than a web site.

Dan Whaley, a San Francisco entrepreneur who’s founded and sold one major dot-com company and is involved with several nonprofit ventures, was drawn in by a Mojo challenge inviting proposals for taking online discourse “beyond the comment thread.”

“This to me is mankind’s biggest problem, is how do we understand what’s credible?” Whaley said. “In order to figure that out, we have to have a feedback channel that works.”

Whaley submitted the outline for Hypothes.is, which he described in the pitch as a platform that “will enable sentence-level (i.e. annotation, or “atomic” commenting) critique of written words combined with a sophisticated yet easy-to-use model of community peer-review.”

Whaley’s bio notes that he wrote the original code and cofounded the online travel reservations company GetThere.com, which Sabre/Travelocity bought in 2000 for a widely reported price of $750 million. Hypothes.is isn’t dependent on the Mojo process, but Whaley said he was impressed by the participants and enjoying discussing his ideas with like-minded people.

“This challenge is kind of like the hashtag for people who are interested in solving this problem,” Whaley said. “In that way it’s attracting people like myself with a wide set of backgrounds.”

Does Web innovation need foundations?

Mojo organizers say they’ve heard some complaints and criticism, mostly in email and project comments. There were questions about whether technology innovation could happen in newsrooms at all. Some newsrooms questioned the selection of the first five partners — operations that seemed to have a leg up already on Web innovation.

There are practical concerns, too. Most legacy news organizations run on closed or proprietary content management systems, built for print or broadcast, that don’t afford easy integration with new technology. Incoming Mozilla journalism leader Daniel Sinker, who will take over Mojo’s leadership, noted that some newsrooms have found ways to work around such obstacles — implementing new features at the front end of systems rather than the back end, for instance, or building apps that work outside the CMS.

“Most limitations around CMS are cultural limitations,” Sinker said.

Others wondered why there was a need at all for a Mojo project, given the seemingly infinite supply of ideas, new tools and startup ventures for online information.

Zamora, however, said Knight sees gaps that the marketplace isn’t filling and a need — as a foundation focused on journalism’s changing role in the digital era — to actively promote news and media innovation. He also emphasized the impact that could come with working through Mozilla, an organization that’s “of the Web, not just on the Web.”

Despite all the players competing on Web technology, Zamora added, few take an open approach that “allows everyone to use services or products… or to learn from their projects, successes or failures.”

Success, he said, will be measured not just by the news partnerships and new products themselves but by whether Mojo succeeds in creating ripples that carry out many circles beyond its core.

“I think one of the main things would be to create this new culture of news organizations being more proactive and more open to constant changing on the Web,” Zamora said.

At Mozilla, Surman also hopes that Mojo’s ideas infect newsrooms. Mojo’s circle of influence seems modest so far – voting was light on the idea pitches and only 24 nonparticipants were following the Learning Lab early this week, according to the web site counter. Surman wants more “community-building” — and more impact.

At the end of a year success would show up not just in the newsroom projects but in relationship building among participants, the broader Web community and among the news partners and their business structures, he said.

He hopes the Mojo project will lead newsrooms to hire more people like those chosen for fellowships “to work on projects like the projects we introduce,” Surman said, describing “a cultural transformation piece, where decisions are being made.”