Proposal: New standards and tools for distributed online reporting

One of the Internet’s strengths as a medium for journalism is its ability to support widely distributed, grassroots news reporting. Whenever a significant earthquake hits Southern California, tens of thousands of residents log on to the local U.S. Geological Survey website to report what they felt. The USGS site processes these surveys in real time to generate zip-code level shake maps that depict the intensity of the quake throughout the region.

There’s no need to install sensors all over town. And no wait for a costly phone survey. The Internet enables the USGS to engage a small army of citizen reporters to collect their information. Journalists, of course, can do the same with their reporting projects.

But such efforts run into problems when there’s no single obvious source for grassroots reporters to submit their information. We saw this with the dozens of websites that attempted to compile missing persons lists after Hurricane Katrina. No one publication had a comprehensive list of the missing. And attempts to aggregate the lists required either finger-numbing cutting and pasting, or equally tedious RegEx coding to parse the data from the various websites.

Sure, the USGS managed to establish its website as the place to go to report earthquakes. But, for most stories, readers and journalists face the “Katrina conundrum” — too many sources trying to collect the same information, without coordination.

It doesn’t have to be that way. Of course, some journalists always will want to go their own way, searching for a scoop. But others see the value in cooperation, in working together to best provide comprehensive information for the public. To do that, website publishers need:

  • A simple online tool with which to collect fielded information from the public.
  • A way to share that information with others collecting similar information, and
  • A way for all those information collectors to know when other collectors have gathered fresh information.

Today, I want to propose that OJR lead an initiative to address these three needs.

Right now, to collect information like the USGS, or a Katrina missing persons list, you need to be a coder who can put together an HTML input form and a script to dump that information into a database. What online journalism needs is a free, open-source tool that does for grassroots reporting what Blogger.com did for online journals – making it easy for a non-coder to set up a grassroots reporting input page with no HTML or database experience.

Second, the information that tool collects ought to be recorded in a standard fielded format, so that it can be easily shared with other collection efforts. There’s no need to build a common database or central server to support this. All that’s needed is for each site collecting data to be able to export it as XML, using a common set of fields. Tools can be written, along the lines of RSS aggregators, to collect those XML fields and aggregate them into comprehensive databases.

Personally, I believe that the RSS standard itself does not support nearly enough fields to transmit an entire database of incident reports. We need something more expansive. Dave Winer’s OPML moves in that direction, but I don’t know that it offers the granularity needed for this project. The point is, I think a common XML format is the solution to the problem, but that we need to have some industry discussion as to what that format might be. Obviously, it ought to be flexible enough to accommodate everything from missing persons lists to fraud reports to (my pet project) theme park accidents. Let’s start talking on what that format might look like. (And, rest assured, I don’t want to recreate the overkill of NewsML.)

Third, we need a weblogs.com-type destination site that information collectors can ping to let identically tagged information collection projects know that they’ve been updated.

We could build a development tool that handles issues two and three itself. But I think that it is important that any development tool work with collection efforts that do not use the tool. That’s why the blogosphere works so well. You don’t have to use Blogger, or any other specific individual tool, to link to and aggregate other blogs. Our distributed reporting efforts should work the same way.

So, who’s interested in helping me refine this idea, and build these tools? The development of blogging tools showed us the power that could be unleashed when we liberated online narrative publishing from the HTML coders and opened it to everyone. Let’s do the same with distributed data reporting. E-mail me at [email protected], and let’s get started.

About Robert Niles

Robert Niles is the former editor of OJR, and no longer associated with the site. You may find him now at http://www.sensibletalk.com.

Comments

  1. Hello Robert,
    I find this project very interesting. We have launched a craigslist style classifieds posting site, based on community involvement(www.citiposts.com). We are an xml based website, and make the contents available in RSS, wireless and audio formats. All of this is done automatically, using our translation engine.
    Once a bunch of sites agree on a simple (but functional) XML format that works for all the sites that exchange information, I think we might already have the code to make this happen.

    Please contact me if you would like further discussions.

    Thanks
    Vish
    (vish[at]21stsolutions[dot]com)

  2. Robert,

    I applaud your grassroots effort to develop the tools necessary to gather and coordinate such information.

    I thought you might be interested in a parallel if not similar effort taking place among those of us in the public safety and service realm.

    Among the most promising tools for both senders and recipients of “emergency incident” information is the seminal Common Alerting Protocol (CAP), a data interchange standard for alerting and event notification applications.

    Some of our nation’s best and brightest are working on this concept in earnest, and the Los Angeles Fire Department is but one of several noteworthy agencies supporting the implementation of CAP.

    The Organization for the Advancement of Structured Information Standards (OASIS) adopted CAP 1.1 as a standard on 10/1/2005, and their Technical Committee welcomes participation by stakeholders and members of the public concerned with disaster management and response.

    I’d encourage anyone interested in learning more about CAP to visit Art Botterell’s ever popular CAP Cookbook.

    While CAP may not directly solve the conundrum you mention, I do think that at the very least, the process that brought forth CAP might be of interest.

    Respectfully Yours in Safety and Service,

    Brian Humphrey
    Firefighter/Paramedic
    Public Information Officer
    Los Angeles Fire Department

  3. Joe Pandolfo says:

    The Independent Media Center website network < "http://www.indymedia.org/en/index.shtml"> has been fastgrowing (since inception at the renowned Seattle WTO protests), but in a structured way. Good model.

  4. That’s an intersting effort, certainly, but more along a narrative model than the data-driven incident reporting process I’m talking about. But two models could, and should, work together.