Curation questions and the start of some answers

Information curation, like data visualization, is one of the buzzwords being used by those trying to guide, and goad, news organizations into thinking about new content models. Jeff Jarvis talks about “curation” as the activities of sorting, choosing, and display.

Mike Shatzkin on the Idea Logical blog said

“Curation is a term that has always referred to the careful selection and pruning of aggregates, such as for a museum or an art exhibition. But the concept in the digital content world means the selection and presentation of these disparate items to help a browser or consumer navigate and select from them. Aggregation without curation is, normally, not very helpful. Curation creates the brand.”

There have been some forays into news site curation. LJWorld created a Kansas Legislature page in 2005 that aggregated links to general news coverage of the state Legislature. But they took the next step of selecting and organizing stories by specific issues like Death Penalty, Concealed Weapons and Sunday Liquor Sales. The page served as a “one-stop shopping” resource by anticipating the kinds of information someone interested in the Legislature might want by including such resources as bios of legislators, legislative calendars and bill finders, and copies of the State of the Union addresses going back several years. LJWorld still has an aggregated page of Legislative coverage, but it is not longer curated – it is just a list of links to news stories.

Losing the topic focus switched the LJWorld’s page from curation to aggregation because an essential step in curation is organization, as they did with the issues, not just listing. Just as a well-curated museum has the Early Asian art area separate from the Surrealist collection, so should news sites provide some subject organization within large news topics.

Curation can also entail finding and providing resources from all over, not just aggregating your own content.

The New York Times Topic Pages are an example of this kind of curation. They have thousands of subject / event / personality specific pages which provide an overview article on the topic, links to all the NYT past coverage (with a searchable database specific to those articles), and, here’s where the curation comes in, sections on “Headlines from around the web” (a listing of articles found using the NYTimes’ news aggregation program Blogrunner which has been sent to selected sites) and “A list of resources from around the Web as selected by researchers and editors of The New York Times.”

We were interested in observing how this kind of curated content was used by people on an information seeking quest. We conducted eyetracking sessions with 37 undergraduate students at the University of Minnesota. The students were told that they were going to write a research report for a class and were given a selection of 10 New York Times Topic Pages from which they could choose the topic of their report: the U.S. Dollar, Earthquakes, the U.S. Federal Budget, Foreclosures, Mortgages and the Markets, Piracy at Sea, Stem Cells, Tornadoes, and Unemployment.

The students sat at a Tobii Technology eye tracking device in our research lab and were told to go to the Topic page of their choice and do whatever they wanted (read, click). After 10 minutes we stopped the session. All of the participants answered a short online questionnaire about the website and their information seeking experience after their eye tracking session.

In analyzing the eyetrack videos we designated each section of the Topic Pages (e.g., “Summary,” “Multimedia,” “Navigator,” etc.) and we coded the participant’s “attention” on the page using a construct describing the “Path to a Click” by researchers at Yahoo which characterizes activity on a site by the frequency and duration of a person’s attention to a particular part of the page. These levels are:

  1. Saw: when the participant’s gaze passed across a section
  2. Noticed: when a section the participant “saw,” then glanced away from, was returned to
  3. Parsed: when the participant “fixated” on a section, clearly taking in the text / image

Other things we analyzed in reviewing the videos were:

  • the sections that contained URLs that were clicked by the participants to figure out whether the placement of the content influenced their clicking behaviors
  • the content of the items that were clicked (headline only, headline plus abstract, headline plus image, etc.)

Among the findings are the following:

ATTENTION TO DIFFERENT PARTS OF THE PAGE. The percent indicates the number of participants.

  • Most parsed sections (areas where most attention and time was spent)
    • Summary (100%)
    • Articles about (70.4%)
    • Multimedia (58%)
  • Most unparsed sections (virtually no attention paid)
    • top-right search box and top global navigation (both 97.4%)
    • Related Topics (83.3%)
    • left-column advertisement (80.3%)
  • Zones in which the most links were clicked: (based on a 3×3 grid placed on the Topic Page)
    • left column middle row
    • middle column top row
    • left column bottom row
  • Statistically, contents in the left column were seen, parsed, and clicked the most (Note: the left column was where the archived stories were displayed)

WHAT WAS CLICKED
Among the 37 students, a total of 138 items were clicked (an average of 3.7 clicks per participant) Of those items that were clicked:

  • 42% were headline only
  • 24.6% were a headline and story abstract
  • 14.4% were a headline, abstract and photo

Of all the items clicked 85.5% were story links. Of the other items clicked, 65% of them were from the “Related Topics” box.

In the post-eyetracking survey, participants were asked a number of questions about the features of the page and their importance to them.

  • 72% of the participants rated the Navigator (links to other websites) as Useful or Very Useful. Very Useful or Useful ratings of the other key areas of content on the page: Articles from the archive (67%), Headlines from around the Web (59%), Overview of the topic (64%).
  • Of the existing or potential functions on the page that could aid a researcher, the following is the ranking by those considered “somewhat important” or “very important”:
    • 100% Bookmark or save an article (81% said “very important”)
    • 70% See article ratings from others
    • 64% Rate an article
    • 56% Sort articles by your rating
  • When asked what it was about an item they clicked on that prompted them to click (multiple responses were possible):
    • 81% cited information in the headline
    • 27% cited a photo
    • 21% cited information the abstract
  • The preference for display of stories is, by far, most recent to oldest. Articles rated highest by others or most read / emailed articles about the topic was preferred over oldest to most recent stories.
  • Students indicated their preference for finding information (if they did not access something like the NYTimes topic page) as major search engines (59.4%), the school library website (35,1%), and online encyclopedias (5.5%).
  • Compared to their alternate sources of information, 52.8% of the participants perceived NYT Topic Pages as being equally credible, 37.8% perceived NYT Topic Pages as more credible, and 9.4% perceived NYT Topic Pages as less credible.
  • Compared to their alternate sources of information, 54% perceived the information on the NYT Topic Pages as having about the same level of completeness, 30% saw it as more complete, and 16% saw it as less complete.

Students were asked an open-ended question about what they thought of the organization of the NY Times Topic page they used. Here is an analysis of their comments:

  • 49% mentioned the site was well organized and made it easy to find information
  • 40% mentioned something about the site being busy / visually cluttered
  • 35% mentioned the appealing design of the site

Some of the comments might serve as suggestions for other news sites looking into creating similarly aggregated / curated topic pages:

  • One student said, “Without a search option, it was kind of hard to find an article that would benefit my ‘research paper’ just from looking at headlines.” Of course there was a search option for the story archive, but it is below the scroll towards the bottom of the page and, apparently, easily overlooked.
  • “It was hard to distinguish between articles and opinion pieces.” This should be an important curatorial distinction.
  • “Have a way to mark an article you’ve already read.” A followed link is just a slightly lighter color; maybe clicked story links could be made more distinct or could “grey out” for easy recognition.

Honing “curatorial” skills in news organizations is one area that holds potential for creating high value resources for both casual and more motivated information seekers. However, there remain many questions about how best to design and organize these pages rich in both internal and external information. It is a research area we intend to continue to pursue and we welcome your feedback about this study and suggestions for future studies.

If you would like to see “hot spot” images from the study go to: http://picasaweb.google.com/norapaul/EyetrackingHotSpotsNYTTopicsPageProject#. The areas that are red indicate the longest “fixation”, green the next longest, and yellow after that. Areas of the page with no color were not viewed by the participant.

About webtech

Comments

  1. Curation = Web shells!!! Very interesting, Nora. The BBC Special Reports have been taking this approach for a long time. We’ve been looking at many topic pages, seeing what works and doesn’t. We’re planning on adding them to WellCommons.com after the redesign launches the week of Feb. 21.

  2. The Sanatoga Post (sanatogapost.com) uses a blog theme that, like many others, allows archiving or curation by category. We’ve found over a period of two years that readers who share our stories with others favor forwarding the category URL rather than a one-story link. When we ask why, they explain that they want those with whom they’re sharing to see a story in context, not stand-alone. The upside for us is more page views and longer time on page.

  3. Very nice case study. I think an interesting question to answer is – “How does curated content help educate the user on a subject?”

    IMO context messaging is a big value provided by curated content. KBucket is a platform for posting curated links.

    As an example for my comment above – This curated page on Wikileaks should prove my point: Wikileaks News. As the study shows most people pay attention to the headlines and the brief description. To strengthen an argument and prove a point, accumulated headlines on a subject offer powerful messaging value.

    This KBucket page let’s the user find out the latest on Wikileaks, compare news coverage and messaging by different news organizations and the emerging news is always in the context of the historical news.