This is one in a series of reports on DiSEL (Digital Story Effects Lab) Research projects conducted in 2007 through a research grant from the University of Minnesota. First in the series was on Navigation through Slide Shows
Why we did the study
One of the great strengths of the Web is the ability to keep news updated and to alert readers immediately to stories they need to know about. This is also one of the biggest organizational changes the Web has brought to newsrooms. Shifting from daily to constant deadlines has caused a rethinking of work flow, editing, and reporting responsibilities.
But questions remain about the best way to ensure that these updated or breaking news items are presented on the page for greatest visibility. Judging from the wide variety of design techniques newsrooms use to designate breaking news, there is no consensus on the best approach.
In May 2007 the top 102 US newspapers’ websites were analyzed to catalog the different ways “breaking” news was being displayed. We looked at labels used to indicate news was updated or new and the design techniques for differentiating “breaking” news from other news items on the homepage.
Labeling: Thirty-four of the online news sites examined had no designation of “breaking” news. Of the 68 sites that did:
- 31% used some version of “Breaking”
- 30% used some version of “Latest”
- 14% used some version of “Update”
- 25% used miscellaneous labels including “Developing News”, “News Flash”, “News Bulletin”, “News Alert”, “Up to the Minute.”
Design: The methods used to designate, design-wise, the “freshest” news items on the page varied, and were often combined.
- 12% tagged individual stories with “New” or “Updated’, usually in a bold color
- 57% put “updated” stories in a box
- 62% timestamped the entire page and / or individual stories
It was clear that no conventions had been established for designating those news items that were freshest or most recently updated.
We designed the study to get at the following questions:
- Did the design choices made to designate updated content affect the user’s recognition of which items were new or updated?
- Would the design aid, or impede, the likelihood that the news user would find, read, and remember news items most recently added to the site?
How we did the study
We worked with the Minneapolis Star Tribune to get daily feeds of their homepage. We inserted into the “live” page a fake “breaking news” story (about a tank truck accident and subsequent chemical spill shutting down a major highway in town.) Each day of the testing, our designer created three test “home pages” using the top three ways updated news is being designated on websites – timestamp, labeled, boxed – to distinguish this “breaking news” story from the others on the page.
Version 1: Timestamped
Version 2: “New” placed next to updated story
Version 3: Updated story headlines in a separate box
Participants: We wanted to study a wide array of online users in this research so we took the eyetracking equipment to two locations in Minneapolis: the student union at the University of Minnesota (where the demographics were largely young, Anglo adults), and the Midtown Global Market Downtown (where the participants represented a wide range of demographics – age, race, education.) In all, 96 participants were tested, divided into each of the three conditions.
After the eyetrack calibration, research participants were asked to go to the homepage and “just look around.” They were told they could look however long they wanted and could click on whatever they wanted. When they indicated they were done, they were asked a couple of questions:
- What are the main stories you recall from the website?
- The response to this question could be checked against the eyetrack video. Was it clear they “saw” the updated story yet did not list it as one of the stories they remembered?
- What was it about these stories that made you remember them?
- The response to this open-response question helped catalog the attributes of the news story that made it memorable.
- Which of the stories on the website was identified as a “news update” // or which of the news stories was the most recently updated?
- This question sought to discover if people, in fact, recognized that there was a story that was designated differently than the others.
- Do you try to find the most current story when you go to a news website?
- This question would allow the researchers to see if there was a difference in response from users who are self-proclaimed “fresh” news seekers from those who are not.
- On a scale of 1 to 5 (with 1 not at all interested and 5 very interested) how interested are you in: Politics, Crime, Traffic Reports, Sports
- This question could help researchers to see if interest in a news topic (in this case traffic) resulted in a higher degree of recall.
Findings
Both the eyetracking videos and the post-exposure survey responses were analyzed and matched – the videos to see whether participants eyes “fixated” on the breaking news visual cue and the survey to see their responses to the post-exposure questions. Here are some of the key findings:
Finding 1: Bigger is Better – or is it? In terms of visual cues, it was clear from the results of the eyetrack sessions that the larger the cue, the more likely it was to be noticed. By visual cue, the percent of participants exposed to that style of display who fixated:
But when participants were asked in the post-exposure survey to say which story on the homepage was the most recent or which was the breaking news item, the participants exposed to the timestamp (35%) and “new” pages (32%) had greater recognition of the freshest news than those exposed to the headline box (20%). Even though 89% of the people exposed to the headline box page clearly looked at it, only 20% of them recalled any breaking news story.
What might this mean? Although the headline box drew more eyes, the headline text size in the box was smaller than the other two display styles. Headline text size may be an important factor in user’s memory of a story. So, although boxing the headlines made that visual element on the page more broadly seen, the small size of the headline type within the box made the recall of the story weaker.
Finding 2: Story Attributes Participants were asked in the post-exposure survey “What was it about the stories you recalled on the page that made you remember them?” Their open responses were analyzed and categorized (for example, if they said “I remembered that story because it happened near where I live.” Or “I drive that highway every day.” the response would have been coded as proximity or familiarity. If they said “I remembered it because of the color photo next to it” it would have been coded as “photo.” Here are the categories and findings:
The attributes of the story and what made people recall them fell into two categories: personal and design. By far (66% vs 24%) it was the personal triggers (interest in the topic, proximity, familiarity, emotional response) rather than design cues (size / position on page, photo, hyperlinked headline) that were given as reasons for recalling the story.
Observations: The old adage “You can lead a horse to water but you can’t make him drink” probably fits here. If there isn’t interest, the design may well not have any impact.
There is a good deal more observations we can make based on this research, look for more results in future OJR columns.
