In January 2008 a group of interactive producers from news websites gathered at the University of Minnesota for the first Eyetracking Research Consortium, part of the Digital Story Effects Lab project run by Nora Paul and Laura Ruel. Following is the first in a series of articles about findings from the studies conducted for the Consortium members. For more information about the Eyetracking Research Consortium, go to www.disel-project.org

San Jose Mercury News

Some of the eyetracking studies conducted with the consortium members were comparisons of different design approaches or navigational schemes and their impact on user behavior. Other members just asked for feedback from users about their experience on the website. With the eyetracking we could record not just what someone said about what they did on the site, but to actually see what they did.

The San Jose Mercury News wanted to see how their site compared with the Contra Costa Times site – a sister site with a slightly different emphasis on visuals on the home page. They wanted an unstructured study of how people engage with the two sites and then to hear reactions.

Between April 29 and May 1, 2008 fifteen undergraduate journalism and mass communication students were brought in for the eyetracking session. All were between the ages of 19 and 22. All self-identified as being very comfortable with the web. They were set-up with the following scenario: “You are considering moving to northern California for a job and decide to look at two regional news websites. Look at the sites as if you were sitting in your own room. Go where you want to on the site, stay for as long as you would normally. Let us know when you are done.” Half of the participants saw the Contra Costa site first then were sent to the San Jose site, the other half did it in reverse order.

San Jose was also interested in the usability of their calendar function. We asked a few of the participants specifically to go to the Things to Do function and search for particular items: a concert on Friday night, the movie “Forgetting Sarah Marshall,” sushi restaurants. We also asked them to look at the My News / My Blogs function. This video shows one of the participants’ use of those two functions:

Comments / Observations: In the post-exposure discussion about the sites we asked participants if they remembered a “Things To Do” feature. Seventy-two percent did not recall anything like that. Some of that can be attributed to the fact that the features is below the scroll and people didn’t scroll down the page:

Other times they did scroll but they simply didn’t fixate on it.

Additional studies / observations: Contra Costa and San Jose use a left hand embedded ad style on their story level pages. We looked at all of the stories that were linked to by the participants to see if the left-embed ad was fixated on. Ninety-nine stories were clicked on by participants within the two sites, an average of 6.6 stories per participant or 3,3 stories per site. The most number of stories clicked by one participant was 11, the least was three.

We looked at the information that was available about the stories that were clicked on (headline only, headline and a blurb, headline and photo and blurb, or headline and photo only.) Fifty-nine of the 99 stories clicked on were headline only, with 17% headline / blurb and 21% headline / blurb / photo. (Two were headline / photo links – they were ARA “stories”, essentially advertisements.)

Of the 99 news stories clicked on 63 were stories with the left-embed ad. We looked at the gaze plot for each of these participants and saw that for 23 of the 63 embedded ads was there some level of “fixation” or about 36%. Sixty-four percent of the time embedded ads were ignored. You could see that the people read right around the ad.

We also looked at whether people went to the right rail ads on the page. Only 22% of the story level viewings indicated any looking at the right rail advertising.

By contrast, though, 56% looked at the left rail. In the left rail was the “most viewed / most blogged” listing and a “top classifieds” box. Sixty-nine percent of the viewings of left rail content was for the “most viewed”, 30% looked at both of the content features in the left rail.

The other content on story level pages is sidebar or related material boxes. Fifty-two of the stories looked at contained sidebar material. When people went to those stories with sidebars, 75% of the time the material was looked at – mostly when there was a box in the upper right with a photo or slide show.

The participants were interviewed after they went through both of the sites. Following is the transcript of one of the interviews, some of the insights into how young adults use news sites and different features and functions are interesting.

Interviewer: You looked at two websites. The first one was the San Jose Mercury News. Let’s start with that. Was there anything that stood out?

Interviewee: I noticed that there were columns. I paid more attention to the middle column.

Interviewer: Which was?

Interviewee: I don’t know. I think it was headlines. There was a most viewed section too. I think I clicked on something there. I kind of skipped over all the political stuff because I blocked those out. That’s not a good attitude.

And I think there was some kind of moving ad that caught my attention because there were women in it moving and laughing.

The weather at the top was nice.

Interviewer: There are different ways you can navigate the site. So what did you prefer and what did you use?

Interviewee: I just remember clicking the titles of stories in all three columns. But they were kind of up at the top a little bit more. I tend to think of stuff down at the bottom as just like useless. There’s not usually very many links down there.

Interviewer: The second website was the Contra Costa site. And the same question here. What kind of features do you remember?

Interviewee: I remember that the most viewed / most sent was way down at the bottom. And it’s usually like way up at the top. But they had celebrity news. Like where I usually look right in the center column with all the most important, I don’t know, bad celebrity stuff in there which I usually don’t see on anything other than AOL. And I thought that was a little funny. Although I never heard of the Contra Costa before. Is that like a location?

Interviewer: Yep, a city in northern California.

Interviewee: Oh, okay. Well, I thought they were both very jumbled. Both websites were very busy. They had lots of columns. And text wasn’t very large. It was very small. And not a very readable typeface. So I didn’t really like the layout of either one.

I’m used to – I spend a lot of time on like AOL. So they’ve got like a box with images and then text. And it kind of switches, too. I don’t know. That’s not really a news website though. It’s kind of like an entertainment thing. I don’t know. I could have done with a little less pictures, a little more text.

Interviewer: In both of them or-

Interviewee: Mostly in the second one, the Contra Costa.

Interviewer: So which of the two do you prefer?

Interviewee: The first one. The San Jose Times.

Interviewer: Okay. What was in it that-

Interviewee: I don’t know it just seemed more informative. But I saw some of the same stories echo between both websites. So I thought the first one was a little more interesting because by the time I got to the second, the Contra Costa one, I had already seen the story. San Jose Times, I just liked it because it seemed more professional.

Interviewer: In what sense? What made you feel it was more professional?

Interviewee: I guess just the layout. There’s a little less jumble. Like the headlines were larger. I don’t think it buried the stories. And they started with topics that were a little more sober. They didn’t go right off the bat with like – stars. I can’t remember.

I kind of look at – the movies were good. Local news was good. And stuff that affects me, I guess.

Interviewer: Okay. Now there were a few places that you could do things – for example, you had an option to create a section for my news. Did you notice those?

Interviewee: No. I noticed that I could personalize the weather.

Interviewer: The weather is important.

Interviewee: I like the weather.

Interviewer: In which of the news websites was that?

Interviewee: San Jose Times had a weather thing up at the top. No, I didn’t really notice any of the interactive stuff I guess.

Interviewer: Did you notice the photo slideshows?

Interviewee: No, just that one ad with the moving, laughing women.

Interviewer: Which website was that on?

Interviewee: I think it was the first one. The San Jose Times. It was over on the right.

Interviewer: Did you notice the web polls or web vote? You know they ask you questions-

Interviewee: No, I never vote on those. They always try to set you up.

Interviewer: In what way?

Interviewee: Like on AOL it’ll ask you the stupidest questions. Like I spend most of the time on AOL. And every single news story that you view, it’ll have some kind of – I think it’s some kind of trick to get people to interact more with the story. But they’ll have a little question at the bottom like, do you agree? Do you think this is right? It could be like a – like the kid who was like drowning the swimming pool. It would be like, do you think the conditions could have been stricter? And I just get so tired of those things.

Interviewer: Now did you notice a bar with Digg and Delicious and Facebook on it?

Interviewee: No, but I don’t share stories. I don’t email stuff out unless it’s about dogs. ‘Cause my mom really likes dogs. And she’s pretty much the only one I’d send that to.

Even without eyetracking, this kind of session with casual users of the site can give valuable feedback into what works, and what doesn’t, about your site’s design and features.