What's the future for news personalization?

I have customized my Google News page, my Yahoo News page, and many other news sites. My RSS reader is deluged by updates from the hundreds of feeds I have subscribed to over the years. Do I read that material? Not very often.

I decided to speak to Calvin Tang, the co-founder of Newsvine, a next-generation news personalization site that tracks reader habits and serves up articles that those individuals might actually read. I wanted to find out how news personalization sites have changed and if they are ready for people like me.

Tang talked about the improvements and the hurdles facing news personalization, and what refinements we might see over the next few years.

OJR: In a broad-brush stroke, what is Newsvine hoping to accomplish?

Tang: There were the three major things that we are going after. First, our aim was to set out to automate the collection, organization, and syndication of the exponentially growing pool of content available on the Web. With the rise of the blogosphere and personal publishing, it seems that there is becoming an ever-increasing amount of content out there.

The second thing we set out to do was to leverage the base of people in the world who had a story to tell but who also lacked an easy way to use publishing platforms and get an audience. Not everybody in the world is tech-savvy enough to set up his or her own blog. That’s why the first wave of citizen-generated content out there was very tech-heavy.

Our third aim was to give people a way to interact with each other in meaningful ways on topics of shared interest and to also be able to discover new material and authors as a result of this interaction.

OJR: How close has Newsvine brought this “Daily Me” concept to reality?

Tang: I would like to think that Newsvine is at the front of the pack as far as personalization of news but I think we, and the industry in general, have a long way to go. Some of our more long-term initiatives involve setting up our systems so that people can get their news in an ever-more increasingly efficient manner. I think that as the amount of content grows it becomes more and more important to organize that in a meaningful way.

OJR: What do you mean by more “efficient” and “meaningful”?

Tang: I mean there are two problems to solve and because you have to solve them both, it becomes difficult. One is that people don’t want to get certain types of information. They want their international news but they may not want sporting news, or something like that. And as a result there are services out there that are narrow in terms of topic–like a site that’s all technology news. That’s good but that can’t be your only news site. People like to discover things that they might enjoy reading but they didn’t necessarily know that they would before they were exposed to it. So giving them a way to sift through the large body of content out there is one problem.

There is also the type of news that everyone should generally know about. If there is a huge event in Iraq that is going to impact our domestic and foreign policy, a reader should have access to that. Now whether or not you spend a lot of time reading about it that’s another question. We think that bringing you the top news is one of the important things. That’s why we present our site with traditional media content right next to citizen-generated content. We don’t favor one or the other. We think that they are complementary in many ways.

OJR: Newsvine obviously doesn’t have the overhead of traditional news organizations because Newsvine does not have a staff of reporters and editors per se. What impact do you think sites like Newsvine will have on the quality of journalism when traffic flows to Newsvine rather than traditional news sites that also depend on advertising to support the reporting process?

Tang: I think that eventually all traditional media companies will have to rely on some form of citizen reporting, partly motivated by financial reasons but also because of access. While the quality of reporting from the average citizen is typically of a lower ‘quality’, in the traditional sense, I think that this is offset by the timeliness and unfiltered nature of accounts offered by citizen media. Traditional journalism will always be a part of the equation, but a combination of new and old media coverage yields a flow of information from event to consumer that is greater than the sum of its parts. It is for this reason that we don’t take sides between traditional media and small media. I believe that consumers will benefit from a convergence of the two models, and that the long-sighted media companies will adapt accordingly.

Take the recent incident involving the UCLA student who was Tasered. Prior to the existence of video-enabled mobile phones and youtube.com, I think we, as consumers of news, would’ve been further from the truth and less affected by it. However, without the follow up research and reporting by professional journalists on the officer’s background, we would be left with an incomplete picture of what led up to the incident.

Currently, a good deal of the reporting done by citizens is largely incidental, a byproduct of proximity, chance and personal initiative. Moving forward, I think economics and consumer appetite will convince publishers to actively procure citizen reports on specific topics or events. Meantime, Newsvine’s base of contributors from around the world grows and improves continually, ready to meet that demand.

OJR: Your users are also providing links and summaries of articles people can click on, sending traffic to those news sites…

Tang: Yeah, absolutely. We have a very liberal linking policy. We don’t do things to keep you on our site. We are happy to send you off to wherever the best information is hoping that a good user experience will bring you back. I would say that’s at the heart of the Newsvine model. There are many sites out there that provide AP news; there are many sites out there that provide links to other content. But Newsvine essentially is a crossroads of content where a rich discussion happens. So I would say that the number one thing we strive for is to create rich discussions around content.

OJR: So who is taking part in the discussions?

Tang: I guess there are a few different definitions of active users. The majority of the visitors to our site just read articles and that is to be expected of most sites. Participation at most sites is somewhere in around one or two percent. At Newsvine depending on your definition of participation, that rate can be much higher. So about 15 percent of our users are actually actively voting and commenting around the site. I would say another four to eight percent are seeding links and one or two percent are writing original articles. That number is low as expected because it is hard to write articles.

OJR: And even more generally, who are these people?

Tang: we have a good proportion of college students on our site but we do have an older crowd. I would say people in their 20s and 30s are probably at the center of that long curve but we have users all the way up to their 70s as some of our most active users.

OJR: Are you getting people from all over the US or also from around the world?

Tang: I would say that we definitely have a heavier presence here in the US and one of the reasons for that is because the AP is a little bit US-centric in terms of their news coverage. But while our viewership is more skewed towards to the US, if you look at our top contributors, there is a very wide mix of people from around the world.

OJR: Do you think newsreaders ready for this concept or do you think these are all the early adopters?

Tang: Since March, when we had our public launch, we’ve been moving beyond early adopters. We’ve had month over month growth for the last six months and we’ve tripled our traffic since May. And a lot of that new traffic is your traditional news consumers. That’s who we are aiming for.

We are not really trying to compete with other sites that employ the newer types of technology and newer sorts of models. We are going after the crowd who is used to getting their news from CNN and MSN, NBC, and Yahoo News.

That intention is built into the design of our site. We don’t launch right off into a five-minute tutorial on how do you use the site and all the things you have to set up. We wanted to make the user experience very good for someone wants simply to point and click and read.

OJR: You mentioned earlier that 15 percent of the audience is actually voting so those are the active members…

Tang: Yeah, voting and commenting…

OJR: How does this affect what news is presented? You are displaying news based on the interests of a small group of people. Do you have some counter-balancing algorithms to still be able to provide a diversified news budget?

Tang: Yes we do. Anytime you have a system in which editorial functions are driven by user behavior, you have to do things to safeguard against a small group of people making changes that affect the rest of the user base.

So while comments and votes do affect the placement of stories on Newsvine that’s not nearly the whole mix of things that go into our ranking algorithm. Some of the other things include page views which all readers of our site affect that and we measure something we call long views, which is the amount of time somebody spends on a page.

So we add weight to an article ranking if somebody spent a few minutes on it rather than just clicking there and clicking away.

In addition to that one, of the strongest contributors to an articles rank is its freshness. So if something comes in right off the wire or is submitted by a user right away, it has a pretty high ranking right off the bat. Imagine that the content, as it comes into the system, cascades down the page, and if it receives a lot of activity in terms of views, and votes, and comments, it can stick or even move back up. So if we didn’t do that, Newsvine would be a static, old style site.

OJR: You mentioned getting contents from the AP and individual contributors. AP is mostly text. I assume that a lot of the material submitted from users is also text based. When do you think you will diversify more into multimedia content?

Tang: Right, it’s funny that you ask that because just this morning I executed an agreement that will bring video to Newsvine. We will always be primarily a text-heavy site. We already have audio, that accompanies some of the articles from the AP but soon we will also have video from one primary partner who I can’t name yet.

OJR: Do you provide any editorial oversight when something is submitted?

Tang: It’s hands off from our standpoint as a company. There is no editorial process that happens prior to an article being published by a Newsvine user.

However we depend on our community for that function. We have both an editorially and an user-driven policing system. There are a couple of ways this happens. As far as inappropriate content not just the correction of facts, we depend on a sophisticated reporting system.

This works amazingly well. It’s much more efficient than if we had somebody manning a desk 24/7 looking at user contributions and deciding yes or no.

The other system that we have in place is an area called the Greenhouse. This is a place that serves two functions. When you sign up for Newsvine you can’t immediately post onto Newsvine, onto our tag pages, or the content can’t get up on the front page. Articles or seeded links will show up the Greenhouse. In addition to keeping spammers out, it serves as a place to showcase new users. So if somebody just signed up it might be hard for them to get their material up in front of a lot of people but they are showcased in this new user area and if their content receives a certain number of votes and comments then we quickly promote them out of the Greenhouse and get them into Newsvine.

With this system, you deter spammers almost completely, because spammers are all about high yield, low effort propagation of their material.

OJR: What refinements can we expect in news personalization in general?

Tang: Well I think that a site can always be improved up on. It can always become more intuitive and the more that sites can do to accommodate users preferences without them explicitly having to set things up the better. For example there is no reason why you shouldn’t come to Newsvine, and we detect where you are based by looking at your IP address. Then we can give you headlines from your local papers in your area. I mean that’s something that we should be doing and we will be doing in the next month or two.

And also based on a user’s behavior we should be presenting you with information or news similar to the stuff that you’ve liked other places. We have a rudimentary function that shows recommended articles page based on the types of articles you voted on. Now here are articles that you did not vote on but voted on by other users, who we think, are similar to you. And in that sense we are showing you things that you might have missed but would have liked. That’s another example of being able to pick up passively on the behavior of a reader.

OJR: Social networking…

Tang: Yeah, but for these social networking sites or customized news sites, the first thing you have to do when you sign up is you have to customize all these things to your tastes. Now the more we can do that for you, I think the better.