Analysis: The trouble with Facebook's plan to rate media credibility

When Facebook announced a plan to rely on surveys of users to rank news organizations' credibility and give greater prominence to "broadly trusted" outlets, it shared few details. On Tuesday, BuzzFeed reported the exact questions Facebook plans to ask:

Q: Do you recognize the following websites

- Yes

- No

Q: How much do you trust each of these domains?

- Entirely

- A lot

- Somewhat

- Barely

- Not at all

Facebook has not described exactly how a trusted outlet will be defined, based on the survey responses, but chief executive Mark Zuckerberg summed up the theory behind using surveys to rate credibility in a Friday post on the social network site:

"The idea is that some news organizations are only trusted by their readers or watchers, and others are broadly trusted across society, even by those who don't follow them directly."

Neither of Facebook's questions would allow it to compare the level of trust reported by a news outlet's core audience with the level of trust "across society," but Zuckerberg's reasoning seems logical. If an outlet is trusted across the political spectrum, not merely by a partisan audience, it's reasonable to assume it is credible.

What might this classification of "broadly trusted" outlets look like? Thanks to a 2014 Pew Research Center survey, we don't have to guess. Pew asked a random national sample of 2,901 adults whether they had heard of 36 news organizations and, if so, to say whether they trusted or distrusted each.

The poll found that while a number of major outlets were more trusted than distrusted among the public overall, very few were trusted across the political spectrum. In fact, only one news organization, the Wall Street Journal, was more trusted than distrusted among all five of Pew's ideological groups, from "consistently liberal" to "consistently conservative."

Theoretically, if Facebook imposed a credibility threshold of being more trusted than distrusted across ideological groups, it would penalize all major TV networks from CBS to Fox News, as well as newspapers such as the New York Times, USA Today and The Washington Post.

The stark political divisions won't be surprising to many, but they point to a central difficulty for Facebook's attempt to judge news organizations' credibility based on public perceptions. Americans' judgments of trustworthiness appear to be based not solely on whether news organizations report accurately and fairly but also on whether reports align with audience members' political opinions.

Anya Schiffrin, director of the technology, media, and communications specialization at Columbia University's School of International and Public Affairs, suggested that Americans are regressing in the way they decide which news sources to believe.

"A hundred years ago, it was almost like today, where people trusted the information from the person that they knew personally," said Schiffrin, who published a study of media trust. "The great thing about the 19th and 20th centuries is that we evolved beyond that. We don't want to get back to a situation where you only trust information if you know the person giving it to you ... It may be that people are watching something because it panders to them."

In a poll published this month by Gallup and the Knight Foundation, only 42 percent of Americans could name a news source that they considered objective. People who identified as very liberal or very conservative were more likely than self-described moderates to say they could think of an objective news source - perhaps because they were thinking of sources that reinforce their views, and were calling those sources objective.

And to Schiffrin's point about backsliding: 32 percent of Americans in the Gallup-Knight survey said news outlets generally are careful to separate fact from opinion, down from 58 percent who said the same in 1984.

A 2009 Pew survey found that Fox News is the only national TV network that consistent conservatives trust more than they distrust; Fox News also was the only network that more Americans considered "mostly conservative" than "mostly liberal." The poll found that CNN, NBC, ABC, CBS and MSNBC were each seen as more liberal than conservative, aligning with the 2014 survey finding consistent liberals trust these networks, while consistent conservatives do not.

Given these results, Facebook will have a significant challenge distilling an effective measure of credibility from a partisan pot of media trust.