What Yelp can tell you about a hospital that official ratings can't

If you've ever taken the time to give Yelp your two cents about a hospital, you'll be happy to know that someone's listening and that they've deemed the crowdsourced information not only useful - but unique.

In what is believed to be the first large-scale analysis of such data, researchers from the University of Pennsylvania looked at 17,000 Yelp reviews of 1,352 hospitals from consumers.

They found that the online information provides a broader sense of a facility than the current gold standard - a U.S. government survey that costs millions of dollars to develop and implement each year.

The Hospital Consumer Assessment of Healthcare Providers and Systems Survey or HCAHPS (pronounced "H-Caps") has been used since 2006 and involves asking discharged patients questions about their stays. It covers 11 main categories including communication with nurses and doctors, responsiveness of staff, the cleanliness and quietness of the hospital environment, and pain management.

Yelp offers consumers the ability to rate hospitals on a scale of one to five stars and write a review to accompany that rating.

The U-Penn researchers used natural language processing to take apart the narratives and put them into buckets that were similar to the categories used by the HCAHPS. They gave as an example a post that had words such as "pain," "nurse," "medication," "gave" and how that might be assigned to the pain category.

Their paper, published in the April issue of Health Affairs, found that Yelp reviews encompassed only about seven of the 11 categories covered by the HCAHPS. That was disappointing.

But there was also a big surprise in the data.

The Yelp reviews had information about 12 additional categories that weren't addressed in the government survey.

Those include the cost of the hospital visit, insurance and billing, ancillary testing, facilities, amenities, scheduling, compassion of staff, family member care, quality of nursing, quality of staff, quality of technical aspects of care, and specific type of medical care.

For positive reviews they included caring doctors, nurses and staff; comforting; surgery/procedure and peri-op; and labor and delivery. And for negative reviews, they included insurance and billing and cost of hospital visit.

"They relate to the interpersonal relationships of patients with physicians, nurses and staff," said Benjamin L. Ranard, a junior fellow at the Penn Social Media and Health Innovation Lab and the study's lead author.

This is important, he said, because "prospective patients are likely to want to know how caring and comforting caregivers are in various departments of a hospital." It's not only useful for consumers, but also for hospital administrators, caregivers and policymakers.

The publication of the paper comes at a key time for Yelp when the social media site is trying to transform itself from a place where you can rave about how closely your pad thai resembles what you ate three years ago while on vacation in Bangkok to a more serious player in other consumer domains.

Yelp, in partnership with ProPublica, a nonprofit news organization, recently added information about a hospital emergency room's average wait time, fines paid by a nursing home and other information about health care facilities.

In 2012, the U-Penn researchers note, almost 75 percent of Americans looked for health information online and, while 42 percent looked at health-related reviews by consumers, only 6 percent had heard of the website where the results of the HCAHPS are publicly available.

"This divergence presents an opportunity for online consumer reviews to augment and even improve formal rating systems," said Raina M. Merchant, an assistant professor of emergency medicine at Penn's Perelman School of Medicine.

The researchers acknowledge that the reviews on sites such as Yelp have their weaknesses - they "are not currently randomized, are largely uncurated, unvalidated, and subject to gaming."

However, they noted that the reviews are "continuously updated, and often reveal in precise detail what the problem or positive occurrence was that affected the patient's or family member's experience."

Oh, and they're free, too.