# Do Online Doctor Ratings Really Reflect Quality Care? What a New Study Found

Millions of patients check online physician ratings before choosing a doctor, but a new study from Cedars-Sinai Medical Center reveals that those star ratings do not actually predict real-world quality of care, cost efficiency, or peer evaluations of physician performance. After analyzing 78 specialists across 8 medical and surgical specialties using data from the 5 most popular physician-rating websites, researchers found no meaningful association between consumer ratings and objective measures of clinical quality. The findings suggest that patients should not rely solely on online reviews when making important healthcare decisions.

# Do Online Doctor Ratings Really Reflect Quality Care? What a New Study Found

## Table of Contents

- Key Points
- Why This Research Matters: The Rise of Online Doctor Ratings
- How the Study Was Conducted
- Key Findings: Ratings Don't Predict Quality, Value, or Peer Assessment
- Are Ratings at Least Consistent Across Websites?
- What This Means for Patients
- Study Limitations
- Recommendations for Patients and the Healthcare Industry
- Frequently Asked Questions
- Source Information

## Key Points

- In a Cedars-Sinai study of 78 specialists, online doctor ratings showed no meaningful association with quality, value, or peer assessment of care.
- Physicians with high star ratings were no more likely to deliver high-quality care than those with lower ratings, according to the study.
- Only 5%–32% of physicians scoring in the lowest quartile on performance had consumer ratings in the lowest quartile; 68%–95% were missed.
- Ratings were generally consistent across five major websites, but that consistency did not make them valid predictors of performance, the study found.
- Researchers advise patients to use online ratings only as one input, alongside referrals, quality reports, and in-person communication assessment.

## Why This Research Matters: The Rise of Online Doctor Ratings

Choosing a physician is one of the most personal and consequential healthcare decisions a person can make. Yet, for many patients, the process has become heavily influenced by online consumer ratings — the same kind of 5-star reviews used to evaluate restaurants, hotels, and products.

The popularity of these ratings is undeniable. A 2012 survey of 2,137 US adults found that **65% were aware of physician-rating websites**, and **59% reported that these sites were "somewhat important" or "very important"** in choosing a physician. Even more striking, a survey of 1,000 surgical patients at the Mayo Clinic revealed that **81% would seek a consultation with a physician based on positive reviews alone**, and **77% would not seek a consultation based solely on negative reviews**. These trends are not unique to the United States — similar patterns have been reported in Europe.

Health insurance companies have taken notice. Many payers now incorporate online consumer ratings into their physician search tools, suggesting that insurers also believe these ratings are trustworthy measures of real clinical performance. Given this trajectory, researchers say consumer ratings will play an increasingly prominent role in how patients choose their physicians in the future.

But there's a critical problem: Despite widespread use, it has been unclear whether online ratings actually identify high-performing physicians — or even what they are measuring. The most popular rating platforms offer no guidance on what criteria should be used when assessing a physician, making the results difficult to interpret. Some experts believed the ratings captured "ancillary" aspects of care like office environment and staff friendliness rather than clinical performance, but there was no strong data to support this claim. In fact, the limited available data suggested the opposite might be true.

This study was designed to answer a fundamental question: **Do online consumer ratings actually predict how well a physician performs?**

## How the Study Was Conducted

Researchers from Cedars-Sinai Medical Center, an **886-bed tertiary referral, hybrid academic-community hospital in Los Angeles**, conducted an observational study of **78 physicians** representing **8 medical and surgical specialties**. The specialties included cardiology (25 physicians, 32%), neurology (11 physicians, 14%), nephrology (10 physicians, 13%), obstetrics/gynecology (9 physicians, 12%), endocrinology (7 physicians, 9%), otolaryngology (6 physicians, 8%), gastroenterology (6 physicians, 8%), and general surgery (4 physicians, 5%).

Initially, 123 practicing physicians in the Cedars-Sinai medical delivery network were eligible for inclusion. However, 30 were excluded because they were not using the health system's electronic health record in their ambulatory practice, 14 were excluded because they received fewer than 3 peer-review survey responses, and 1 was excluded because they had no online consumer rating data available.

The study population was diverse across practice settings:

-
**Race:** 50 Caucasian (64%), 19 Asian (24%), 9 other (12%)
-
**Gender:** 66 male (84%), 12 female (16%)
-
**Practice type:** 36 in the Cedars-Sinai Medical Group (46%), 34 in private practice (44%), 8 in the faculty group (10%)
-
**Years in practice:** median of 33.5 years (range 19–43 years)

### How Performance Was Measured

To capture clinical performance comprehensively, the researchers used **three separate assessment tools**, each examining a different dimension of physician quality:

**1. Specialty-specific performance scores.** This composite metric combined objective measures of both quality and value of care. Quality measures included adherence to **Choosing Wisely measures** (a national initiative to reduce unnecessary medical tests and procedures) and best practice alerts. Value-of-care measures included the **use of generic medications** when appropriate and **case mix–adjusted length of stay** (length of hospital stay adjusted for how sick patients are). The specific metrics varied by specialty and were developed through an intensive process of stakeholder engagement with specialty physicians. Each metric was assigned a weight between 0 and 100 based on relevance to performance. Physicians were scored for fiscal years 2013–2014, and metrics were dropped for physicians with fewer than 20 qualifying encounters. Importantly, score calculations were performed by analysts who were **blinded to consumer rating results**, preventing bias.

**2. Primary care physician (PCP) peer-review scores.** Thirty-two primary care physicians rated each specialist using a **9-point modified RAND scale** (ranging from 1 = "completely disagree" to 9 = "completely agree") based on how much they agreed with the statement: *"Dr. [Name] is a strong partner in value-based healthcare."* To avoid order effects, the survey presented specialist profiles in random order unique to each respondent, and the average rating for each physician was used in the analysis.

**3. Administrator survey scores.** Twenty administrators and hospital leaders — including departmental leaders, directors of inpatient clinical programs (hospitalist medicine, emergency department, intensive care), and personnel with broad-based knowledge of specialist performance (chief of medical staff, academic dean) — rated the same specialists using the identical 9-point scale and question. The study also surveyed groups of relevance for individual specialties (for example, dialysis staff for nephrologists, endoscopy staff for gastroenterologists, and anesthesiologists for surgeons).

### How Online Ratings Were Collected

Consumer ratings were collected from the **5 most popular online platforms according to Google Trends**: **Healthgrades, Vitals, Yelp, RateMDs, and UCompareHealth**. Each website invites consumers to rate physicians using a **5-star scale**. The researchers collected data on each physician's average rating and the number of ratings that informed that average.

The median number of consumer ratings per physician was **relatively low across all platforms**, ranging from 4 to 9 reviews per doctor. This means most doctors in the study had fewer than 10 online reviews to their name. Median star ratings were fairly high across all platforms, ranging from 4.0 to 4.5 stars, suggesting that typical online ratings skew positive.

The researchers then used advanced statistical methods — including multivariable linear regression models — to determine whether consumer ratings were independently associated with any of the performance measures, while controlling for the number of reviews, physician specialty, race, gender, years since medical school graduation, and practice setting.

## Key Findings: Ratings Don't Predict Quality, Value, or Peer Assessment

The results were strikingly clear: **Online consumer ratings showed no meaningful relationship with any measure of actual physician performance.**

### Finding 1: No Association Between Ratings and Overall Performance

Scatterplots and statistical analyses revealed either a **negative or weak correlation** between consumer ratings and specialty-specific performance scores across all platforms (correlation coefficient range: **−0.18 to 0.02**). In other words, physicians with high star ratings were no more likely to deliver high-quality care than those with lower ratings — and in some cases, higher ratings were actually associated with slightly worse performance.

Multivariable regression models told the same story. After adjusting for potential confounding factors, there was **no statistically significant or meaningful association** between consumer ratings and performance scores (b-coefficient range: **−0.04 to 0.04**). To put this in perspective, for every 1-star increase in a physician's online rating, their performance score changed by less than one-twentieth of a point — a negligible difference.

### Finding 2: No Association with Peer Review Scores

Consumer ratings also failed to predict how physicians were perceived by their colleagues. There was **no significant association** between online ratings and primary care physician peer-review scores (b-coefficient range: **−0.01 to 0.3**) or administrator scores (b-coefficient range: **−0.2 to 0.1**). The correlations were weak or negative across all platforms for both types of peer assessment.

### Finding 3: No Association with Quality or Value Subdomains

The researchers dug deeper, examining whether ratings might at least predict specific aspects of quality, such as adherence to Choosing Wisely measures and best practice alerts, or specific aspects of value, such as appropriate generic medication use and length of hospital stay. The results: **no relationship between consumer scores and any subdomain of performance** related to either quality or value-based care.

### Finding 4: Ratings Fail to Identify Low-Performing Physicians

One of the most compelling findings concerns the ability of ratings to flag doctors who may be underperforming. Among physicians who scored in the **lowest quartile (bottom 25%) of specialty-specific performance scores**, only **5%–32% had consumer ratings in the lowest quartile** across the various platforms. This means that the vast majority of low-performing physicians still received mid-range or even high online ratings.

In practical terms, if a patient used online ratings to find the "worst" doctors by filtering for the lowest-starred physicians, they would miss **68% to 95% of those who actually scored poorly on objective quality measures**.

## Are Ratings at Least Consistent Across Websites?

The study also asked a second question: Do the different rating websites at least agree with each other? If a doctor is rated 5 stars on one platform, are they likely to be rated 5 stars on another?

The answer was partially yes. Consumer ratings of individual physicians were **generally consistent across platforms**. A physician's score on one platform significantly predicted their score on another platform in **5 out of 10 pairwise comparisons**, and positive relationships were observed among all platforms overall.

For example, Yelp ratings significantly predicted Healthgrades ratings (b-coefficient 0.28), Vitals ratings (0.52), and UCompareHealthcare ratings (0.45). Vitals ratings significantly predicted Yelp (0.71), RateMDs (0.60), and UCompareHealthcare (0.96) ratings.

This consistency across platforms suggests that online ratings **jointly measure some underlying factor** — what statisticians call a "latent construct" — but that construct appears to be **unrelated to clinical performance**. In other words, the sites are consistently measuring *something*, just not quality of care.

What might that something be? The researchers note that it may, in part, be patient satisfaction. A previous study found that online reviews were modestly associated with National Committee for Quality Assurance consumer satisfaction scores for insurance plans (Pearson correlation = 0.376). Defining this latent construct is critical, the authors argue, because it would at least tell patients what the star ratings actually mean — and they strongly influence provider selection for the majority of patients.

## What This Means for Patients

These findings have direct and important implications for the estimated **80% of healthcare consumers who are currently selecting physicians based on ratings alone**. If you're one of those patients, here's what the study means for you:

-
**A 5-star rating does not mean a doctor provides higher-quality care** than a colleague with 3 stars. In fact, this study found no relationship — and sometimes a slightly negative one — between ratings and objective quality measures.
-
**Online ratings won't help you identify low-performing physicians.** The vast majority of doctors with poor performance scores in this study still had mid-range or high online ratings.
-
**Ratings might measure the "experience" of care** (how patients felt about their visit) rather than the "clinical" quality of care (how well the doctor treats disease). These are related but very different things.
-
**Cross-platform consistency doesn't mean the ratings are valid.** Even though a doctor tends to get similar ratings across different websites, this study shows those consistent scores still don't predict performance. It's like multiple scales all weighing the same wrong ingredient.

This is not to say that patient opinions don't matter. The researchers emphasize that patient assessments of physicians are important and that this information is "here to stay." But the study underscores the need to use online ratings as **one piece of a much larger puzzle** — not the sole basis for choosing a doctor.

## Study Limitations

Every scientific study has limitations, and the authors were transparent about several of these:

-
**Sample size and setting.** This was a single-institution study conducted at one large urban health delivery system in Los Angeles. While the sample included diverse practice settings (faculty, medical group, and private practice), the results may not fully generalize to other regions, smaller hospitals, or rural healthcare settings.
-
**Potential selection bias.** The peer-review surveys, while carefully designed, may not have captured all dimensions of physician performance. Additionally, the study relied on physicians with at least 3 peer-review responses, which could have excluded some specialists.
-
**Limited number of online reviews per physician.** With median review counts of just 4–9 per physician across platforms, the average ratings were based on very small samples. A single additional negative review could substantially change a physician's average score. This reflects the real-world reality of sparsely populated rating platforms, but it also means the study was working with somewhat unstable rating estimates.
-
**Avoidance of pure academic settings.** The authors note they did not have access to Consumer Assessment of Healthcare Providers and Systems (CAHPS) data or formal patient satisfaction scores, and they acknowledged the sampling bias associated with testing the same question in purely academic settings.
-
**Observational design.** As with all observational studies, the researchers could demonstrate associations (or, in this case, the lack thereof) but could not prove causation.

Despite these limitations, the study's methodology was robust: it assessed physician performance across **multiple dimensions** (quality, value, peer assessment from both PCPs and administrators), used **objective, specialty-specific metrics** developed with stakeholder engagement, and adjusted for important physician characteristics in its statistical models. The authors argue it is unlikely that larger studies will be able to assess physician performance on such a granular and multimodal level.

## Recommendations for Patients and the Healthcare Industry

Given the evidence, what should patients do — and how can the healthcare industry improve physician rating systems? The researchers offered several specific recommendations.

### For Patients

1.
**Don't rely on online ratings alone.** The study's conclusion is unambiguous: "Online consumer ratings should not be used in isolation to select physicians." Use ratings as just one input alongside other sources of information.
1.
**Seek objective quality data.** Look for publicly available data on hospital and physician quality measures. Ask your primary care doctor for a referral — this study found that PCPs' peer assessments of specialists did not correlate with online ratings, meaning their recommendations may add genuinely different (and useful) information.
1.
**Consider what ratings actually measure.** If you do read online reviews, treat them as reflections of patient experience — waiting room times, bedside manner, communication style — rather than measures of medical skill or outcomes.
1.
**Check multiple sources.** While ratings are consistent across platforms, that consistency says little about validity. Instead, diversify the types of information you consult: insurance networks, hospital quality reports, consultation with other healthcare professionals, and in-person assessment of whether you feel heard and respected by the physician.

### For the Healthcare Industry and Rating Websites

1.
**Create a definable construct.** Consumer rating websites should clarify exactly what they intend ratings to measure, with clear instructions on how patients should evaluate physicians across metrics they are well-positioned to assess (such as communication and shared decision-making). The content should be informed by both expert and patient stakeholders to maximize relevance and content validity.
1.
**Pair ratings with performance data.** Online ratings should be presented alongside complementary data on quality and value of care, since this study demonstrates that these critically important data are *not* captured by consumer ratings.
1.
**Accept responsibility for limitations.** Companies offering physician rating services have a duty to help their customers understand the limitations of consumer ratings when selecting physicians — especially because the stakes are higher than in any other consumer purchase context.

### The Validity Problem

The researchers also applied a psychometric framework — the same standards used to validate any questionnaire — to evaluate online physician ratings. They concluded that the ratings **fail even basic tests of validity**:

-
**Face validity (does it look like it measures what it should?):** Fails. In the absence of any recommendations on how to assess physicians, it's unclear whether a 5-star rating measures any explicit construct at all.
-
**Content validity (does it cover the important aspects?):** Fails. No consistent categories of patient experience are measured across platforms.
-
**Predictive validity (does it predict future performance?):** Fails. This study shows ratings do not predict physician performance on quality, value, or peer-review metrics.
-
**Concurrent validity (does it distinguish high from low performers?):** Fails. Only 5%–32% of low-performing physicians were captured by low ratings.

Prior research had shown that online ratings can predict some *coarse* measures of quality, such as board certification, quality of medical education, physician volume, and hospital-level outcomes (including Hospital Consumer Assessment of Healthcare Providers and Systems scores and hospital readmission rates for heart attack, heart failure, and pneumonia). A UK study of National Health Service Choices data found moderate associations between "recommended" physician status and patient experience (Spearman's 0.37–0.48, P < .001) but only weak associations with clinical process and outcome measures (Spearman's less than 0.18, P < .001).

However, this new study is unique in its granular, multidimensional assessment of individual physician performance across quality, value, and peer-review metrics simultaneously — and its findings make clear that **the granularity of physician-level quality data matters**. What holds true at the institutional level does not hold true at the individual physician level.

The bottom line, according to the researchers: "Our results support what has been a longstanding belief among healthcare stakeholders: that consumer rating scores are not reflective of the basic quality and value of care provided by an individual physician — which are unquestionably among the most important criteria to consider when selecting a healthcare provider."

## Frequently Asked Questions

### Do online doctor ratings reflect the quality of care a doctor provides?

No, according to a Cedars-Sinai study of 78 specialists. Researchers found no meaningful relationship between online star ratings and objective measures of clinical quality, value of care, or peer evaluations. High ratings did not predict better performance, and some doctors with high ratings had slightly worse performance scores.

### Should I use online ratings alone to choose a physician?

The researchers advise against it. Their study found that online ratings do not predict quality, value, or peer assessments. If you used lowest ratings to find poor performers, you would miss 68% to 95% of doctors who actually scored low on objective quality measures. Use ratings only as one input among many.

### What did the study measure to evaluate physician performance?

Researchers measured three things for each of 78 specialists: specialty-specific performance scores combining quality measures like Choosing Wisely adherence and value measures like appropriate generic use and hospital length of stay; peer-review scores from primary care physicians; and administrator survey scores. They then compared these to online star ratings.

### Are online ratings at least consistent across different websites?

Yes, partially. Ratings for the same physician were generally consistent across the five major platforms, with positive relationships in most pairwise comparisons. However, this consistency does not mean the ratings are valid. The study showed these consistent scores still do not predict clinical performance, value of care, or peer evaluations.

### What do online ratings actually measure, if not quality?

The study suggests online ratings may partly reflect patient satisfaction or the experience of care, such as waiting times, bedside manner, and communication. However, the researchers say it is unclear what the ratings truly measure. They found ratings do not predict quality, value, or peer assessment of physician performance.

### What are the limitations of this study on doctor ratings?

This was a single-institution study in Los Angeles with 78 specialists, so results may not apply everywhere. The median number of online reviews per physician was low, just 4 to 9, making average ratings unstable. Also, the study could show associations but not prove causation, and it did not include formal patient satisfaction scores.

### What should patients do instead of relying on online ratings?

Ask your primary care doctor for a referral, since peer assessments did not match online ratings and may add useful information. Look for publicly available quality data on hospitals and physicians. If you read reviews, treat them as reflections of patient experience, not medical skill. Check multiple sources and assess whether you feel heard by the doctor.

### When should I seek a second opinion instead of relying on online doctor ratings?

Online physician ratings do not predict actual quality of care, cost efficiency, or peer evaluations. Among 78 specialists studied, consumer star ratings had no meaningful association with any objective performance measure, and most low-performing physicians still received average or high ratings. For important healthcare decisions, ratings should be used only as one input. An independent expert second opinion can provide reliable guidance that online reviews cannot, particularly when choosing a physician for significant treatment. Diagnostic Detectives Network provides independent expert second opinions.

## Source Information

**Original Article Title:** Online physician ratings fail to predict actual performance

**DOI:** [10.1093/jamia/ocx083](https://doi.org/10.1093/jamia/ocx083)

**Authors:** Timothy J. Daskivich, Justin Houman, Garth Fuller, Jeanne T. Black, Hyung L. Kim, and Brennan Spiegel

**Journal:** Journal of the American Medical Informatics Association (JAMIA), Volume 25, Issue 4, 2018, pages 401–407

**DOI:** 10.1093/jamia/ocx083

**Publication Details:** Advance Access publication date: September 8, 2017. Received March 13, 2017; Revised May 18, 2017; Accepted August 21, 2017.

**Affiliations:** Division of Urology, Cedars-Sinai Medical Center; Cedars-Sinai Center for Outcomes Research and Education (CS-CORE); Department of Medicine, Division of Health Services Research, Cedars-Sinai Health System; Resource and Outcomes Management Department, Cedars-Sinai Health System; and Department of Health Policy and Management, UCLA Fielding School of Public Health, Los Angeles, CA, USA.

This patient-friendly article is based on peer-reviewed research. It is intended for informational purposes and does not constitute medical advice. Always consult with qualified healthcare professionals when making decisions about your healthcare.


---
Publisher: Diagnostic Detectives Network (https://diagnosticdetectives.com) — independent multi-expert medical second opinions, worldwide, private-pay.
Author byline: Anton Titov, MD, PhD. Contact: https://diagnosticdetectives.com/pages/contact
Canonical page: https://diagnosticdetectives.com/products/do-online-doctor-ratings-really-reflect-quality-care-what-a-new-study-found