Online ratings look like direct summaries of customer opinion, but the scores people see are also shaped by platform design and ranking algorithms. A rating system collects reviews; an algorithm decides which reviews are shown, how they are ordered, and sometimes how they are combined into a displayed score. Research has found that ranking and recommendation systems can change exposure to reviews, while studies of online marketplaces document patterns such as rating inflation and differences in who leaves feedback. Understanding these mechanisms helps consumers interpret star ratings more carefully and helps businesses and platforms assess whether their systems are fair.
Algorithmic selection and online rating visibility
Algorithmic selection is the automated process of choosing, ranking, or filtering information for a particular user or page. The European Commission’s Digital Services Act describes recommender systems as automated systems that suggest or prioritise information, including through profiling. In ratings, this can mean that a platform gives more prominence to recent reviews, reviews judged helpful, verified purchases, or feedback that matches a user’s interests. The visible review set is therefore not necessarily a representative sample of every submitted rating.
This distinction matters because consumers often treat a prominent average or a handful of featured reviews as a neutral summary. Yet the displayed score may use a particular calculation, and the visible comments may have been selected or ordered by rules users cannot easily see. A 2021 review of online customer ratings by Luca and Zervas, published in the Journal of Economic Perspectives, describes how platform incentives and reviewer behavior can affect the information conveyed by ratings. The score is useful evidence, but it is not the same thing as an unbiased census of customer experience.
Ranking and review ordering
Review ranking determines which comments appear first. Platforms may sort by “most helpful,” relevance, recency, or a personalized estimate of usefulness rather than displaying reviews in chronological or random order. This can influence decisions even when the underlying set of ratings does not change: people tend to notice prominent items more than buried ones. A study by Luca and Zervas on Yelp found that businesses were more likely to receive suspicious reviews when competitors did, illustrating how strategic behavior can shape the material ranking systems are asked to evaluate.
Personalized recommendations and exposure
Personalization uses signals such as a person’s past activity, location, or preferences to select what they see. In rating contexts, personalization may affect which products, businesses, or reviews are surfaced, even if the platform does not alter the underlying star score. The National Institute of Standards and Technology’s work on identifying and managing bias in artificial intelligence emphasizes that bias can enter at multiple stages, including data collection, model design, and deployment. Consequently, two users may encounter different evidence about the same service or product.
Rating calculations and online rating distortion
A displayed rating is a numerical summary produced from submitted scores, but platforms do not all calculate or present it in the same way. A simple arithmetic mean gives every rating equal weight. Other systems may weight recent ratings, discount suspected manipulation, or use Bayesian adjustments that pull scores toward a broader average when there are few reviews. These choices can be reasonable, but they change what the number means. Consumers should look for information about the scale, review count, time period, and any explanation of how the score is calculated.
Rating inflation and selection effects
Rating inflation describes a pattern in which scores cluster toward the high end, making small differences harder to interpret. It can arise when satisfied customers are more likely to post, dissatisfied customers disengage, or a platform’s interface prompts feedback selectively. In a 2019 study of Uber drivers’ ratings, researchers found that ratings were highly concentrated near the maximum, limiting their ability to distinguish performance. The study demonstrates a broader measurement problem: a high average does not automatically mean that nearly all customers had an excellent experience.
Review selection and missing voices
Selection bias occurs when the people who submit ratings differ systematically from the full customer population. For example, customers with unusually positive or negative experiences may be more motivated to leave reviews, while people facing language, accessibility, or technology barriers may be less likely to participate. The displayed average then describes reviewers rather than all customers. The Pew Research Center has reported that online reviews are a common part of Americans’ consumer decision-making, but participation and trust vary across demographic groups and contexts. Review volume alone cannot establish that a sample is representative.
Manipulation detection and false positives
Platforms use automated tools to detect spam, coordinated posting, and deceptive reviews. Detection can improve reliability, but imperfect systems may remove legitimate criticism or leave manipulative content visible. The Federal Trade Commission’s 2024 rule on consumer reviews and testimonials prohibits specified deceptive practices, including certain fake reviews and incentives conditioned on particular sentiment. The rule addresses conduct; it does not guarantee that every platform’s detection or ranking process is accurate. Users should treat unusually repetitive, vague, or tightly clustered reviews as signals to investigate further, not as proof of fraud.
Consequences of algorithm bias in online ratings
Algorithmic bias in ratings is not limited to a faulty mathematical formula. It can result from unbalanced data, unequal participation, business incentives, or design decisions about what counts as a useful review. The consequences reach beyond individual purchases: ratings can influence income, reputation, access to customers, and trust in digital marketplaces.
Consumer decisions and unequal visibility
Consumers may mistake a highly visible review for a typical one, or assume that a small score difference reflects a meaningful quality gap. Search placement and featured feedback can also direct attention toward some businesses and away from others. The Organisation for Economic Co-operation and Development has noted that online ratings and reviews can reduce information gaps for consumers, while warning that fake or misleading reviews weaken that benefit. Comparing review counts, reading a range of comments, and checking whether feedback is recent can help users avoid overreliance on a single displayed number.
Business reputation and economic effects
For businesses, a small change in rating visibility may affect customer traffic, especially where platforms use ratings in search or recommendation features. A 2016 study by Michael Luca in the Harvard Business School Working Paper found that a one-star increase in a restaurant’s Yelp rating was associated with a 5–9% increase in revenue for independent restaurants in the study’s setting. This finding does not mean every platform or business will experience the same effect, but it illustrates why rating systems can carry material economic consequences and why opaque ranking decisions deserve scrutiny.
Fairness, accountability, and platform design
Fairness in rating systems means more than equal treatment in theory: platforms should test whether their rules produce systematically different visibility or outcomes for particular groups, businesses, or types of feedback. The NIST AI Risk Management Framework recommends governing, mapping, measuring, and managing AI risks throughout a system’s lifecycle. Applied to ratings, this supports practices such as documenting ranking criteria, auditing outcomes, offering accessible review processes, and giving users meaningful explanations when content is filtered or a score is adjusted.
How to read online ratings more critically
Consumers do not need to reverse-engineer a platform to make better decisions. They can treat ratings as one input among several and examine how the evidence was assembled. Businesses and regulators, meanwhile, can encourage clearer explanations and independent audits of systems that materially influence reputation or access to customers.
- Check the number of reviews as well as the average score; a high score based on few ratings is less stable.
- Read recent and critical reviews alongside positive ones, and look for specific, verifiable details.
- Compare ratings across more than one platform, while remembering that each platform may attract a different reviewer population.
- Notice whether the page explains its sorting method, verification labels, or score calculation.
- For consequential choices, confirm claims through independent sources such as professional credentials, inspection records, or direct conversations.
Conclusion: interpreting algorithm-shaped ratings
Online ratings are shaped by both human behavior and algorithmic decisions. Ranking and personalization determine which feedback is visible; scoring methods and selection effects shape the average; and automated moderation can affect which reviews remain. Research on rating inflation, review behavior, and economic effects shows why a star score should be read as a platform-produced signal rather than a perfect measure of overall quality. Consumers can compare sources and inspect the underlying comments, while platforms and regulators should support transparent methods, representative participation, and ongoing bias audits. Further reading from NIST, the FTC, and the OECD can help users and organizations assess how digital rating systems are designed and governed.
Sources: European Commission, Digital Services Act, https://digital-strategy.ec.europa.eu/en/policies/digital-services-act-package; Luca, Michael, and Georgios Zervas, “Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud,” Journal of Economic Perspectives, 2021, https://www.aeaweb.org/articles?id=10.1257/jep.35.1.45; National Institute of Standards and Technology, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, 2022, https://doi.org/10.6028/NIST.SP.1270; Anderson, Michael, and Jeremy Magruder, “Learning from the Crowd: Regression Discontinuity Estimates of the Effects of an Online Review Database,” The Economic Journal, 2012, https://doi.org/10.1111/j.1468-0297.2011.02456.x; Pew Research Center, “Online Reviews,” 2016, https://www.pewresearch.org/internet/2016/12/19/online-reviews/; Federal Trade Commission, Trade Regulation Rule on the Use of Consumer Reviews and Testimonials, 2024, https://www.ftc.gov/legal-library/browse/rules/rule-use-consumer-reviews-testimonials; National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023, https://doi.org/10.6028/NIST.AI.100-1; Organisation for Economic Co-operation and Development, Online Consumer Reviews: The Challenge of Ensuring a Safe and Positive Online Environment, 2019, https://www.oecd.org/sti/consumer/online-consumer-reviews.htm; Luca, Michael, Reviews, Reputation, and Revenue: The Case of Yelp.com, Harvard Business School Working Paper, 2016, https://www.hbs.edu/ris/Publication%20Files/12-016_46c2b7f2-7f10-4b9e-8e4e-5b1a2c8a5c52.pdf
