Haoyao Ruan / Mathematics / Faculty Mentor: Xinlei (Sherry) Wang
Rank aggregation (RA), which aims to combine multiple ordinal rankings into a single, reliable ranking, has broad applications in fields such as elections, information retrieval, genomic research, and educational evaluation. In academic and professional settings, evaluation systems incorporating both peer-and self-assessments are crucial for understanding individual performance and ranker competence. However, subjective biases can lead to unreliable or unfair outcomes, especially in a small sample setting. Despite the broad applications of RA, existing methods often fail to address nuanced biases and the unique data mechanism of such evaluations. We propose BayeSRank (Bayesian Bias Detection in Peer-and Self-Ranking), a novel Bayesian method for bias-aware RA that jointly models both the importance of individual items and systematic biases of rankers. Through simulations and real-world data examples, we demonstrate BayeSRank’s superiority in generating interpretable, unbiased evaluations, especially in noisy data settings. Our work enhances fairness, transparency, and reliability in peer and self-evaluation systems, offering theoretical and practical implications for bias-aware ranking.
Leave a Reply