Preference Rating for Visual Stimuli Based on Elo Ratings
Elo-ratings for pairwise comparisons of visual stimuli
transform preference data
generate random data of pairwise preference ratings
reliability with progressive rater inclusion
indiviual stimulus ratings
calculate reliability-index
update stimulus ratings after one rating event
summarize elochoice object
calculate ratings from sequence of rating events, allowing for more th...
Allows calculating global scores for characteristics of visual stimuli as assessed by human raters. Stimuli are presented as sequence of pairwise comparisons ('contests'), during each of which a rater expresses preference for one stimulus over the other (forced choice). The algorithm for calculating global scores is based on Elo rating, which updates individual scores after each single pairwise contest. Elo rating is widely used to rank chess players according to their performance. Its core feature is that dyadic contests with expected outcomes lead to smaller changes of participants' scores than outcomes that were unexpected. As such, Elo rating is an efficient tool to rate individual stimuli when a large number of such stimuli are paired against each other in the context of experiments where the goal is to rank stimuli according to some characteristic of interest. Clark et al (2018) <doi:10.1371/journal.pone.0190393> provide details.