Automatic Construction of Forced-Choice Tests
Construct Forced-Choice Blocks Aligned with the Specifications in a Bl...
Build Variable Names for the Pairwise/Rank Responses in the TIRT Model
Calculation of Item Block "Energy" with IIAs Included
Calculation of Item Block "Energy"
Build a Blueprint Data Frame for the Focal FC Scale
Convert the Latent Utility Values into Thurstonian IRT Pairwise/Rank R...
Calculate the Empirical Reliability of the Latent Trait Scores, Follow...
Function for Checking If All Items in a Vector Are Unique
Fit the Thurstonian IRT Model with Long Format Response Data
Conduct Confirmatory Factor Analysis (CFA) and Obtain Parameter Estima...
Helper Function for Outputting IIA Characteristics of Each Block
Generate Simulated Person and Item Parameter Matrices for the Thurston...
Convert the TIRT Pairwise/Rank Response Data into Long Format Compatib...
Construction of Random Item Blocks
Scatter Plot for True vs Estimated Scores, True Score vs Absolute Erro...
Predict trait scores based on estimated model
Calculate the Overall RMSE of the Trait Scores, or the RMSE in a Certa...
Automatic Item Pairing Method in Forced-Choice Test Construction
Forced-choice (FC) response has gained increasing popularity and interest for its resistance to faking when well-designed (Cao & Drasgow, 2019 <doi:10.1037/apl0000414>). To established well-designed FC scales, typically each item within a block should measure different trait and have similar level of social desirability (Zhang et al., 2020 <doi:10.1177/1094428119836486>). Recent study also suggests the importance of high inter-item agreement of social desirability between items within a block (Pavlov et al., 2021 <doi:10.31234/osf.io/hmnrc>). In addition to this, FC developers may also need to maximize factor loading differences (Brown & Maydeu-Olivares, 2011 <doi:10.1177/0013164410375112>) or minimize item location differences (Cao & Drasgow, 2019 <doi:10.1037/apl0000414>) depending on scoring models. Decision of which items should be assigned to the same block, termed item pairing, is thus critical to the quality of an FC test. This pairing process is essentially an optimization process which is currently carried out manually. However, given that we often need to simultaneously meet multiple objectives, manual pairing becomes impractical or even not feasible once the number of latent traits and/or number of items per trait are relatively large. To address these problems, autoFC is developed as a practical tool for facilitating the automatic construction of FC tests (Li et al., 2022 <doi:10.1177/01466216211051726>), essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods. Given characteristics of each item (and item responses), FC measures can be constructed either automatically based on user-defined pairing criteria and weights, or based on exact specifications of each block (i.e., blueprint; see Li et al., 2024 <doi:10.1177/10944281241229784>). Users can also generate simulated responses based on the Thurstonian Item Response Theory model (Brown & Maydeu-Olivares, 2011 <doi:10.1177/0013164410375112>) and predict trait scores of simulated/actual respondents based on an estimated model.