The Fill-Mask Association Test
Download and save BERT models to local cache folder.
Scrape the initial commit date of BERT models.
Get basic information of BERT models.
Remove BERT models from local cache folder.
Check if mask words are in the model vocabulary.
A simple function equivalent to list.
Run the fill-mask pipeline and check the raw results.
Combine multiple query data.tables and renumber query ids.
Prepare a data.table of queries and variables for the FMAT.
Run the fill-mask pipeline on multiple models (CPU / GPU).
FMAT: The Fill-Mask Association Test
Intraclass correlation coefficient (ICC) of BERT models.
Reliability analysis (Cronbach's ) of LPR.
Set (change) HuggingFace cache folder temporarily.
Specify models that require special treatment to ensure accuracy.
[S3 method] Summarize the results for the FMAT.
Compute a vector of weights with a decay rate.
The Fill-Mask Association Test ('FMAT') <doi:10.1037/pspa0000396> is an integrative, probability-based social computing method using Masked Language Models to measure conceptual associations (e.g., attitudes, biases, stereotypes, social norms, cultural values) as propositional semantic representations in natural language. Supported language models include 'BERT' <doi:10.48550/arXiv.1810.04805> and its variants available at 'Hugging Face' <https://huggingface.co/models?pipeline_tag=fill-mask>. Methodological references and installation guidance are provided at <https://psychbruce.github.io/FMAT/>.