Validating Topic Coherence and Topic Labels
Check Agreement Rate between Identical Trails
Combine the mass of words with the same root
Evaluate results
Get results from Mturk
Mix the gold-standard tasks with the tasks need to be validated
Pick the optimal label from candidate labels
Plot results
Reform tasks to facilitate sending to Mturk
Send prepared task to Mturk and record the API-returned HIT ids.
Tidy eval helpers
Topic_Model_Validation Repository Overview
Create validation tasks for labels assigned to the topics in the topic...
Create validation tasks for topic model selection
By creating crowd-sourcing tasks that can be easily posted and results retrieved using Amazon's Mechanical Turk (MTurk) API, researchers can use this solution to validate the quality of topics obtained from unsupervised or semi-supervised learning methods, and the relevance of topic labels assigned. This helps ensure that the topic modeling results are accurate and useful for research purposes. See Ying and others (2022) <doi:10.1101/2023.05.02.538599>. For more information, please visit <https://github.com/Triads-Developer/Topic_Model_Validation>.