wdm function

Automated Test Assembly via Weighted (positive) Deviations Method

Ingests item metadata jointly with target test form constraints and uses the Weighted (positive) Deviations Method (WDM) to construct a test form based on the desired objectives.

wdm( ipool,
 id,
 constraints,
 first = NA,
 refine = TRUE,
 permutate = FALSE,
 tieselect = 1,
 verbose = TRUE,
 aprioriadd = NA,
 posthocadd = NA )

Arguments

• ipool: Item by characteristic (property) metadata pool.
• id: Name of unique item identifier.
• constraints: Complex list object identifying the constraints to be applied in the ATA (see makeconstobj for guided process).
• first: How should item selection start: id of the item to be selected first from the pool, NA (default) - select randomly from the pool.
• refine: Should the final test form be refined against the remaining item pool? Default is TRUE.
• permutate: Assemble test forms starting with each item sequentially (as many forms as items in pool) and define final test form based on eligible constraint compliant solutions; Default is FALSE (currently not available).
• tieselect: How should tied items be resolved: 1 (default) - select the first item in the list of candidates (sensitive to data sorting); not applicable for situations with all categorical constraints only, 0 - randomly select candidate; not applicable for situations with all categorical constraints only
• verbose: Should progress of wdm() be printed to the console? Default = TRUE.
• aprioriadd: Force item addition (via IDs) to test before ATA, which affects item selection and constraint attainment success (currently not available).
• posthocadd: Force item addition (via IDs) to test after ATA, which affects final form specifications (currently not available).

Returns

A complex list object with test assembly specific estimates: - wde: Estimates of the computational steps deriving the positive weighted deviations and item selection.

• evaluation: Final assembled test form additive properties across constraints.

• considered: Estimates of the computational steps when refine = TRUE evaluating selected items and selecting replacements.

• excluded: Items from pool excluded.

• excluded_set: Item sets excluded. Only included if input constobj includes a set_id.

• included: Items from pool included in new test form.

• included_set: Item sets from pool included in new test form. Only included if input constobj includes a set_id.

• initial_ids: Item sets from pool included in new test form.

• initial_setids: Item sets from pool included in new test form. Only included if input constobj includes a set_id.

• final_ids: Final item ids in the test form.

• final_setids: Final set ids in the test form. Only included if input constobj includes a set_id.

Examples

# Specifying constraints
constin <- list(
  nI = 5,                                     # Number of items on the future test
  nC = 4,                                     # Number of constraints to be satisfied
  nameC = c("Content_A","Content_B","p","iSx"), # Name of constraint; must be numeric and must
  # reflect variable name in input
  lowerC = c(2, 3, 3.00, 0.50),               # Lower bound total constraint value on ATA form
  upperC = c(2, 3, 3.50, 0.60),               # Upper bound total constraint value on ATA form
  wC = c(1, 1, 1, 1),                         # Constraint weight used for weighted sum of
                                              # (positive) deviations St
  set_id = NA                                 # Aggregation ID for units / sets 
)

# Running WDM example from Parshall et al. (2002)
testWDM <- wdm( ipool = metadata_example,
                id = "Item",
                constraints = constin,
                first = 2)

# Summary of results
summary(testWDM)

References

Parshall, C. G., et al. (2002). Automated test assembly for online administration. In C. G. Parshall, J. A. Spray, J. C. Kalohn, & T. Davey, Practical considerations in computer based testing (pp.106-125). New York, NY: Springer-Verlag New York, Inc.

Sanders, P. F., & Verschoor, A. J. (1998). Parallel test construction using classical item parameters. Applied Psychological Measurement, 22, 212-223.

Swanson, L., & Stocking, M. L. (1993). A Model and heuristic for solving Very large item selection problems. Applied Psychological Measurement, 17, 151-166.

Author(s)

Gulsah Gurkan (gurkangulsah@gmail.com), Michael Chajewski (mchajewski@hotmail.com)