Securely simulate CAT administration of your items pools right in your browser!
WebCATSim is a web-based tool that lets you evaluate item pools, explore model selection and constraints, and evaluate CAT performance all in a single location. This tool is powered by WebCATCore, a web-based CAT engine that supports 1,2,3 PL models, common item selection algorithms, exposure controls, content balancing, and stop criteria. Simulate your CAT across customizable uniform, normal, and lognormal theta distributions. Analyze your simulation through various performance metrics along side intuitive charts and graphs. Even export your simulation data to compile your own analysis.
Features
WebCATSim supports the following features:
CAT Simulation
Simulation
Import custom item pools
Configurable scaling constant d
Item Pool Total Information Curve Chart
True θ Uniform Distribution
True θ Normal Distribution
True θ Lognormal Distribution
Custom True θ List import
Unlimited θ per Simulation
Item Selection Algorithms
Point Fisher Information
a-Stratified (ASTR)
a-Stratified with b-Blocking (BASTR)
a-Stratified with Content Blocking (CASTR)
Closest b-Parameter Proximity
Progressive Method (Revuelta & Ponsoda)
Random Item Selection
CAT Stop Criteria
Standard Error Threshold
Closest b-Parameter Proximity
Progressive Method (Revuelta & Ponsoda)
Minimum Item Information Threshold
∆θ Minimum Threshold
|θ−𝛽| Minimum Threshold
Minimum Number of Items to Administer
Item Exposure Controls
Randomesque (Kingsbury & Zara)
Content Balancing
Constrained CAT (CCAT)
Initial Item Administration
Fixed item exposures
- only available in full version of WebCATSim
CAT Analysis
CAT Performance Metrics
Efficiency
Bias
MSE
RMSE
Pearson Correlation Coef.
STANDARD ERROR ANALYSIS METRICS
Mean SE
Min SE
Max SE
Std. Dev. of SE
% <= 0.30
% <= 0.25
ITEM EXPOSURE AND UTILIZATION
Item ID vs Frequency Chart
Auto-generated summary description
Item Pool Size
Items Exposed
Over-Exposed (exp. > 0.2)
Under-Utilized (exp. < 0.02)
Scaled Chi-Square
Test-Overlap Rate
STOP CRITERIA ANALYSIS
Total Stop Criteria Met by % Chart
Auto-generated summary description
Final Ability Estimate vs % Met Stop Criteria Chart
Total Simulations
Standard Error Criteria
Number of Items Administered Criteria
Minimum Item Information Criteria
∆θ Minimum Threshold
|θ−𝛽| Minimum Threshold
FINAL ABILITY ESTIMATE ANALYSIS
Average Final Ability Estimate vs Simulated θ Chart
Average Simulated θ vs Final Ability Estimate Chart
Average Standard Error vs Final Ability Estimate Chart
Average Test Information vs Final Ability Estimate Chart
Item Administration Analysis
Average Number of Items Administered vs Final Ability Score Chart
Average Number of Items Administered
Content Balancing Analysis
Content Code vs Target Weight and Mean Administration Chart
Content Code Target Value and Mean metrics
Ranged Detail Analysis
Average Standard Error vs Final Ability Estimate within Range Chart
Ranged Details: Mean SE, % <= 0.30, % <= 0.25
- only available in full version of WebCATSim
Want to learn more?
© 2017 Florida State University