Applying Bayesian checks of cancellation axioms for interval scaling in limited samples - Behavior Research Methods Interval scales are frequently assumed in educational and psychological research involving latent variables, but are rarely verified. This paper outlines methods for investigating the interval scale assumption when fitting the Rasch model to item response data. We study a Bayesian method for evaluating an item response dataset’s adherence to the cancellation axioms of additive conjoint measurement under the Rasch model, and compare the extent to which the axiom of double cancellation holds in the data at sample sizes of 250 and 1000 with varying test lengths, difficulty spreads, and levels of adherence to the Rasch model in the data-generating process. Because the statistic produced by the procedure is not directly interpretable as an indicator of whether an interval scale can be established, we develop and evaluate procedures for bootstrapping a null distribution of violation rates against which to compare results. At a sample size of 250, the method under investigation is not well powered to detect the violations of interval scaling that we simulate, but the procedure works quite consistently at N = 1000. That is, at moderate but achievable sample sizes, empirical tests for interval scaling are indeed possible.

Out today in BRM!

We investigate the small(er) sample performance of an MCMC method for checking whether item response data produce an interval scale using the Rasch model. These checks are viable at achievable sample sizes in survey research.

Open access: link.springer.com/article/10.3...

Wyatt Read presented this work today at the UD Steele Symposium!

In anticipation of IOMW 2025, we’ve posted a preprint of the paper we’ll be discussing there. Take a look! osf.io/preprints/ps...

Ever wanted to investigate DIF by more than one variable at once? My colleague Ethan McCormick and I have created an NCME ITEMS module on DIF analysis with multiple background variables using moderated nonlinear factor analysis. Take a look! www.ncme.org/itemsportal/...

Growth across Grades and Common Item Grade Alignment in Vertical Scaling Using the Rasch Model Vertical scales are frequently developed using common item nonequivalent group linking. In this design, one can use upper-grade, lower-grade, or mixed-grade common items to estimate the linking const...

onlinelibrary.wiley.com/doi/full/10....

Out now! @derekbriggs.bsky.social, Laurie Davis and I investigate differences in how much students appear to grow on a vertical scale based solely on the grade alignment of (a large number of) common items. Turns out, the differences are pretty big!

Growth on 2019 State Achievement Tests: Empirical Benchmarks and the Role of Scale Choice Benchmarks for “years of learning”–expectations for annual academic growth in standard deviation units–are popular to contextualize the practical significance of a causal effect. Though they are ap...

It’s new article day! Follow the link for updated benchmarks for grade 3-8 annual growth and an investigation of what drives the considerable variability in growth across different scales. doi.org/10.1080/1934...

Fun find for the ncme archives committee today!

Come be my colleague! We are hiring for a TT assistant professor in education statistics and data science. Please get in touch with any questions! careers.udel.edu/cw/en-us/job...

Not sure there would always be a hypothesis about the rank order of the loadings, but I’d hope that the work that goes into writing good items creates the other ones implicitly

I’m teaching intro EFA next week and this is essentially the perspective I’m teaching from: we’ve spent most of the semester on writing good questions aligned to a theory about the construct of interest, and that doesn’t suddenly disappear when we collect responses and stick them into an EFA

Wasn’t really sure how to start posting here but it turns out the answer was a boring (well, not to me) methods question: anyone ever fit the 2PL IRT model as a nonlinear mixed effects model using the lme4 R package?