Part of the goal of this site is to help build a more shared vocabulary across the social, behavioral, and economic sciences. There is a great deal of valuable methods knowledge that still does not travel well across disciplinary boundaries.
My own thinking on this was shaped by training across economics, psychology, psychometrics, Bayesian statistics, and experimental research. One thing that became clear over time is how often adjacent fields ask similar questions while using very different language.
In many fields, measurement, experimental design, hypothesis testing, and causal inference are treated as separate topics. But in actual research, they are part of the same inferential chain. Weakness in one part can easily undermine the rest.
A core theme is that strong causal claims depend on more than design alone. Before we can make meaningful comparisons, we need to know what we are measuring, how well we are measuring it, and whether those measurements support the claims we want to make.
I’ve launched a website on measurement, experimentation, and causal inference:
danrschley.github.io/Measurement-...
I built it to share methods ideas that are often taught separately, but are deeply connected in practice.
My wonderful coauthor Andreas Alfons and I wrote this paper for both statisticians and behavioral scientists. Statisticians get a clear entry point into mediation models; empirical scholars get an accessible guide to the statistical issues that arise when real data violate standard assumptions.
Our simulation study tests many realistic data situations. The results are consistent: OLS-based mediation works well only under ideal conditions. Robust methods stay stable across a much broader range of distributions researchers face in their data.
We review standard mediation methods and show where they struggle—skewness, heavy tails, outliers, rounding, censoring. These issues affect many common tools, including widely used OLS-based approaches like PROCESS.
Bootstrapped CIs only address the distribution of the a*b indirect effect. They do not fix problems in a or b caused by nonnormality or outliers. If those paths are distorted, the indirect effect can be too, no matter how many bootstrap samples you use.
Most mediation models rely on OLS, which is sensitive to skewed data and outliers. Our paper introduces readers to robust statistical tools—like MM-estimation and median regression—and points to R packages that you can use today.
Even if the causal assumptions for mediation hold, statistical assumptions can fail. Features like skewed distributions or outliers can distort standard estimates. Our new paper details which data issues cause trouble and reviews methods built to handle them: dx.doi.org/10.1002/wics...
Yes this! I've been teaching this example in my methods course for years. For instance, there are lots of findings like this showing people underestimate wealth inequality. No, these studies show that people underestimate big things and overestimate small things. www.google.com/url?sa=t&sou...