We are dumbing down the field because Psych Departments need as many butts in seats as possible (at public universities) and because we get evaluated by citations. A technical pub in Psychometrika is understood by <1% of psychologists; while p-hacked nonsense in Psych Science is "understood" widely.
In France, if you want to build a home above a certain size, you’re legally required to use a licensed architect.
Can you guess what that size is
Change 1: Making it easier to choose “no religion” As the share of religious “nones” grows, censuses that measure religion are increasingly making “no religion” the first response category in religious identity questions rather than the last. Some respondents may choose this option simply because it appears first, a primacy effect. In 2011, “no religion” was the last religion response category in Australia. In 2016, Australia made “no religion” the first option and the religiously unaffiliated share rose 9 percentage points from 24% to 33%. Reporting on Australia’s census results, CNN noted that the “nones” outnumbered Catholics for the first time. However, the report didn’t mention that there was a change in response categories that may have contributed to this shift (Berlinger 2017). During this period, the rise of religious “nones” was lower—about three points—on International Social Survey Programme surveys that kept their measure constant. Other questionnaire design decisions also have made it easiest to choose “no religion.” In 1991, Canada’s census introduced a write-in box for respondents to spell out their religious identity. However, identifying with “no religion” only required filling in a bubble. New Zealand and Poland introduced a similar change in recent censuses. The relative ease of choosing “no religion” may contribute, at least in part, to the rise of the “nones” in these countries.
Change 2: Shifting between one-step and two-step measures of religious identity One-step measures of religious identity ask a question like, “What is your religion, if any?” and typically offer “no religion” as a response category. By contrast, a two-step question first asks a filter question like “Do you have a religion?” and if respondents say they do, they are invited to specify it. Outside Muslim-majority countries, these two types of questions tend to produce different results (Brenner et al. 2024; Hackett 2014; Voas and Bruce 2004; Voas 2015). The share of religious “nones” is higher in surveys that use a two-step measure. People with low levels of religious commitment who might volunteer a religious identity in a one-step question tend to fall into the “no religion” bucket in a two-step question. Slovakia changed from a one-step to a two-step question between 2011 and 2021. In 2021, the unaffiliated share of the population was 25%, up from 15% in the 2011 census. It is possible, however, that much of the apparent change between census waves in Slovakia may have been a measurement artifact. Interestingly, the European Social Survey in 2012 used a two-step question and found an identical unaffiliated share, 25%, as the 2021 Slovakia census, which used a two-step question. The opposite change could create an illusion of decline in the share of religious nones. For example, as Lithuania switched from a two-step question in 2001 to a one-step question in 2011, the unaffiliated share of the population dropped from 10% to 7%. These types of changes are not limited to censuses. We observed that the unaffiliated share in Sweden dropped about 10 points between the 2010 ISSP, which used a two-step measure, and the 2011 wave, which used a one-step measure.
Change 3: Changes in survey mode Changes in survey mode may affect social desirability and coverage biases. When monthly surveys carried out in Spain by the Center for Sociological Research changed from in-person to phone interviews in April 2020, there was an immediate jump of 5 percentage points in the share of respondents who said they were atheist, agnostic or indifferent toward religion (González and Cabrera 2023). Many censuses and surveys have transitioned from in-person interviews to self-administered questionnaires. For example, in Hungary, the option to complete the census online or by mail in 2011 may have contributed significantly to the 9-point rise in the unaffiliated share of the population from the 2001 face-to-face census. By contrast, with the same options for completing the census in place in 2021, the unaffiliated share rose only 2 additional points. A change from primarily face-to-face interviews to a mail-to-web format saw the “no religion” share of U.S. General Social Survey respondents rise 5 points from 2018 to 2021. Some of this increase may have been the result of more religious Americans, including older adults, being less willing to take the survey online (Schnabel et al. 2024).
The need to test the impact of measurement changes When media coverage overlooks methodological issues, it may mislead the public (Hackett 2013; 2023; Hackett and Tong 2025; Hackett 2026). Reporters may emphasize what appears to be a large change while overlooking, omitting or being unaware of measurement artifacts that exaggerate the change (Berlinger 2017). Survey and census organizations should research how changes in the way religion is measured affect results. Without such study, it is difficult to distinguish real social change from methodological artifacts. For all who seek to understand religious change, including journalists, religious leaders, policymakers, researchers and the public, it’s vital to disentangle the two (Hackett 2020). Organizations should publish the results of experiments to measure the impact of measurement change. Approaches may include: Experiments with split-samples: When changing question format, respondents can be randomly assigned to the new and old conditions. Differences between the groups can be used to quantify the measurement effect. Experiments with two modes: When survey modes are changing, a mode experiment may use the old and new mode of data collection (Pew Research Center 2021). External data comparisons: While it would be ideal for organizations to conduct their own experiments, they may also evaluate the extent to which religious change has occurred on other high-quality surveys that have maintained consistent methodology during the period of interest. Organizations should thoroughly describe their methodological changes and draw attention to how changes may affect trend data (Sullivan et al. 2012). Researchers, journalists, and the public need help understanding whether apparent change is primarily the result of measurement change.
NEW: We highlight 3 changes in measurement of religious “nones” & call for organizations that make these changes to study their effects.
From Matthew Conrad & me. www.surveypractice.org/article/1595...
Bessent is the higher-order factor explaining the covariation between stupidity, evil, and insanity.
I thought that the goal was to HARK our way to some mildly counterintuitive finding that we can sell to Psych Science or AMJ so that we can get on that sweet sweet b-school & TED Talk gravy train.
Anytime. Chicago is drivable from Ames.
I can't really afford to go to conferences (unless I save up for a few years) and have probably pissed off too many people in my field to be invited for play dates ;-)
Correction - one of two. I forgot about a talk I gave at a conference at few years ago.
It is still the only in-person talk I have ever given anywhere (conference or otherwise) in over 20 years in academia. Thanks for the invite.
This is why Bluesky needs a "I'm horrified but this is important" alternative to "liking/loving" a post.
Told my wife to have dinner ready at 6 or I'll obliterate her entire civilization. So anyway, she now charges me a fee to use the bathroom that used to be free, and I didn't get any dinner, but I'm pretty sure I won that exchange.
I lol'd at this.😆
An exasperated Jason Collins bangs the drum he’s been beating for 10 years over a pure zombie study.
The original research suggesting a little disfluency in communication stimulates more analytical reasoning is a spectacular replication failure:
buff.ly/VxIUvYr
I Work Very Hard, And I Would Like To Try Cake By A Horse Hello. I am a horse. I work very hard at my job of being a horse. When humans say move the heavy thing, I move the heavy thing. When humans sit on top of me and pull on my head, I carry them where they want to go. The main food the humans give me is hay and oats. But I am thinking it would be nice to have a different food. I am thinking I would like to try cake. Yes, yes. Cake. I know all about it. When humans eat cake, it is in glad times. It is the food for a celebration, such as when a woman becomes 47. I have seen cake on the Fourth of July. When humans have a cake, they stand around it and clap hands and smile and say happy birthday at each other. Sometimes there are beautiful markings on a cake, such as balloons or a pink shape. Sometimes the top of a cake is on fire and a boy must blow on the fire with mouth wind. This is the scariest cake. I do not want this kind. But I will eat any other cake. Any cake that is not the fire cake that tries to kill the boy. Please understand: I do not get money for doing work. I do not get to go inside the house. All I am either doing my horse job or standing in my pen or eating food off the floor. I always do these things. But I have never once gotten cake and I would like it very much. I have noticed that human children get to eat cake. But I am bigger than the children. I am more helpful to the farm. Children do not move the heavy things like me or let anyone ride on them. And yet they get cake. Maybe the humans will realize this. Maybe they will say, "You know who deserves cake? That horse. That horse whose back we are always on." Every day I dream about what it will be like if I get to eat cake. Here is what will happen. First, I will walk to the cake and putt my nose at it like hrrfff to make and stomping my hooves to make sure it is not a snake. Then I will trot in a circle to show that I am a horse and I am large. After that, I will nuzzle the cake to …
The horse op-ed is an instant classic. I can't tell you how much joy this piece gives me.
It should be taught in every introductory writing class in no small part because the horse arguments are so compelling. "I have noticed that human children get to eat cake. But I am bigger than the children."
Brent, you seem mad. Have you tried some mindfulness exercises - or even a simple power pose?
📄Published Today in Nature:
500 researchers reproduced 100 studies across the social & behavioral sciences to assess their analytical robustness (led by @balazsaczel.bsky.social & @szaszibarnabas.bsky.social).
Article: www.nature.com/articles/s41...
Preprint: osf.io/preprints/me...
TLDR: 1/11
I'd rather poke my eyes out with a flaming poker than cite some people in my discipline.
Does not sound like a loss of power. Sounds like the direction of the effect is reversed at the subgroup level. This can happen with observational data but I'm not sure how it could happen with experimental data unless there is a massive failure of random assignment.
.... 2) figure out how the main effect is canceled out (i.e., via some other causal path). All of these causal inference issues are so well known and it is so sad to see this stuff repeated in JAP. Of course, JAP has a long history of publishing mediation nonsense like this.
I dig the field experiment but not the inference drawn from it. If you claim that there is an effect "through" anxiety when there is no main effect then you need to do two things: 1) actually manipulate anxiety in a follow-up to show that it has an effect on the DV, and ...
I completely buy the central message but the demand characteristics and causal inferences from observational data in that paper give me heartburn.
2. 1 December 2025: Editor decides to issue two corrections. No explanation about how these impossible results made their way into the papers.
3. 1 March 2026: I ask when corrections will be published.
25 March 2026: I am told that corrections will be published in May or June.
Science in action!!
Timeline of a correction:
1. 8 April 2025: I contact an APA journal editor to inform them of serious statistical impossibilities in two papers by the same authors. Impossibilities that can be verified on the back of a napkin in 2 minutes.
I am very doubtful that the Trump, Musk, Zuckerberg etc. collective have ever uttered a sincere prayer to any deity in their lives.
Absolutely. One or two pubs in the "right" journals (i.e., those that determine b-school rankings) can result in what is effectively a multi-million dollar payoff. The incentives to cheat are immense and explain the almost complete absence of open science practices in many business journals.
The favorite part of my job as a Psychology Professor at my institution is knowing that I earn slightly less than half the salary of an Assistant Professor in the Business School with 3 publications.
I've apparently reached the age where I can incur fairly painful back injuries by *checks notes* watching my daughter compete at a Taekwondo tournament.
I believe that it is one form of collider bias.
You can correct for range restriction on HSGPA and SAT. Especially doable for SAT where the standard deviation of all test takers is known. Another problem is that these analyses really need to take place at the level of the major because of big mean differences in SAT, HSGPA, and GPA across majors.
