“In addition, it is fairly reasonable to assume that confronting participants with tricky questions makes them suspicious of and more attentive to the experimental questions. Conditioned by the CRT, they will be more inclined to try to spot a trick in the scenarios. In several studies, this strategy has proven to be successful. Pinillos et al. (2011, p. 126) have shown that conditioning the participants with the CRT prompts more careful responses by demonstrating the possibility that initial gut responses may be wrong.”

Byrd 2025 Figure 3b shows that reflection test scores are actually lower in the condition in which people get a long reflection test *before* they complete other tasks (compared to when they get other tasks before the reflection test).

Another #failedReplication of a reflection test priming effect on the #sideEffectEffect: doi.org/10.1111/mila...

But our data above cast doubt on this paper's expectation that reflection test primes actually cause reflection (we saw *less* reflection in the test-first group).

#cogSci #xPhi #Psych

The intention attribution scenarios (and response options), two self-report reasoning personality scales, and one behavioral test of reflection (belief bias or logical reflection).

More about the eight belief bias (or logical reflection) items and its scoring.

Results: self-report (but not behavioral) reflection measures predicted intention attribution.

Further replication of our #failedReplication of relationships between reflection and intention attribution (doi.org/10.1093/anal...):

Trotti et al. found only self-reported (but not actual) tendencies to think reflectively predicted a #sideEffectEffect: doi.org/10.1007/s103...