Document Type
Article
Publication Date
3-14-2023
Abstract
Experimenter demand is a clear threat to the validity of experimental results. To understand the extent of this threat for lab studies, we apply the quantitative frame- work from de Quidt, Haushofer and Roth (2018) to explore whether experimenter demand can generate flawed qualitative inference in experimental studies, using four classic behavioral findings. In these four settings we examine the extent to which demand can alter the nature of a comparative-static conclusion, a stronger test of the potential distortions resulting from experimenter demand. Starting with the laboratory population, we demonstrate that even in a stark environment with deliberate researcher attempts to manipulate participant behavior, quantitative effects are small and experimenter demand effects are not large enough to impact the core qualitative inferences in our four experimental comparisons. This result is then extended to two commonly used online populations, Prolific and mTurk–which show larger quantitative demand effects, but again, not large enough to alter the qualitative conclusions.
Recommended Citation
Danz, David; Lepper, Marissa; Lezama, Guillermo; Mustafi, Priyoma; Verterlund, Lise; Wilson, Alistair; and Winichakul, K. Pun, "Simon Doesn’t Say: Minimal Qualitative Distortions from Experimenter Demand" (2023). Economics: Faculty Publications, Smith College, Northampton, MA.
https://scholarworks.smith.edu/eco_facpubs/102
Comments
Author’s submitted manuscript.