Document Type

Article

Publication Date

3-14-2023

Abstract

Experimenter demand is a clear threat to the validity of experimental results. To understand the extent of this threat for lab studies, we apply the quantitative frame- work from de Quidt, Haushofer and Roth (2018) to explore whether experimenter demand can generate flawed qualitative inference in experimental studies, using four classic behavioral findings. In these four settings we examine the extent to which demand can alter the nature of a comparative-static conclusion, a stronger test of the potential distortions resulting from experimenter demand. Starting with the laboratory population, we demonstrate that even in a stark environment with deliberate researcher attempts to manipulate participant behavior, quantitative effects are small and experimenter demand effects are not large enough to impact the core qualitative inferences in our four experimental comparisons. This result is then extended to two commonly used online populations, Prolific and mTurk–which show larger quantitative demand effects, but again, not large enough to alter the qualitative conclusions.

Comments

Author’s submitted manuscript. Title updated 8/26/2024.

Included in

Economics Commons

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.