Pavel Atanasov, J. Witkowski, Barbara Mellers, Philip Tetlock (Under Review), The person-situation debate revisited: Forecasting skill matters more than elicitation method.
Philip Tetlock, Lu Yunzi, Barbara Mellers (2022), False Dichotomy Alert: Improving Subjective-Probability Estimates vs. Raising Awareness of Systemic Risk, International Journal of Forecasting.
E. Karger, J.T Monrad, Barbara Mellers, Philip Tetlock (Under Review), Reciprocal scoring: A method for forecasting unanswerable questions.
Katherine L. Milkman, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Predrag Pandiloski, Yeji Park, Aneesh Rai, Max Bazerman, John Beshears, Lauri Bonacorsi, Colin Camerer, Edward Chang, Gretchen B. Chapman, Robert Cialdini, Hengchen Dai, Lauren Eskreis-Winkler, Ayelet Fishbach, James J. Gross, Samantha Horn, Alexa Hubbard, Steven J. Jones, Dean Karlan, Tim Kautz, Erika Kirgios, Joowon Klusowski, Ariella Kristal, Rahul Ladhania, George Loewenstein, Jens Ludwig, Barbara Mellers, Sendhil Mullainathan, Silvia Saccardo, Jann Spiess, Gaurav Suri, Joachim H. Talloen, Jamie Taxer, Yaacov Trope, Lyle Ungar, Kevin Volpp, Ashley Whillans, Jonathan Zinman, Angela Duckworth (2021), Megastudies Improve the Impact of Applied Behavioural Science,, 600, pp. 478-483.
Abstract: Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.
Ike Silver, Barbara Mellers, Philip Tetlock (2021), Wise teamwork: Collective confidence calibration predicts the effectiveness of group discussion, Journal of Experimental Social Psychology .
Siyuan Yin, H. Arkes, John McCoy, Morris A. Cohen, Barbara Mellers (2021), Conflicting Goals Influence Physicians’ Expressed Beliefs to Patients and Colleagues, Medical Decision Making.
Ville Satopää, Marat Salikhov, Philip Tetlock, Barbara Mellers (2021), Decomposing the Effects of Crowd-Wisdom Aggregators: The Bias-Information-Noise (BIN) Model, International Journal of Forecasting.
Christopher Karvetski, Carolyn Meinel, Daniel Maxwell, Lu Yunzi, Barbara Mellers, Philip Tetlock (2021), Forecasting the Accuracy of Forecasters from Properties of Forecasting Rationales, International Journal of Forecasting.
Barbara Mellers, Siyuan Yin, Jonathan Z. Berman (2021), Reconciling Loss Aversion and Gain Seeking in Judged Emotions, Current Directions.
Ville Satopää, Marat Salikhov, Barbara Mellers, Philip Tetlock (2021), Bias, Information, Noise: The BIN Model of Forecasting, Management Science.
