The power of experiments: Decision making in a data‐driven world By MichaelLuca, Max H.Bazerman, Cambridge, Massachusetts, USA: The MIT Press, 2021, 232 pp, $19.95, paperback
本书介绍实验方法在商业、政策等领域的应用，通过案例展示如何用实验回答实际问题，适合学生、管理者及政策制定者了解实验的价值与局限。
What should a letter from the national Tax Authority include to maximize tax payments? How can discrimination be minimized in online markets? Is it possible that a brief two-line notification to students' parents can result in a substantial reduction in class absenteeism? A new book — The Power of Experiments — by Michael Luca, the Lee J. Styslinger III Associate Professor of Business Administration at Harvard Business School, and Max H. Bazerman, Jesse Isidor Straus Professor of Business Administration at the Harvard Business School, demonstrate how economic experiments offer us precise answers to these and numerous other questions through rigorous mechanisms. While the primary goal of the book is to popularize the experimental approach, it holds significant value for a diverse audience. It caters to students taking their first steps in behavioral experiments, as well as those seeking to integrate such methods into their regular practices, ranging from small-scale businessmen to policy makers and form educators to CEOs of large corporations. Furthermore, the book's ability to engage readers through the authors' enthusiasm makes it a captivating read that remains both enjoyable and insightful for professionals. The book is divided into three parts. The first part — Breaking Out of the Lab — offers a concise yet vividly illustrated history of the development of behavioral economics and the expansion of experimental methods from the natural to social sciences. Luca & Bazerman embark on this journey by briefly discussing the emergence of experimental methods in science (chapter one). While familiar to those acquainted with the history of experimental rise, the examples chosen by the authors adeptly showcase the evolution of human understanding and advancements in experimental approaches, all skillfully woven into a compelling narrative. One particularly noteworthy example, also highlighted by Luca & Bazerman in their book, is the earliest documented attempt at a policy experiment (ØStbye & Rochon, 1993): the Book of Daniel 1.1-15 (1.1-20) which I like to use in the introductory course to behavioral economics, and — what is even more important — students in the course enjoy it as well! In chapter two, the authors delve into the pioneering attempts by scholars to utilize experiments in psychology, a foundation that economists later built upon. In chapter three, the authors delve into the significance and pertinence of randomized control trials for informing public policy. The second part — Experiments in the Tech Sector — constitutes the core of the book's narrative, delving into the use and combination of various intentions of companies to conduct experiments. These intentions range from incidental ones to decisions forced by public campaigns aiming to shed light on existing problems, as well as fully conscious and intentional endeavors. Concluding on the value of experiments for business, six major points can be considered essential takeaways. First, while reasoning for an action by labeling it as ‘intuitive’ or ‘obvious’ is popular in arguments (even among educated individuals), it does not provide causal evidence. On the other hand, experimental results do. The second lesson emphasizes that a properly thought-out experiment should address at least one of the following issues: (a) testing the strength of a theory, (b) providing a deeper comprehension of the dynamics of the investigated matter and potential trade-offs, (c) evaluating products, services, and policies, and (d) conducting initial explorations when there is no existing theory. While one experiment is better than none, a series of experiments generally yields more fruitful data, a greater extent of evidence, and enhanced robustness, as stated in the third lesson. The fourth lesson touches upon a caveat: although short-term outcomes may be enticing to focus on exclusively (especially due to incentives favoring quicker, cheaper assessments of intervention effects on tomorrow rather than several years later), it is crucial not to neglect long-term results. The fifth lesson highlights that spillover effects are not rare, and neglecting proper planning for an experiment can easily undermine the entire study. Lastly, the sixth lesson underscores that transparency is a virtue, particularly in individual involvement in experimental studies. The third part of the book — Experimenting for the Social Good — shifts the focus to a more academic yet highly socially relevant research perspective. While the previous part mainly presented anecdotal evidence on several cases of successful and not-so-successful experimental interventions in the tech sector, this part delves into a more fundamental approach. Chapter eleven is dedicated to Todd Rogers' research on voting behavior, with an emphasis on the contributions of experiments to this research agenda. Chapters twelve and thirteen discuss recent experimental research findings that have proven effective in addressing issues in various domains: education (e.g., school enrollment, class absenteeism, academic performance), finance (e.g., investment decisions, insurance and retirement plans), and health (e.g., physical exercise, eating habits). Chapter fourteen provides a brief overview of the backlash that the experimental approach has faced, including experiment aversion and moral concerns. The final chapter summarizes five clusters of ideas that are present throughout the book. The first two lessons posit that (i) the shift towards an experimental approach in the social sciences has been occurring for several decades, generating valuable novel data useful for both academic research and effective public policy, and (ii) such a shift is ongoing. The third lesson emphasizes that experiments are not a panacea and can bring negative societal effects, especially if firms prioritize profit maximization without considering broader social consequences. Despite the book's focus on the tech sector, the fourth point highlights that experiments can provide insights in numerous other domains as well. Lastly, the fifth point acknowledges the potential for further research that should and can be pursued. The Power of Experiments has several advantages. First, it aims to popularize the benefits of experimental methods in general and in the business domain in particular. This is valuable both within academia, as it accelerates the dissemination of experimental approaches in fields historically less exposed to these techniques, and for real-world public policy decisions. Secondly, the book is written in a way that makes it easy to read, even for those unfamiliar with experimental methods in social science: it flows smoothly and is digestible. Finally, and appealing to a broader audience, the book is both inspiring and candid about the limitations and potential misuses of experiments from both technical and ethical perspectives. In the context of recent books like Experimentation Works by Thomke (2020), The Voltage Effect by List (2022), and Mixed Signals by Gneezy (2023), The Power of Experiments artfully combines the fascinating achievements of behavioral economics with a captivating narrative. Despite the mentioned positive aspect of The Power of Experiments, there are several minor points of critique. One is related to the first part of the book where the history of behavioral and experimental economics is presented. In many ways, the narrative offers a selective review, which inevitably involves a trade-off between completeness and chapter length, on the one hand, and the reader's investment of time and effort, on the other hand. Nevertheless, the explicit USA-centrism in the history of experimental methods in social sciences — along with the notable scarcity of discussion about contributors to the advancement of behavioral economics from Germany and the Netherlands — is worth mentioning. Another point pertains to chapter fourteen, which discusses the ethics of randomized control trials. It is primarily devoted to the question of whether or not a firm should conduct experiments and does not fully address other significant issues, such as the research ideas from a normative standpoint and practices researchers should aim for, why certain seemingly neutral practices can be considered misconduct, and so forth. However, the focus on this particular aspect could be attributed to the author's expectation of the book's main audience, which is intended to be businessmen and policy-makers. Lastly, I anticipated encountering a discussion of the relatively novel approach of mega-studies, which is gaining popularity. Although still relatively new to the field, this approach has already yielded significant results in promoting physical exercises (Milkman et al., 2021) and encouraging effective vaccination (Milkman et al., 2022). Overall, The Power of Experiments is a great book catering to both newcomers and experts in the field. For the former, it provides a smooth and non-technical introduction to the major concepts of experimental economics, while for the latter, it offers insights into recent developments of experimental methods in the tech sector. Despite the authors' enthusiastic endorsement of experiments as a powerful tool for establishing causality, an important disclaimer runs throughout the book: while experiments are a strong technique for claiming causality where plausible, it's crucial to remain aware of their limitations. Thus, if someone argues a point based on experimental evidence, it's a positive signal, however, we must not overlook the nuances of experimental design and procedures. The devil is in the details.