Power of Experiments - Decision Making in a Data-Driven World

de Michael Luca, Max H. Bazerman

"Whether or not we know it, we participate in hundreds of experiments a year. Every time you log onto Amazon, scroll through Netflix listings, or read a Facebook post, you are probably part of an experiment designed to help the platform learn more about how users interact with the site. Digital technology facilitates a culture of experimentation, which is why tech companies have been so active in experimentation. However, governments also experiment to learn more about how services can be improved. For example, in the UK, the Behavioral Insights Team sent out ten different versions of a letter to delinquent taxpayers to determine the most effective wording for getting people to actually pay. ("Nine out of ten people in the UK pay their tax on time. You are currently in the very small minority of people who have not paid us yet." proved to be the most effective.) This book explores the evolving role of experiments in corporate and government decision making. It recognizes that an experiment is simply a tool. An experiment can make us better or worse off, depending on who is running it, what their goals are, and what they mean when they say the experiment "worked." The authors of this book, both experts in behavioral science from Harvard Business School, are enthusiastic about the promise of experimental methods, while keenly aware of the potential pitfalls. Their goal is to demystify experiments, and help readers gain an appreciation of the value of experiments-whether in their own organizations, or as part of the world in which we live. Middle section of book discusses specific experiments that various tech companies have undertaken. These include experiments to reduce racial discrimination on Airbnb (and the somewhat unsatisfying decisions Airbnb made when confronted with evidence of racial discrimination on the platform), an experiment eBay ran to determine the best use of its ad dollars on Google (short version: eBay didn't need to buy an ad targeting people who searched for the word "eBay" on Google-the native search results led to an equivalent number of clicks as the paid ad), an experiment involving discounts on Alibaba, StubHub's decision to include hidden fees after previously advertising "No Hidden Fees," experiments that Uber did to test the idea of Express Pool, and the controversy surrounding an experiment that Facebook did where they exposed users to content that was either skewed more positive or more negative, and then tracked users' stated moods. By reading about prior experiments run within organizations, managers and other decision makers will learn to approach problems in their organizations with an experimental mindset. The final section of the book looks at the ways experiments can test policy interventions designed to improve educational, health, and financial outcomes"--