Trustworthy Online Controlled Experiments : A Practical Guide to a/B Testing by Diane Tang, Ron Kohavi and Ya Xu (2020, Trade Paperback)

Cambridge University Press

1108724264

9781108724265

2309818127

Diane Tang, Ya Xu, Ron Kohavi

Trustworthy Online Controlled Experiments : a Practical Guide to A/B Testing

Trade Paperback

English

Web / Social Media, Databases / Data Mining, Web / User-Generated Content

2020

Textbook

Computers

288 Pages

8.9 in

0.6 in

6 in

14.1 Oz

2019-042021

Scholarly & Professional

Hm741.K68 2020

'This book is a great resource for executives, leaders, researchers or engineers looking to use online controlled experiments to optimize product features, project efficiency or revenue. I know firsthand the impact that Ronny's work had on Bing and Microsoft, and I'm excited that these learnings can now reach a wider audience.' Harry Shum, Executive Vice President, Microsoft Artificial Intelligence and Research Group

Preface - how to read this book; 1. Introduction and motivation; 2. Running and analyzing experiments: an end-to-end example; 3. Twyman's law and experimentation trustworthiness; 4. Experimentation platform and culture; Part II: 5. Speed matters: an end-to-end case study; 6. Organizational metrics; 7. Metrics for experimentation and the Overall Evaluation Criterion (OEC); 8. Institutional memory and aeta-analysis; 9. Ethics in controlled experiments; Part III: 10. Complementary techniques; 11. Observational causal studies; Part IV: 12. Client-side experiments; 13. Instrumentation; 14. Choosing a randomization unit; 15. Ramping experiment exposure: trading off speed, quality, and risk; 16. Scaling experiment analyses; Part V: 17. The statistics behind online controlled experiments; 18. Variance estimation and improved sensitivity: pitfalls and solutions; 19. The A/A test; 20. Triggering for improved sensitivity; 21. Guardrail metrics; 22. Leakage and interference between variants; 23. Measuring long-term treatment effects.

2020

302.231

23

Yes