Leemay Nassery – författare

Before you trust critical business systems to an AI model, you need to answer a few questions. Will it be fast enough? Will the system satisfy user expectations? Is it safe and can you trust the output? This book will help you answer these questions before you roll out an AI system, and make sure it runs smoothly after you deploy. 

AI Model Evaluation teaches you how to effectively evaluate and assess machine learning models for better scaling and integration. Each chapter looks at a different way to test a model, starting with offline evaluations and moving into live A/B tests, shadow traffic deployments and LLM-based feedback loops. The book uses a hands-on example grounded in a movie recommendation engine. 

After reading this book, you will be able to evaluate both model behaviour and engineering system performance. You will have the tools to ensure your AI systems are effective and reliable in production. This book is for practitioners with experience in machine learning, data science, or software engineering. 

Whether you''re a catalyst for organizational change or have the support you need to create an engineering culture that embraces A/B testing, this book will help you do it right. The step-by-step instructions will demystify the entire process, from constructing an A/B test to breaking down the decision factors to build an engineering platform. When you''re ready to run the A/B test of your dreams, you''ll have the perfect blueprint.

With smart, tactful approaches to orchestrating A/B testing on a product, you''ll quickly discover how to reap all the benefits that A/B testing has to offer - benefits that span your users, your product, and your team. Take the reins today, and be the change you want to see in your engineering and product organizations.

Develop a hypothesis statement that''s backed with metrics that demonstrate if your prediction for the experiment is correct. Build more inclusive products by leveraging audience segmentation strategies and ad-hoc post analysis to better understand the impact of changes on specific user groups. Determine which path is best for your team when deciding whether to go with a third-party A/B test framework or to build the A/B testing platform in-house. And finally, learn how to cultivate an experimentation-friendly culture within your team.

Leverage the A/B testing methodology to demonstrate the impact of changes on a product to your users, your key business metrics, and the way your team works together. After all, if you aren''t measuring the impact of the changes you make, how will you know if you''re truly making improvements?

Whether you''re a catalyst for organizational change or have the support you need to create an engineering culture that embraces A/B testing, this book will help you do it right. The step-by-step instructions will demystify the entire process, from constructing an A/B test to breaking down the decision factors to build an engineering platform. When you''re ready to run the A/B test of your dreams, you''ll have the perfect blueprint.

With smart, tactful approaches to orchestrating A/B testing on a product, you''ll quickly discover how to reap all the benefits that A/B testing has to offer - benefits that span your users, your product, and your team. Take the reins today, and be the change you want to see in your engineering and product organizations.

Develop a hypothesis statement that''s backed with metrics that demonstrate if your prediction for the experiment is correct. Build more inclusive products by leveraging audience segmentation strategies and ad-hoc post analysis to better understand the impact of changes on specific user groups. Determine which path is best for your team when deciding whether to go with a third-party A/B test framework or to build the A/B testing platform in-house. And finally, learn how to cultivate an experimentation-friendly culture within your team.

Leverage the A/B testing methodology to demonstrate the impact of changes on a product to your users, your key business metrics, and the way your team works together. After all, if you aren''t measuring the impact of the changes you make, how will you know if you''re truly making improvements?