This book gives practitioners a concrete, systematic framework for designing evals that make AI systems safe, robust, and customer-ready before they reach production. Drawing on real-world failures, from chatbots that went off the rails to shopping assistants that hallucinated product information, it shows how seemingly small evaluation gaps can cascade into legal, financial, and reputational crisis, and how to close those gaps with disciplined, systematic testing.Moving from foundational concepts to advanced practice, Reliable Evals for LLMs and AI Agents introduces the four core levers of effective evals: sets, templates, metrics, and evaluators. It then extends these to the unique challenges of autonomous AI agents, where systems perceive, reason, act, and adapt in iterative loops that demand fundamentally different eval approaches. Along the way, it guides readers through benchmark selection, custom eval set design, statistical rigor in metrics, human and LLM-as-a-judge rating strategies, and the infrastructure needed to automate evals at scale.For engineering leaders, applied researchers, data scientists, and product teams shipping LLM- and agent-powered experiences, this volume offers a blueprint for building eval flywheels that continuously improve AI quality. It shows how to progress from ad-hoc checks to production-grade eval systems, align model metrics with real user satisfaction, integrate offline evals with online A/B testing, and design accessible interfaces that democratize rigorous testing across an organization.

This book gives practitioners a concrete, systematic framework for designing evals that make AI systems safe, robust, and customer-ready before they reach production. Drawing on real-world failures, from chatbots that went off the rails to shopping assistants that hallucinated product information, it shows how seemingly small evaluation gaps can cascade into legal, financial, and reputational crisis, and how to close those gaps with disciplined, systematic testing.Moving from foundational concepts to advanced practice, Reliable Evals for LLMs and AI Agents introduces the four core levers of effective evals: sets, templates, metrics, and evaluators. It then extends these to the unique challenges of autonomous AI agents, where systems perceive, reason, act, and adapt in iterative loops that demand fundamentally different eval approaches. Along the way, it guides readers through benchmark selection, custom eval set design, statistical rigor in metrics, human and LLM-as-a-judge rating strategies, and the infrastructure needed to automate evals at scale.For engineering leaders, applied researchers, data scientists, and product teams shipping LLM- and agent-powered experiences, this volume offers a blueprint for building eval flywheels that continuously improve AI quality. It shows how to progress from ad-hoc checks to production-grade eval systems, align model metrics with real user satisfaction, integrate offline evals with online A/B testing, and design accessible interfaces that democratize rigorous testing across an organization.

This book provides a conceptual overview of the evolution of Chinese philosophy from its earliest beginnings to the end of the imperial era, highlighting 38 of the most essential terms in the Chinese philosophical tradition. Written by prominent contemporary scholars from Mainland China, the respective chapters cover topics ranging from cosmology, benti metaphysics, human nature, self-cultivation, and methodology, to views on history and politics. Each chapter addresses one of the constitutive terms of the Chinese philosophical tradition and provides clear historical information on how it was used and developed during the key periods of Chinese philosophy.Highlighting both central concepts and essential structures of Chinese philosophy, the book allows readers to view the history of Chinese philosophy from the perspective of the Chinese themselves. Offering content that is both academically rigorous and accessible for a wider audience, this book is an indispensable reference guide for all students of Chinese philosophy.

This book provides a conceptual overview of the evolution of Chinese philosophy from its earliest beginnings to the end of the imperial era, highlighting 38 of the most essential terms in the Chinese philosophical tradition. Written by prominent contemporary scholars from Mainland China, the respective chapters cover topics ranging from cosmology, benti metaphysics, human nature, self-cultivation, and methodology, to views on history and politics. Each chapter addresses one of the constitutive terms of the Chinese philosophical tradition and provides clear historical information on how it was used and developed during the key periods of Chinese philosophy.Highlighting both central concepts and essential structures of Chinese philosophy, the book allows readers to view the history of Chinese philosophy from the perspective of the Chinese themselves. Offering content that is both academically rigorous and accessible for a wider audience, this book is an indispensable reference guide for all students of Chinese philosophy.

This book gives practitioners a concrete, systematic framework for designing evals that make AI systems safe, robust, and customer-ready before they reach production. Drawing on real-world failures, from chatbots that went off the rails to shopping assistants that hallucinated product information, it shows how seemingly small evaluation gaps can cascade into legal, financial, and reputational crisis, and how to close those gaps with disciplined, systematic testing.Moving from foundational concepts to advanced practice, Reliable Evals for LLMs and AI Agents introduces the four core levers of effective evals: sets, templates, metrics, and evaluators. It then extends these to the unique challenges of autonomous AI agents, where systems perceive, reason, act, and adapt in iterative loops that demand fundamentally different eval approaches. Along the way, it guides readers through benchmark selection, custom eval set design, statistical rigor in metrics, human and LLM-as-a-judge rating strategies, and the infrastructure needed to automate evals at scale.For engineering leaders, applied researchers, data scientists, and product teams shipping LLM- and agent-powered experiences, this volume offers a blueprint for building eval flywheels that continuously improve AI quality. It shows how to progress from ad-hoc checks to production-grade eval systems, align model metrics with real user satisfaction, integrate offline evals with online A/B testing, and design accessible interfaces that democratize rigorous testing across an organization.

This book gives practitioners a concrete, systematic framework for designing evals that make AI systems safe, robust, and customer-ready before they reach production. Drawing on real-world failures, from chatbots that went off the rails to shopping assistants that hallucinated product information, it shows how seemingly small evaluation gaps can cascade into legal, financial, and reputational crisis, and how to close those gaps with disciplined, systematic testing.Moving from foundational concepts to advanced practice, Reliable Evals for LLMs and AI Agents introduces the four core levers of effective evals: sets, templates, metrics, and evaluators. It then extends these to the unique challenges of autonomous AI agents, where systems perceive, reason, act, and adapt in iterative loops that demand fundamentally different eval approaches. Along the way, it guides readers through benchmark selection, custom eval set design, statistical rigor in metrics, human and LLM-as-a-judge rating strategies, and the infrastructure needed to automate evals at scale.For engineering leaders, applied researchers, data scientists, and product teams shipping LLM- and agent-powered experiences, this volume offers a blueprint for building eval flywheels that continuously improve AI quality. It shows how to progress from ad-hoc checks to production-grade eval systems, align model metrics with real user satisfaction, integrate offline evals with online A/B testing, and design accessible interfaces that democratize rigorous testing across an organization.

This book provides a conceptual overview of the evolution of Chinese philosophy from its earliest beginnings to the end of the imperial era, highlighting 38 of the most essential terms in the Chinese philosophical tradition. Written by prominent contemporary scholars from Mainland China, the respective chapters cover topics ranging from cosmology, benti metaphysics, human nature, self-cultivation, and methodology, to views on history and politics. Each chapter addresses one of the constitutive terms of the Chinese philosophical tradition and provides clear historical information on how it was used and developed during the key periods of Chinese philosophy.Highlighting both central concepts and essential structures of Chinese philosophy, the book allows readers to view the history of Chinese philosophy from the perspective of the Chinese themselves. Offering content that is both academically rigorous and accessible for a wider audience, this book is an indispensable reference guide for all students of Chinese philosophy.

This book provides a conceptual overview of the evolution of Chinese philosophy from its earliest beginnings to the end of the imperial era, highlighting 38 of the most essential terms in the Chinese philosophical tradition. Written by prominent contemporary scholars from Mainland China, the respective chapters cover topics ranging from cosmology, benti metaphysics, human nature, self-cultivation, and methodology, to views on history and politics. Each chapter addresses one of the constitutive terms of the Chinese philosophical tradition and provides clear historical information on how it was used and developed during the key periods of Chinese philosophy.Highlighting both central concepts and essential structures of Chinese philosophy, the book allows readers to view the history of Chinese philosophy from the perspective of the Chinese themselves. Offering content that is both academically rigorous and accessible for a wider audience, this book is an indispensable reference guide for all students of Chinese philosophy.