Transformers in Action

Understand the architecture that underpins todays most powerful AI models.Transformers are the superpower behind large language models (LLMs) like ChatGPT, Gemini, and Claude. Transformers in Action gives you the insights, practical techniques, and extensive code samples you need to adapt pretrained transformer models to new and exciting tasks.Inside Transformers in Action youll learn:How transformers and LLMs workModeling families and architecture variantsEfficient and specialized large language modelsAdapt HuggingFace models to new tasksAutomate hyperparameter search with Ray Tune and OptunaOptimize LLM model performanceAdvanced prompting and zero/few-shot learningText generation with reinforcement learningResponsible LLMsTransformers in Action takes you from the origins of transformers all the way to fine-tuning an LLM for your own projects. Author Nicole Koenigstein demonstrates the vital mathematical and theoretical background of the transformer architecture practically through executable Jupyter notebooks. Youll discover advice on prompt engineering, as well as proven-and-tested methods for optimizing and tuning large language models. Plus, youll find unique coverage of AI ethics, specialized smaller models, and the decoder encoder architecture.Foreword by Luis Serrano.About the technologyTransformers are the beating heart of large language models (LLMs) and other generative AI tools. These powerful neural networks use a mechanism called self-attention, which enables them to dynamically evaluate the relevance of each input element in context. Transformer-based models can understand and generate natural language, translate between languages, summarize text, and even write codeall with impressive fluency and coherence.About the bookTransformers in Action introduces you to transformers and large language models with careful attention to their design and mathematical underpinnings. Youll learn why architecture matters for speed, scale, and retrieval as you explore applications including RAG and multi-modal models. Along the way, youll discover how to optimize training and performance using advanced sampling and decoding techniques, use reinforcement learning to align models with human preferences, and more. The hands-on Jupyter notebooks and real-world examples ensure youll see transformers in action as you go.What's insideOptimizing LLM model performanceAdapting HuggingFace models to new tasksHow transformers and LLMs work under the hoodMitigating bias and responsible ethics in LLMsAbout the readerFor data scientists and machine learning engineers.About the authorNicole Koenigstein is the Co-Founder and Chief AI Officer at the fintech company Quantmate.Table of ContentsPart 11 The need for transformers2 A deeper look into transformersPart 23 Model families and architecture variants4 Text generation strategies and prompting techniques5 Preference alignment and retrieval-augmented generationPart 36 Multimodal models7 Efficient and specialized small language models8 Training and evaluating large language models9 Optimizing and scaling large language models10 Ethical and responsible large language models