GenAI on AWS

OLIVIER BERGERET is a technical leader at Amazon Web Services (AWS), working on database and analytics services. He has over 25 years of experience in data engineering and analytics. Since joining AWS in 2015, he’s supported the launch of most of AWS AI services including Amazon SageMaker and AWS DeepRacer. He is a regular speaker and presenter at various data, AI and cloud events such as AWS re:Invent, AWS Summits and third-party conferences. ASIF ABBASI is a Principal Solutions Architect at AWS and has spent the last 20 years working in various roles with focus around Data Analytics, AI/ML, DWH Strategic and Technical Implementations, J2EE Enterprise applications design/development and Project Management. Asif is an Amazon Certified SA, Hortonworks Certified Hadoop professional and Administrator, Certified Spark Developer, SAS Certified Predictive Modeler, along with being a Sun Certified Enterprise Architect and a Teradata Certified Master. JOEL FARVAULT is a Principal Solutions Architect Analytics at Amazon Web Services. He has 25 years’ experience working on enterprise architecture, data strategy, and analytics, mainly in the financial services industry. Joel has led data transformation projects on fraud analytics, business intelligence, and data governance. He is also a lecturer on Data Analytics at IA School, at Neoma Business School and at Ecole Superieure de Genie Informatique (ESGI). Joel holds several associate and specialty certifications on AWS.

• Acknowledgments xiiiAbout the Authors xvForeword xviiIntroduction xixChapter 1: A Brief History of AI 1The Precursors of the Mechanical or “Formal” Reasoning 2The Digital Computer Era 4Cybernetics and the Beginning of the Robotic Era 6Birth of AI and Symbolic AI (1955–1985) 10Subsymbolic AI Era (1985–2010) 14Deep Learning and LLM (2010–Present) 16Key Takeaways 17Chapter 2: Machine Learning 19What Is Machine Learning? 19Types of Machine Learning 20Supervised Learning 21Unsupervised and Semi-Supervised Learning 22Reinforcement Learning 23Methodology for Machine Learning 24Implementation of Machine Learning 26Machine Learning Applications 27Natural Language Processing (NLP) 27Computer Vision 27Recommender System 27Predictive Analytics 28Fraud Detection 28Machine Learning Frameworks and Libraries 28TensorFlow 28PyTorch 31Scikit-learn 34Keras 35Apache Spark MLlib 37Future Trends in Machine Learning 40Rise of Edge Computing and Edge AI 40Convergence with Emerging Technologies 40Advancements in Unsupervised Learning, Reinforcement Learning, and Generative Models 41Increased Specialization and Customization 41Explainable and Trustworthy AI 42Key Takeaways 42References 43Chapter 3: Deep Learning 45Deep Learning vs. Machine Learning 45Computer Vision Example 46Natural Language Processing Example 47The History of Deep Learning 47Understanding Deep Learning 52Neurons 52Weights and Biases 54Layers 54Activation Function(s) 55An Introduction to the Perceptron 58Overcoming Perceptron Limitations 59FeedForward Neural Networks 60Backpropagation 60Parameters vs. Hyperparameters 62Hyperparameters in Artificial Neural Networks 64Loss Functions – a Measure of Success of a Neural Network 64Optimization Algorithms 64Neural Network Architectures 68Putting It All Together 71Deep Learning on AWS 71Chipsets and EC2 Instances 71AWS P5 Instances 72AWS Inferentia 72Amazon Elastic Inference 73Pre-built Containers: Deep Learning AMIs and Containers 74Deep Learning AMIs 74Deep Learning Containers 74Managed Services for Building, Training, and Deployment 74Pre-trained Services 75Key Takeaways 77References 77Chapter 4: Introduction to Generative AI 79Generative AI Core Technologies 80Neural Networks 80Generative Adversarial Networks (GANs) 80Variational Autoencoders (VAEs) 81Recurrent Neural Networks (RNNs) and Long Short-Term Memory Networks (LSTMs) 82Limitations of Recurrent Neural Networks 84Transformer Models 85Self-Attention 86Parallelism 86Diffusion Models 86Autoregressive Models 87Reinforcement Learning (RL) 87Transfer Learning and Fine-Tuning 87Optimization Algorithms 87Transformer Architecture: Deep Dive 87Deep Dive 89Step 1: Tokenization (Preprocessing) 89Step 2: Embedding 89Step 3: Encoder 92Step 4: Encoder Output to Decoder Input 97Step 5: Decoder 98Step 6: Translation Generation 99Step 7: Detokenization 99Terminology in Generative AI 99Prompt 104Inference 105Context Window 106Prompt Engineering 106In-Context Learning (ICL) 107Zero-Shot/One-Shot/Few-Shot Inference 108Inference Configuration 109Maximum Length 110Diversity (Top P/Nucleus Sampling) 111Top K 111Randomness (Temperature) 112System Prompts 112Prompt Engineering 113Key Elements of a Prompt 113Designing Effective Prompts 114Prompting Techniques 115Zero-Shot Prompting 115Few-Shot Prompting 115Chain-of-Thought Prompting 116Advanced Prompting Techniques 117Self-Consistency 118Tree of Thoughts (ToT) 119Retrieval-Augmented Generation (RAG) 120Automatic Reasoning and Tool-Use (ART) 122ReAct Prompting 123Coherence Enhancement 124Progressive Prompting 126Handling Prompt Misuse 127Prompt Injection 127Prompt Leaking 128Mitigating Bias 129Mitigating Bias in Prompt Engineering 130Generative AI Business Value 133Building Value Within Your Enterprises 135Technology: Creating a Flexible and Strong System 136People: Training and Adapting the Team 136Processes: Good Management and Fair Use of AI 136Why a Solid Foundation Is Crucial 136Summary 137References 137Chapter 5: Introduction to Foundation Models 139Definition and Overview of Foundation Models 139Characteristics of Foundation Models 142Examples of Foundation Models 144Types of Foundation Models 147The Large Language Model (LLM) 154Natural Language Processing 155Early Approaches to NLP 156Evolution Toward Text-Based Foundation Model 160Applications of Foundation Models 162Challenges and Considerations 163Infrastructure 163Ethics 164Areas of Evolution 165Key Takeaways 167References 168Chapter 6: Introduction to Amazon SageMaker 169Data Preparation and Processing 172Data Preparation 172Data Processing 173Model Development 174Model Training and Tuning 175Model Deployment 177Model Management 178Security 179Compliance and Governance 180Model Explainability and Responsible AI 181MLOps with Amazon SageMaker 181Boost Your Generative AI Development with SageMaker JumpStart 182No-Code ML with Amazon SageMaker Canvas 183Amazon Bedrock 184Choosing the Right Strategy for the Development of Your Generative AI Application with Amazon SageMaker 186Conclusion 187References 188Chapter 7: Generative AI on AWS 191AWS Services for Generative AI 192Generative AI Trade-Off Triangle 192How AWS Solves the Generative AI Trade-Off Triangle 192Generative AI on AWS: The Fundamentals 193Infrastructure for FM Training and Inference 194Models and Tools to Build Generative AI Apps 194Applications to Boost Productivity 195Amazon Bedrock 196Foundation Models with Bedrock 197AI21 Labs – Jurassic 197Amazon Titan 198Anthropic’s Claude 3 199Cohere’s Family of Models 201Key Features of Cohere 201Cohere Models on Amazon Bedrock 203Meta’s Family of Models – Llama 204When to Use Which Model 207Mistral’s Family of Models 208When to Use Which Model 209Stability.ai’s Family of Models – Stable Diffusion XL 1.0 209Poolside Family of Models 210Luma’s Family of Models 211Amazon’s Nova Family of Models 212Model Evaluation in Amazon Bedrock 213Common Approaches to Customizing Your FMs 214Amazon Bedrock Prompt Management 214Amazon Bedrock Flows 216Data Automation in Amazon Bedrock 219GraphRAG in Amazon Bedrock 220Knowledge Bases in Amazon Bedrock 222How Knowledge Bases Work 223Pre-Processing Data 224Runtime Execution 224Creating a Knowledge Base in Amazon Bedrock 225Agents for Amazon Bedrock 225How Agents Work 226Components of an Agent at Build Time 226Components of an Agent at Runtime 228Guardrails for Amazon Bedrock 230Security in Amazon Bedrock 231Amazon Q 232Amazon Q Business 232Amazon Q in QuickSight 235Amazon Q Developer 238Amazon Q Connect 239Amazon Q in AWS Supply Chain 241Summary 241Chapter 8: Customization of Your Foundation Model 243Introduction to LLM Customization 244Continued Pre-Training (Domain Adaptation Fine-Tuning) 244Fine-Tuning 245Prompt Engineering 245Retrieval Augmented Generation (RAG) 246Choosing Between These Customization Techniques 246Cost of Customization 249Customizing Foundation Models with AWS 250Continuous Pre-Training with Amazon Bedrock 250Creation of a Training and a Validation Dataset 250Launch of a Continued Pre-Training Job 251Analysis of Our Results and Adjustment of Our Hyperparameters 252Deployment of Our Model 254Use Your Customized Model 255Instruction Fine-Tuning with Amazon Bedrock 257Instruction Fine-Tuning with Amazon SageMaker JumpStart 257Conclusion 260Chapter 9: Retrieval-Augmented Generation 263What Is RAG? 263Background and Motivation 264Overview of RAG 266Building a RAG Solution 269Design Considerations 269Best Practices 270Common Patterns 271Performance Optimization 271Scaling Considerations 272The Future of RAG Implementations 273Retrieval Module 274Retrieval Techniques and Algorithms 276Augmentation Module 278Generation Module 280RAG on AWS 282Custom Data Pipeline to Build RAG 284Core Components of a RAG Pipeline 284Implementation Approaches 286Basic Solution: LangChain Implementation 286Advanced Solution: Spark-Based Pipeline 287Data Ingestion (Examples) 288Parallel Processing (example) 288Case Studies and Applications 290Question-Answering Systems 290Dialogue Systems 290Knowledge-Intensive Tasks 291Implementation Considerations and Best Practices 291Challenges and Future Directions 292Example Notebooks 293References 293Chapter 10: Generative AI on AWS Labs 295Lab 1: Introduction to Generative AI with Bedrock 295Option 1: PartyRock Prompt Engineering Guide (for Non-Technical and Technical Audiences) 297Option 2: Amazon Bedrock Labs (for Technical Audiences) 298Overview of Amazon Bedrock and Streamlit 298Supported Regions 298Costs When Running from Your Own Account 298Quotas When Running from Your Own Account 299Time to Complete 299Lab 2: Dive Deep into Gen AI with Amazon Bedrock 299Lab 3: Building an Agentic LLM Assistant on AWS 300What Is an Agentic LLM Assistant? 300Why Build an Agentic LLM Assistant? 301About This Workshop 301Architecture 301Labs 302Lab 4: Retrieval-Augmented Generation Workshop 303Managed RAG Workshop 304Naive RAG Workshop 304Advance RAG Workshop 304Audience 304Lab 5: Amazon Q for Business 304Next Steps 307Lab 6: Building a Natural Language Query Engine for Data Lakes 308Summary 310Reference 310Chapter 11: Next Steps 311The Future of Generative AI: Key Dimensions and Staying Informed 311Technical Evolution and Capabilities 312The Evolution of Scale and Architecture 312The Multimodal Revolution 312The Efficiency Breakthrough 313The Context Window Revolution 313Real-time Processing and Generation 313The Future Technological Landscape 314Application Domains 314Enterprise Applications: The Quiet Revolution 315The Scientific Frontier: Accelerating Discovery 315Healthcare: Personalized Medicine and Diagnosis 315Education and Training: Personalizing Learning 316Environmental Applications: Tackling Global Challenges 316The Future of Applications 317Ethical and Societal Implications 317Digital Identity and Deep Fakes: The Crisis of Trust 318Labor Markets and Economic Disruption 318Privacy and Data Rights in the Age of AI 318Bias and Fairness: The Hidden Challenges 319Democratic Access and Digital Divides 319Environmental and Sustainability Concerns 319The Path Forward: Governance and Responsibility 319Looking to the Future 320Staying Current in the Rapidly Evolving AI Landscape 320Glossary 323Index 333