This book takes you step-by-step through creating your own AI models that can generate images from text. Youll explore two methods of image generationvision transformers and diffusion modelsand learn vital AI development techniques as you go.Dive into the powerful models behind AI image generators. The best way to learn is to build something from scratch, and in this book youll build your very own diffusion model and vision transformer. As you work through each stage of development, youll develop an understanding of how these models can be customized, applied, and integrated for impressive multimodal AI.Build a Text-to-Image Generator (from Scratch) teaches you how to:Build and train models to generate high resolution images based on text descriptionsEdit an existing image based on text promptsBuild and train a model to add captions to imagesBuild and train a vision transformer to classify imagesFine-tune LLMs for downstream tasks such as classification, text or image generationBetter differentiate real images from deepfakesAbout the technologyAI-generated images appear everywhere from high-end advertising to casual social media feeds. Text-to-image tools like Dall-e, Midjourney, and Flux make it easy to create AI art, but how do they work? In this book, youll find out by building your own text-to-image generator!About the bookBuild a Text-to-Image Generator (from Scratch) explores both transformer-based image generation and diffusion models. Youll work hands-on to build a pair of simple generation models that can classify images, automatically add captions, reconstruct images, and enhance existing graphics. Author Mark Liu guides you every step of the way with clear explanations, informative diagrams, and eye-opening examples you can build on your own laptop.What's insideBuild a vision transformer to classify imagesEdit images using text promptsFine-tune image modelsAbout the readerRequires basic knowledge of generative AI models and intermediate Python skills.About the authorMark Liu is the founding director of the Master of Science in Finance program at the University of Kentucky. He is also the author of Learn Generative AI with PyTorch.Table of ContentsPart 11 A tale of two models: Transformers and diffusions2 Build a transformer3 Classify images with a vision transformer4 Add captions to imagesPart 25 Generate images with diffusion models6 Control what images to generate in diffusion models7 Generate high-resolution images with diffusion modelsPart 38 CLIP: A model to measure the similarity between image and text9 Text-to-image generation with latent diffusion10 A deep dive into Stable DiffusionPart 411 VQGAN: Convert images into sequences of integers12 A minimal implementation of DALL-EPart 513 New developments and challenges in text-to-image generationA Installing PyTorch and enabling GPU training locally and in Colab
