Holden Karau - Böcker

Serverless computing enables developers to concentrate solely on their applications rather than worry about where they've been deployed. With the Ray general-purpose serverless implementation in Python, programmers and data scientists can hide servers, implement stateful applications, support direct communication between tasks, and access hardware accelerators.In this book, authors Holden Karau and Boris Lublinsky show you how to scale existing Python applications and pipelines, allowing you to stay in the Python ecosystem while avoiding single points of failure and manual scheduling. If your data processing has grown beyond what a single computer can handle, this book is for you.Written by experienced software architecture practitioners, Scaling Python with Ray is ideal for software architects and developers eager to explore successful case studies and learn more about decision and measurement effectiveness. This book covers distributed processing (the pure Python implementation of serverless) and shows you how to:Implement stateful applications with Ray actorsBuild workflow management in RayUse Ray as a unified platform for batch and streamingImplement advanced data processing with RayApply microservices with Ray platformImplement reliable Ray applications

Modern systems contain multi-core CPUs and GPUs that have the potential for parallel computing. But many scientific Python tools were not designed to leverage this parallelism. With this short but thorough resource, data scientists and Python programmers will learn how the Dask open source library for parallel computing provides APIs that make it easy to parallelize PyData libraries including NumPy, pandas, and scikit-learn.Authors Holden Karau and Mika Kimmins show you how to use Dask computations in local systems and then scale to the cloud for heavier workloads. This practical book explains why Dask is popular among industry experts and academics and is used by organizations that include Walmart, Capital One, Harvard Medical School, and NASA.With this book, you'll learn:What Dask is, where you can use it, and how it compares with other toolsHow to use Dask for batch data parallel processingKey distributed system concepts for working with DaskMethods for using Dask with higher-level APIs and building blocksHow to work with integrated libraries such as scikit-learn, pandas, and PyTorchHow to use Dask with GPUs

Apache Spark is amazing when everything clicks. But if you haven't seen the performance improvements you expected or still don't feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau, Rachel Warren, and Anya Bida walk you through the secrets of the Spark code base, and demonstrate performance optimizations that will help your data pipelines run faster, scale to larger datasets, and avoid costly antipatterns.Ideal for data engineers, software engineers, data scientists, and system administrators, the second edition of High Performance Spark presents new use cases, code examples, and best practices for Spark 3.x and beyond. This book gives you a fresh perspective on this continually evolving framework and shows you how to work around bumps on your Spark and PySpark journey.With this book, you'll learn how to:Accelerate your ML workflows with integrations including PyTorchHandle key skew and take advantage of Spark's new dynamic partitioningMake your code reliable with scalable testing and validation techniquesMake Spark high performanceDeploy Spark on Kubernetes and similar environmentsTake advantage of GPU acceleration with RAPIDS and resource profilesGet your Spark jobs to run fasterUse Spark to productionize exploratory data science projectsHandle even larger datasets with SparkGain faster insights by reducing pipeline running times

Apache Spark is amazing when everything clicks. But if you haven't seen the performance improvements you expected, or still don't feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau and Rachel Warren demonstrate performance optimizations to help your Spark queries run faster and handle larger data sizes, while using fewer resources. Ideal for software engineers, data engineers, developers, and system administrators working with large-scale data applications, this book describes techniques that can reduce data infrastructure costs and developer hours. Not only will you gain a more comprehensive understanding of Spark, you'll also learn how to make it sing.With this book, you'll explore: How Spark SQL's new interfaces improve performance over SQL's RDD data structure The choice between data joins in Core Spark and Spark SQL Techniques for getting the most out of standard RDD transformations How to work around performance issues in Spark's key/value pair paradigm Writing high-performance Spark code without Scala or the JVM How to test for functionality and performance when applying suggested improvements Using Spark MLlib and Spark ML machine learning libraries Spark's Streaming components and external community packages