Holden Karau – författare

Serverless computing enables developers to concentrate solely on their applications rather than worry about where they''ve been deployed. With the Ray general-purpose serverless implementation in Python, programmers and data scientists can hide servers, implement stateful applications, support direct communication between tasks, and access hardware accelerators.

In this book, experienced software architecture practitioners Holden Karau and Boris Lublinsky show you how to scale existing Python applications and pipelines, allowing you to stay in the Python ecosystem while reducing single points of failure and manual scheduling. Scaling Python with Ray is ideal for software architects and developers eager to explore successful case studies and learn more about decision and measurement effectiveness.

If your data processing or server application has grown beyond what a single computer can handle, this book is for you. You''ll explore distributed processing (the pure Python implementation of serverless) and learn how to:

Implement stateful applications with Ray actorsBuild workflow management in RayUse Ray as a unified system for batch and stream processingApply advanced data processing with RayBuild microservices with RayImplement reliable Ray applications

Serverless computing enables developers to concentrate solely on their applications rather than worry about where they''ve been deployed. With the Ray general-purpose serverless implementation in Python, programmers and data scientists can hide servers, implement stateful applications, support direct communication between tasks, and access hardware accelerators.

In this book, experienced software architecture practitioners Holden Karau and Boris Lublinsky show you how to scale existing Python applications and pipelines, allowing you to stay in the Python ecosystem while reducing single points of failure and manual scheduling. Scaling Python with Ray is ideal for software architects and developers eager to explore successful case studies and learn more about decision and measurement effectiveness.

If your data processing or server application has grown beyond what a single computer can handle, this book is for you. You''ll explore distributed processing (the pure Python implementation of serverless) and learn how to:

Implement stateful applications with Ray actorsBuild workflow management in RayUse Ray as a unified system for batch and stream processingApply advanced data processing with RayBuild microservices with RayImplement reliable Ray applications

Serverless computing enables developers to concentrate solely on their applications rather than worry about where they've been deployed. With the Ray general-purpose serverless implementation in Python, programmers and data scientists can hide servers, implement stateful applications, support direct communication between tasks, and access hardware accelerators.In this book, authors Holden Karau and Boris Lublinsky show you how to scale existing Python applications and pipelines, allowing you to stay in the Python ecosystem while avoiding single points of failure and manual scheduling. If your data processing has grown beyond what a single computer can handle, this book is for you.Written by experienced software architecture practitioners, Scaling Python with Ray is ideal for software architects and developers eager to explore successful case studies and learn more about decision and measurement effectiveness. This book covers distributed processing (the pure Python implementation of serverless) and shows you how to:Implement stateful applications with Ray actorsBuild workflow management in RayUse Ray as a unified platform for batch and streamingImplement advanced data processing with RayApply microservices with Ray platformImplement reliable Ray applications

Modern systems contain multi-core CPUs and GPUs that have the potential for parallel computing. But many scientific Python tools were not designed to leverage this parallelism. With this short but thorough resource, data scientists and Python programmers will learn how the Dask open source library for parallel computing provides APIs that make it easy to parallelize PyData libraries including NumPy, pandas, and scikit-learn.

Authors Holden Karau and Mika Kimmins show you how to use Dask computations in local systems and then scale to the cloud for heavier workloads. This practical book explains why Dask is popular among industry experts and academics and is used by organizations that include Walmart, Capital One, Harvard Medical School, and NASA.

With this book, you''ll learn:

What Dask is, where you can use it, and how it compares with other toolsHow to use Dask for batch data parallel processingKey distributed system concepts for working with DaskMethods for using Dask with higher-level APIs and building blocksHow to work with integrated libraries such as scikit-learn, pandas, and PyTorchHow to use Dask with GPUs

Modern systems contain multi-core CPUs and GPUs that have the potential for parallel computing. But many scientific Python tools were not designed to leverage this parallelism. With this short but thorough resource, data scientists and Python programmers will learn how the Dask open source library for parallel computing provides APIs that make it easy to parallelize PyData libraries including NumPy, pandas, and scikit-learn.

Authors Holden Karau and Mika Kimmins show you how to use Dask computations in local systems and then scale to the cloud for heavier workloads. This practical book explains why Dask is popular among industry experts and academics and is used by organizations that include Walmart, Capital One, Harvard Medical School, and NASA.

With this book, you''ll learn:

What Dask is, where you can use it, and how it compares with other toolsHow to use Dask for batch data parallel processingKey distributed system concepts for working with DaskMethods for using Dask with higher-level APIs and building blocksHow to work with integrated libraries such as scikit-learn, pandas, and PyTorchHow to use Dask with GPUs

Modern systems contain multi-core CPUs and GPUs that have the potential for parallel computing. But many scientific Python tools were not designed to leverage this parallelism. With this short but thorough resource, data scientists and Python programmers will learn how the Dask open source library for parallel computing provides APIs that make it easy to parallelize PyData libraries including NumPy, pandas, and scikit-learn.Authors Holden Karau and Mika Kimmins show you how to use Dask computations in local systems and then scale to the cloud for heavier workloads. This practical book explains why Dask is popular among industry experts and academics and is used by organizations that include Walmart, Capital One, Harvard Medical School, and NASA.With this book, you'll learn:What Dask is, where you can use it, and how it compares with other toolsHow to use Dask for batch data parallel processingKey distributed system concepts for working with DaskMethods for using Dask with higher-level APIs and building blocksHow to work with integrated libraries such as scikit-learn, pandas, and PyTorchHow to use Dask with GPUs

Apache Spark is amazing when everything clicks. But if you haven't seen the performance improvements you expected or still don't feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau, Adi Polak, and Rachel Warren walk you through the secrets of the Spark code base and demonstrate performance optimizations that will help your data pipelines run faster, scale to larger datasets, and avoid costly antipatterns.Ideal for data engineers, software engineers, data scientists, and system administrators, the second edition of High Performance Spark presents new use cases, code examples, and best practices for Spark 4.x and beyond. This book gives you a fresh perspective on this continually evolving framework and shows you how to work around bumps on your Spark and PySpark journey. With this book, you'll learn how to:Accelerate your ML workflows with integrations including PyTorchHandle key skew and take advantage of Spark's new dynamic partitioningMake your code reliable with scalable testing and validation techniquesMake Spark high performanceDeploy Spark on Kubernetes and similar environmentsTake advantage of GPU acceleration with RAPIDS and resource profilesGet your Spark jobs to run fasterUse Spark to productionize exploratory data science projectsHandle even larger datasets with SparkGain faster insights by reducing pipeline running times

Apache Spark is amazing when everything clicks. But if you haven't seen the performance improvements you expected or still don't feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau, Adi Polak, and Rachel Warren walk you through the secrets of the Spark code base and demonstrate performance optimizations that will help your data pipelines run faster, scale to larger datasets, and avoid costly antipatterns.Ideal for data engineers, software engineers, data scientists, and system administrators, the second edition of High Performance Spark presents new use cases, code examples, and best practices for Spark 4.x and beyond. This book gives you a fresh perspective on this continually evolving framework and shows you how to work around bumps on your Spark and PySpark journey. With this book, you'll learn how to:Accelerate your ML workflows with integrations including PyTorchHandle key skew and take advantage of Spark's new dynamic partitioningMake your code reliable with scalable testing and validation techniquesMake Spark high performanceDeploy Spark on Kubernetes and similar environmentsTake advantage of GPU acceleration with RAPIDS and resource profilesGet your Spark jobs to run fasterUse Spark to productionize exploratory data science projectsHandle even larger datasets with SparkGain faster insights by reducing pipeline running times

Apache Spark is amazing when everything clicks. But if you haven't seen the performance improvements you expected or still don't feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau, Rachel Warren, and Anya Bida walk you through the secrets of the Spark code base, and demonstrate performance optimizations that will help your data pipelines run faster, scale to larger datasets, and avoid costly antipatterns.Ideal for data engineers, software engineers, data scientists, and system administrators, the second edition of High Performance Spark presents new use cases, code examples, and best practices for Spark 3.x and beyond. This book gives you a fresh perspective on this continually evolving framework and shows you how to work around bumps on your Spark and PySpark journey.With this book, you'll learn how to:Accelerate your ML workflows with integrations including PyTorchHandle key skew and take advantage of Spark's new dynamic partitioningMake your code reliable with scalable testing and validation techniquesMake Spark high performanceDeploy Spark on Kubernetes and similar environmentsTake advantage of GPU acceleration with RAPIDS and resource profilesGet your Spark jobs to run fasterUse Spark to productionize exploratory data science projectsHandle even larger datasets with SparkGain faster insights by reducing pipeline running times

Data in all domains is getting bigger. How can you work with it efficiently? This book introduces Apache Spark, the open source cluster computing system that makes data analytics fast to write and fast to run. With Spark, you can tackle big datasets quickly through simple APIs in Python, Java, and Scala.

Written by the developers of Spark, this book will have data scientists and engineers up and running in no time. You’ll learn how to express parallel jobs with just a few lines of code, and cover applications from simple batch jobs to stream processing and machine learning.

Quickly dive into Spark capabilities such as distributed datasets, in-memory caching, and the interactive shellLeverage Spark’s powerful built-in libraries, including Spark SQL, Spark Streaming, and MLlibUse one programming paradigm instead of mixing and matching tools like Hive, Hadoop, Mahout, and StormLearn how to deploy interactive, batch, and streaming applicationsConnect to data sources including HDFS, Hive, JSON, and S3Master advanced topics like data partitioning and shared variables

Data in all domains is getting bigger. How can you work with it efficiently? This book introduces Apache Spark, the open source cluster computing system that makes data analytics fast to write and fast to run. With Spark, you can tackle big datasets quickly through simple APIs in Python, Java, and Scala.

Written by the developers of Spark, this book will have data scientists and engineers up and running in no time. You’ll learn how to express parallel jobs with just a few lines of code, and cover applications from simple batch jobs to stream processing and machine learning.

Quickly dive into Spark capabilities such as distributed datasets, in-memory caching, and the interactive shellLeverage Spark’s powerful built-in libraries, including Spark SQL, Spark Streaming, and MLlibUse one programming paradigm instead of mixing and matching tools like Hive, Hadoop, Mahout, and StormLearn how to deploy interactive, batch, and streaming applicationsConnect to data sources including HDFS, Hive, JSON, and S3Master advanced topics like data partitioning and shared variables

If you''re training a machine learning model but aren''t sure how to put it into production, this book will get you there. Kubeflow provides a collection of cloud native tools for different stages of a model''s lifecycle, from data exploration, feature preparation, and model training to model serving. This guide helps data scientists build production-grade machine learning implementations with Kubeflow and shows data engineers how to make models scalable and reliable.

Using examples throughout the book, authors Holden Karau, Trevor Grant, Ilan Filonenko, Richard Liu, and Boris Lublinsky explain how to use Kubeflow to train and serve your machine learning models on top of Kubernetes in the cloud or in a development environment on-premises.

Understand Kubeflow''s design, core components, and the problems it solvesUnderstand the differences between Kubeflow on different cluster typesTrain models using Kubeflow with popular tools including Scikit-learn, TensorFlow, and Apache SparkKeep your model up to date with Kubeflow PipelinesUnderstand how to capture model training metadataExplore how to extend Kubeflow with additional open source toolsUse hyperparameter tuning for trainingLearn how to serve your model in production

If you''re training a machine learning model but aren''t sure how to put it into production, this book will get you there. Kubeflow provides a collection of cloud native tools for different stages of a model''s lifecycle, from data exploration, feature preparation, and model training to model serving. This guide helps data scientists build production-grade machine learning implementations with Kubeflow and shows data engineers how to make models scalable and reliable.

Using examples throughout the book, authors Holden Karau, Trevor Grant, Ilan Filonenko, Richard Liu, and Boris Lublinsky explain how to use Kubeflow to train and serve your machine learning models on top of Kubernetes in the cloud or in a development environment on-premises.

Understand Kubeflow''s design, core components, and the problems it solvesUnderstand the differences between Kubeflow on different cluster typesTrain models using Kubeflow with popular tools including Scikit-learn, TensorFlow, and Apache SparkKeep your model up to date with Kubeflow PipelinesUnderstand how to capture model training metadataExplore how to extend Kubeflow with additional open source toolsUse hyperparameter tuning for trainingLearn how to serve your model in production

If you're training a machine learning model but aren't sure how to put it into production, this book will get you there. Kubeflow provides a collection of cloud native tools for different stages of a model's lifecycle, from data exploration, feature preparation, and model training to model serving. This guide helps data scientists build production-grade machine learning implementations with Kubeflow and shows data engineers how to make models scalable and reliable.Using examples throughout the book, authors Holden Karau, Trevor Grant, Ilan Filonenko, Richard Liu, and Boris Lublinsky explain how to use Kubeflow to train and serve your machine learning models on top of Kubernetes in the cloud or in a development environment on-premises. Understand Kubeflow's design, core components, and the problems it solvesLearn how to set up Kubeflow on a cloud provider or on an in-house clusterTrain models using Kubeflow with popular tools including scikit-learn, TensorFlow, and Apache SparkLearn how to add custom stages such as serving and predictionKeep your model up-to-date with Kubeflow PipelinesUnderstand how to validate machine learning pipelines

Learn how to use Spark to process big data at speed and scale for sharper analytics. Put the principles into practice for faster, slicker big data projects.

About This Book

A quick way to get started with Spark – and reap the rewardsFrom analytics to engineering your big data architecture, we''ve got it coveredBring your Scala and Java knowledge – and put it to work on new and exciting problems

Who This Book Is For

This book is for developers with little to no knowledge of Spark, but with a background in Scala/Java programming. It''s recommended that you have experience in dealing and working with big data and a strong interest in data science.

What You Will Learn

Install and set up Spark in your clusterPrototype distributed applications with Spark''s interactive shellPerform data wrangling using the new DataFrame APIsGet to know the different ways to interact with Spark''s distributed representation of data (RDDs)Query Spark with a SQL-like query syntaxSee how Spark works with big dataImplement machine learning systems with highly scalable algorithmsUse R, the popular statistical language, to work with SparkApply interesting graph algorithms and graph processing with GraphX

In Detail

When people want a way to process big data at speed, Spark is invariably the solution. With its ease of development (in comparison to the relative complexity of Hadoop), it''s unsurprising that it''s becoming popular with data analysts and engineers everywhere.

Beginning with the fundamentals, we''ll show you how to get set up with Spark with minimum fuss. You''ll then get to grips with some simple APIs before investigating machine learning and graph processing – throughout we''ll make sure you know exactly how to apply your knowledge.

You will also learn how to use the Spark shell, how to load data before finding out how to build and run your own Spark applications. Discover how to manipulate your RDD and get stuck into a range of DataFrame APIs. As if that''s not enough, you''ll also learn some useful Machine Learning algorithms with the help of Spark MLlib and integrating Spark with R. We''ll also make sure you''re confident and prepared for graph processing, as you learn more about the GraphX API.

Style and approach

This book is a basic, step-by-step tutorial that will help you take advantage of all that Spark has to offer.