David Mertz – författare

Most machine learning guides cover data cleaning briefly or skip it entirely. However, many data scientists and analysts spend most of their time on data cleaning and data quality tasks, and their effectiveness can make or break project success. In Data Cleaning for Effective Data Science, leading data science trainer David Mertz provides the most systematic guide to cleaning data for any project, using any library or toolset. Mertz introduces many powerful techniques for analyzing, manipulating, and pre-processing data sources. He offers best practices for working with leading data formats such as JSON, CSV, SQL RDBMSes, HDF5, NoSQL databases, files in image formats, binary serialized data structures, and more. Mertz also focuses on crucial issues within the data itself, including missing data, outliers, biasing trends, class imbalance, value imputation, over/under-sampling, normalization and/or randomization, and anomalies. This guide is organized around downloadable datasets, each illuminating specific issues with data integrity or quality. Each chapter explores the best ways to diagnose, analyze, and remediate these issues, offering hands-on practice using tools such as Python, Pandas, sklearn.preprocessing, scipy.stats, R, and Tidyverse. While the examples are demonstrated with widely-used tools, Mertz's concepts are applicable with any toolset. Each chapter also links to additional datasets with more problems, exercises, and solutions.

Move Beyond Python Code That "Mostly Works" to Code That Is Expressive, Robust, and Efficient

Python is arguably the most-used programming language in the world, with applications from primary school education to workaday web development, to the most advanced scientific research institutes. While there are many ways to perform a task in Python, some are wrong, inelegant, or inefficient. Better Python Code is a guide to "Pythonic" programming, a collection of best practices, ways of working, and nuances that are easy to miss, especially when ingrained habits are borrowed from other programming languages.

Author David Mertz presents concrete and concise examples of various misunderstandings, pitfalls, and bad habits in action. He explains why some practices are better than others, based on his 25+ years of experience as an acclaimed contributor to the Python community. Each chapter thoroughly covers related clusters of concepts, with chapters sequenced in ascending order of sophistication.

Whether you are starting out with Python or are an experienced developer pushing through the limitations of your Python code, this book is for all who aspire to be more Pythonic when writing better Python code.

Use the right kind of loops in Python Learn the ins and outs of mutable and immutable objects Get expert advice to avoid Python "gotchas" Examine advanced Python topics Navigate the "attractive nuisances" that exist in Python Learn the most useful data structures in Python and how to avoid misusing them Avoid security mistakes Understand the basics of numeric computation, including floating point numbers and numeric datatypes

"My high expectations for this engaging Python book have been exceeded: it offers a great deal of insight for intermediate or advanced programmers to improve their Python skills, includes copious sharing of precious experience practicing and teaching the language, yet remains concise, easy to read, and conversational."--From the Foreword by Alex Martelli

Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.

Think about your data intelligently and ask the right questionsKey FeaturesMaster data cleaning techniques necessary to perform real-world data science and machine learning tasksSpot common problems with dirty data and develop flexible solutions from first principlesTest and refine your newly acquired skills through detailed exercises at the end of each chapterBook DescriptionData cleaning is the all-important first step to successful data science, data analysis, and machine learning. If you work with any kind of data, this book is your go-to resource, arming you with the insights and heuristics experienced data scientists had to learn the hard way.In a light-hearted and engaging exploration of different tools, techniques, and datasets real and fictitious, Python veteran David Mertz teaches you the ins and outs of data preparation and the essential questions you should be asking of every piece of data you work with.Using a mixture of Python, R, and common command-line tools, Cleaning Data for Effective Data Science follows the data cleaning pipeline from start to end, focusing on helping you understand the principles underlying each step of the process. You'll look at data ingestion of a vast range of tabular, hierarchical, and other data formats, impute missing values, detect unreliable data and statistical anomalies, and generate synthetic features. The long-form exercises at the end of each chapter let you get hands-on with the skills you've acquired along the way, also providing a valuable resource for academic courses.What you will learnIngest and work with common data formats like JSON, CSV, SQL and NoSQL databases, PDF, and binary serialized data structuresUnderstand how and why we use tools such as pandas, SciPy, scikit-learn, Tidyverse, and BashApply useful rules and heuristics for assessing data quality and detecting bias, like Benford's law and the 68-95-99.7 ruleIdentify and handle unreliable data and outliers, examining z-score and other statistical propertiesImpute sensible values into missing data and use sampling to fix imbalancesUse dimensionality reduction, quantization, one-hot encoding, and other feature engineering techniques to draw out patterns in your dataWork carefully with time series data, performing de-trending and interpolationWho this book is forThis book is designed to benefit software developers, data scientists, aspiring data scientists, teachers, and students who work with data. If you want to improve your rigor in data hygiene or are looking for a refresher, this book is for you.Basic familiarity with statistics, general concepts in machine learning, knowledge of a programming language (Python or R), and some exposure to data science are helpful.