David Mertz – författare

Most machine learning guides cover data cleaning briefly or skip it entirely. However, many data scientists and analysts spend most of their time on data cleaning and data quality tasks, and their effectiveness can make or break project success. In Data Cleaning for Effective Data Science, leading data science trainer David Mertz provides the most systematic guide to cleaning data for any project, using any library or toolset. Mertz introduces many powerful techniques for analyzing, manipulating, and pre-processing data sources. He offers best practices for working with leading data formats such as JSON, CSV, SQL RDBMSes, HDF5, NoSQL databases, files in image formats, binary serialized data structures, and more. Mertz also focuses on crucial issues within the data itself, including missing data, outliers, biasing trends, class imbalance, value imputation, over/under-sampling, normalization and/or randomization, and anomalies. This guide is organized around downloadable datasets, each illuminating specific issues with data integrity or quality. Each chapter explores the best ways to diagnose, analyze, and remediate these issues, offering hands-on practice using tools such as Python, Pandas, sklearn.preprocessing, scipy.stats, R, and Tidyverse. While the examples are demonstrated with widely-used tools, Mertz's concepts are applicable with any toolset. Each chapter also links to additional datasets with more problems, exercises, and solutions.

Move Beyond Python Code That "Mostly Works" to Code That Is Expressive, Robust, and Efficient

Python is arguably the most-used programming language in the world, with applications from primary school education to workaday web development, to the most advanced scientific research institutes. While there are many ways to perform a task in Python, some are wrong, inelegant, or inefficient. Better Python Code is a guide to "Pythonic" programming, a collection of best practices, ways of working, and nuances that are easy to miss, especially when ingrained habits are borrowed from other programming languages.

Author David Mertz presents concrete and concise examples of various misunderstandings, pitfalls, and bad habits in action. He explains why some practices are better than others, based on his 25+ years of experience as an acclaimed contributor to the Python community. Each chapter thoroughly covers related clusters of concepts, with chapters sequenced in ascending order of sophistication.

Whether you are starting out with Python or are an experienced developer pushing through the limitations of your Python code, this book is for all who aspire to be more Pythonic when writing better Python code.

Use the right kind of loops in Python Learn the ins and outs of mutable and immutable objects Get expert advice to avoid Python "gotchas" Examine advanced Python topics Navigate the "attractive nuisances" that exist in Python Learn the most useful data structures in Python and how to avoid misusing them Avoid security mistakes Understand the basics of numeric computation, including floating point numbers and numeric datatypes

"My high expectations for this engaging Python book have been exceeded: it offers a great deal of insight for intermediate or advanced programmers to improve their Python skills, includes copious sharing of precious experience practicing and teaching the language, yet remains concise, easy to read, and conversational."--From the Foreword by Alex Martelli

Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.

Learn how AI-assisted coding using ChatGPT and GitHub Copilot can dramatically increase your productivity (and fun) writing regular expressions and other programs.Regular Expression Puzzles and AI Coding Assistants is the story of two competitors. On one side is David Mertz, an expert programmer and the author of the Web's most popular Regex tutorial. On the other are the AI powerhouse coding assistants, GitHub Copilot and OpenAI ChatGPT. Here's how the contest works: David invents 24 Regex problems he calls puzzles and shows you how to tackle each one. When he's done, he has Copilot and ChatGPT work the same puzzles. What they produce intrigues him. Which side is likelier to get it right? Which will write simple and elegant code? Which makes smarter use of lesser known Regex library features? Read the book to find out. David also offers AI best practices, showing how smart prompts return better results. By the end, you'll be a master at solving your own Regex puzzles, whether you use AI or not. About the technology Groundbreaking large language model research from OpenAI, Google, Amazon, and others have transformed expectations of machine-generated software. But how do these AI assistants, like ChatGPT and GitHub Copilot, measure up against regular expressions—a workhorse technology for developers used to describe, find, and manipulate patterns in text. Regular expressions are compact, complex, and subtle. Will AI assistants handle the challenge? About the book Regular Expression Puzzles and AI Coding Assistants is the perfect starting point for programmers of any experience level who want to understand the capabilities—and the limitations—of these exciting new tools. Author David Mertz presents 24 challenging regex puzzles, their traditional human-made solutions, and the fascinating answers given by popular AI assistants. Alongside these eye-opening puzzles you’ll learn how to write prompts, integrate AI-generated coding suggestions, and interact with the assistant to get the results you want. By the end of the book, you'll have a clear understanding of where AI assistants can reliably write code for you and where you’ll still need a human touch. Plus, you’ll learn a lot about regular expressions! About the reader Code examples use simple Python and Regular Expressions. No experience with AI coding tools required. About the author David Mertz is the founder of KDM Training and an acclaimed contributor to the Python community. He is also the author of The Puzzling Quirks of Regular Expressions, Cleaning Data for Effective Data Science: Doing the Other 80% of the Work, and other books. Table of Contents 1 The map and the territory 2 Quantifiers and special sub-patterns 3 Pitfalls and sand in the gears 4 Creating functions using regex 5 Easy, difficult, and impossible tasks 6 Conclusions Appendix A: Learning to use regular expressions

Think about your data intelligently and ask the right questionsKey FeaturesMaster data cleaning techniques necessary to perform real-world data science and machine learning tasksSpot common problems with dirty data and develop flexible solutions from first principlesTest and refine your newly acquired skills through detailed exercises at the end of each chapterBook DescriptionData cleaning is the all-important first step to successful data science, data analysis, and machine learning. If you work with any kind of data, this book is your go-to resource, arming you with the insights and heuristics experienced data scientists had to learn the hard way.In a light-hearted and engaging exploration of different tools, techniques, and datasets real and fictitious, Python veteran David Mertz teaches you the ins and outs of data preparation and the essential questions you should be asking of every piece of data you work with.Using a mixture of Python, R, and common command-line tools, Cleaning Data for Effective Data Science follows the data cleaning pipeline from start to end, focusing on helping you understand the principles underlying each step of the process. You'll look at data ingestion of a vast range of tabular, hierarchical, and other data formats, impute missing values, detect unreliable data and statistical anomalies, and generate synthetic features. The long-form exercises at the end of each chapter let you get hands-on with the skills you've acquired along the way, also providing a valuable resource for academic courses.What you will learnIngest and work with common data formats like JSON, CSV, SQL and NoSQL databases, PDF, and binary serialized data structuresUnderstand how and why we use tools such as pandas, SciPy, scikit-learn, Tidyverse, and BashApply useful rules and heuristics for assessing data quality and detecting bias, like Benford's law and the 68-95-99.7 ruleIdentify and handle unreliable data and outliers, examining z-score and other statistical propertiesImpute sensible values into missing data and use sampling to fix imbalancesUse dimensionality reduction, quantization, one-hot encoding, and other feature engineering techniques to draw out patterns in your dataWork carefully with time series data, performing de-trending and interpolationWho this book is forThis book is designed to benefit software developers, data scientists, aspiring data scientists, teachers, and students who work with data. If you want to improve your rigor in data hygiene or are looking for a refresher, this book is for you.Basic familiarity with statistics, general concepts in machine learning, knowledge of a programming language (Python or R), and some exposure to data science are helpful.