Feature Engineering and Selection

Max Kuhn, Ph.D., is a software engineer at RStudio. He worked in 18 years in drug discovery and medical diagnostics applying predictive models to real data. He has authored numerous R packages for predictive modeling and machine learning. Kjell Johnson, Ph.D., is the owner and founder of Stat Tenacity, a firm that provides statistical and predictive modeling consulting services. He has taught short courses on predictive modeling for the American Society for Quality, American Chemical Society, International Biometric Society, and for many corporations.Kuhn and Johnson have also authored Applied Predictive Modeling, which is a comprehensive, practical guide to the process of building a predictive model. The text won the 2014 Technometrics Ziegel Prize for Outstanding Book.

"The book is timely and needed. The interest in all things 'data science' morphed into everybody pretending to do, or know, Machine Learning. Kuhn and Johnson happen to actually know this—as evidenced by their earlier and still-popular tome entitled ‘Applied Predictive Modeling.’ The proposed ‘Feature Engineering and Selection’ builds on this and extends it. I expect it to become as popular with a wide reach as both a textbook, self-study material, and reference."~Dirk Eddelbuettel, University of Illinois at Urbana-Champaign"As a reviewer, it has been exciting and edifying to see this book develop into what is likely to become one of the foundational works on feature engineering. It is launching propitiously on the current tide of interest in both interpretable models and AutoML."~Robert Horton, Microsoft"In recent years, the statistics literature has featured new developments in modeling and predictive analytics. Approaches such as cross-validation and statistical/machine learning techniques have become widespread. The author's previous book ("Applied Predictive Modeling", APM) provided a wide-ranging introduction and integration of these methods and suggested a workflow in R to carry out exploratory and confirmation analyses. With this project, the authors have identified an important and interesting component of these methods that describes building better models by focusing on the predictors (feature engineering)…The authors focus on the variables that go into the model (and how they are represented) and argue that such issues are as important (or more important) than the particular methods that are applied to an analysis...The proposed book is likely to serve as a textbook (for a number of undergraduate and graduate courses in a variety of disciplines) and reference (for a large number of statisticians seeking principled and well-organized modeling)."~Nicholas Horton, Amherst College"I think this book is great and a joy to read…I like the pragmatic and practical approach taken in the book, and the examples given are very illustrative. The emphasis on how and when to use resampling is refreshing and something that the community needs to hear more." ~Andreas C. Muller, Columbia University

• 1. Introduction. 2. Illustrative Example: Predicting Risk of Ischemic Stroke. 3. A Review of the Predictive Modeling Process. 4. Exploratory Visualizations. 5. Encoding Categorical Predictors. 6. Engineering Numeric Predictors. 7. Detecting Interaction Effects. 8. Handling Missing Data. 9. Working with Profile Data. 10. Feature Selection Overview. 11. Greedy Search Methods. 12. Global Search Methods.