Book of Alternative Data

ALEXANDER DENEV is Head of AI, Financial Services - Risk Advisory at Deloitte LLP. Prior to that he led Quantitative Research & Advanced Analytics at IHS Markit. Previously, he held roles at the Royal Bank of Scotland, Societe Generale, and European Investment Bank. Denev is a visiting lecturer at the University of Oxford where he graduated with a degree in Mathematical Finance. He is author of numerous papers and books on novel methods of financial modeling with applications ranging from stress testing to asset allocation. SAEED AMEN is the founder of Cuemacro, where he consults on systematic trading. For 15 years, he has developed systematic trading strategies and quantitative indices including at major investment banks, Lehman Brothers and Nomura. He is also a visiting lecturer at Queen Mary University of London and a co-founder of the Thalesians, a quant think tank.

• Preface xvAcknowledgments xviiPart 1 Introduction and Theory 11 Alternative Data: The Lay of the Land 31.1 Introduction 31.2 What is “Alternative Data”? 51.3 Segmentation of Alternative Data 71.4 The Many Vs of Big Data 91.5 Why Alternative Data? 111.6 Who is Using Alternative Data? 151.7 Capacity of a Strategy and Alternative Data 161.8 Alternative Data Dimensions 191.9 Who Are the Alternative Data Vendors? 231.10 Usage of Alternative Datasets on the Buy Side 241.11 Conclusion 262 The Value of Alternative Data 272.1 Introduction 272.2 The Decay of Investment Value 272.3 Data Markets 292.4 The Monetary Value of Data (Part I) 312.4.1 Cost Value 342.4.2 Market Value 342.4.3 Economic Value 352.5 Evaluating (Alternative) Data Strategies with and without Backtesting 352.5.1 Systematic Investors 362.5.2 Discretionary Investors 382.5.3 Risk Managers 392.6 The Monetary Value of Data (Part II) 392.6.1 The Buyer’s Perspective 402.6.2 The Seller’s Perspective 412.7 The Advantages of Maturing Alternative Datasets 452.8 Summary 463 Alternative Data Risks and Challenges 473.1 Legal Aspects of Data 473.2 Risks of Using Alternative Data 503.3 Challenges of Using Alternative Data 513.3.1 Entity Matching 523.3.2 Missing Data 543.3.3 Structuring the Data 553.3.4 Treatment of Outliers 563.4 Aggregating the Data 573.5 Summary 584 Machine Learning Techniques 594.1 Introduction 594.2 Machine Learning: Definitions and Techniques 604.2.1 Bias, Variance, and Noise 604.2.2 Cross-Validation 614.2.3 Introducing Machine Learning 624.2.4 Popular Supervised Machine Learning Techniques 644.2.5 Clustering-Based Unsupervised Machine Learning Techniques 704.2.6 Other Unsupervised Machine Learning Techniques 714.2.7 Machine Learning Libraries 714.2.8 Neutral Networks and Deep Learning 724.2.9 Gaussian Processes 804.3 Which Technique to Choose? 824.4 Assumptions and Limitations of the Machine Learning Techniques 844.4.1 Causality 844.4.2 Non-stationarity 854.4.3 Restricted Information Set 864.4.4 The Algorithm Choice 864.5 Structuring Images 874.5.1 Features and Feature Detection Algorithms 874.5.2 Deep Learning and CNNs for Image Classification 894.5.3 Augmenting Satellite Image Data with Other Datasets 904.5.4 Imaging Tools 914.6 Natural Language Processing (NLP) 914.6.1 What is Natural Language Processing (NLP)? 914.6.2 Normalization 934.6.3 Creating Word Embeddings: Bag-of-Words 944.6.4 Creating Word Embeddings: Word2vec and Beyond 944.6.5 Sentiment Analysis and NLP Tasks as Classification Problems 964.6.6 Topic Modeling 964.6.7 Various Challenges in NLP 974.6.8 Different Languages and Different Texts 984.6.9 Speech in NLP 994.6.10 NLP Tools 1004.7 Summary 1025 The Processes behind the Use of Alternative Data 1055.1 Introduction 1055.2 Steps in the Alternative Data Journey 1065.2.1 Step 1. Set up a Vision and Strategy 1065.2.2 Step 2. Identify the Appropriate Datasets 1075.2.3 Step 3. Perform Due Diligence on Vendors 1085.2.4 Step 4. Pre-assess Risks 1095.2.5 Step 5. Pre-assess the Existence of Signals 1095.2.6 Step 6. Data Onboarding 1105.2.7 Step 7. Data Preprocessing 1105.2.8 Step 8. Signal Extraction 1115.2.9 Step 9. Implementation (or Deployment in Production) 1125.2.10 Maintenance Process 1135.3 Structuring Teams to Use Alternative Data 1145.4 Data Vendors 1165.5 Summary 1186 Factor Investing 1196.1 Introduction 1196.1.1 The CAPM 1196.2 Factor Models 1206.2.1 The Arbitrage Pricing Theory 1226.2.2 The Fama-French 3-Factor Model 1236.2.3 The Carhart Model 1246.2.4 Other Approaches (Data Mining) 1256.3 The Difference between Cross-Sectional and Time Series Trading Approaches 1266.4 Why Factor Investing? 1266.5 Smart Beta Indices Using Alternative Data Inputs 1276.6 ESG Factors 1286.7 Direct and Indirect Prediction 1296.8 Summary 132Part 2 Practical Applications 1337 Missing Data: Background 1357.1 Introduction 1357.2 Missing Data Classification 1367.2.1 Missing Data Treatments 1377.3 Literature Overview of Missing Data Treatments 1397.3.1 Luengo et al. (2012) 1397.3.2 Garcia-Laencina et al. (2010) 1437.3.3 Grzymala-Busse et al. (2000) 1467.3.4 Zou et al. (2005) 1477.3.5 Jerez et al. (2010) 1477.3.6 Farhangfar et al. (2008) 1487.3.7 Kang et al. (2013) 1497.4 Summary 1498 Missing Data: Case Studies 1518.1 Introduction 1518.2 Case Study: Imputing Missing Values in Multivariate Credit Default Swap Time Series 1528.2.1 Missing Data Classification 1538.2.2 Imputation Metrics 1548.2.3 CDS Data and Test Data Generation 1548.2.4 Multiple Imputation Methods 1578.2.5 Deterministic and EOF-Based Techniques 1608.2.6 Results 1648.3 Case Study: Satellite Images 1738.4 Summary 1768.5 Appendix: General Description of the MICE Procedure 1788.6 Appendix: Software Libraries Used in This Chapter 1799 Outliers (Anomalies) 1819.1 Introduction 1819.2 Outliers Definition, Classification, and Approaches to Detection 1829.3 Temporal Structure 1839.4 Global Versus Local Outliers, Point Anomalies, and Micro-Clusters 1849.5 Outlier Detection Problem Setup 1849.6 Comparative Evaluation of Outlier Detection Algorithms 1859.7 Approaches to Outlier Explanation 1899.7.1 Micenkova et al. 1899.7.2 Duan et al. 1919.7.3 Angiulli et al. 1929.8 Case Study: Outlier Detection on Fed Communications Index 1949.9 Summary 2019.10 Appendix 2029.10.1 Model-Based Techniques 2029.10.2 Distance-Based Techniques 2029.10.3 Density-Based Techniques 2039.10.4 Heuristics-Based Approaches 20310 Automotive Fundamental Data 20510.1 Introduction 20510.2 Data 20610.3 Approach 1: Indirect Approach 21110.3.1 The Steps Followed 21210.3.2 Stage 1 21310.4 Approach 2: Direct Approach 22310.4.1 The Data 22310.4.2 Factor Generation 22410.4.3 Factor Performance 22510.4.4 Detailed Factor Results 22910.5 Gaussian Processes Example 23810.6 Summary 23910.7 Appendix 24010.7.1 List of Companies 24010.7.2 Description of Financial Statement Items 24110.7.3 Ratios Used 24210.7.4 IHS Markit Data Features 24310.7.5 Reporting Delays by Country 24411 Surveys and Crowdsourced Data 24511.1 Introduction 24511.2 Survey Data as Alternative Data 24511.3 The Data 24711.4 The Product 24711.5 Case Studies 24911.5.1 Case Study: Company Event Study (Pooled Survey) 24911.5.2 Case Study: Oil and Gas Production (Q&A Survey) 25211.6 Some Technical Considerations on Surveys 25411.7 Crowdsourcing Analyst Estimates Survey 25511.8 Alpha Capture Data 25611.9 Summary 25611.10 Appendix 25612 Purchasing Managers’ Index 25912.1 Introduction 25912.2 PMI Performance 26112.3 Nowcasting GDP Growth 26212.4 Impacts on Financial Markets 26312.5 Summary 26613 Satellite Imagery and Aerial Photography 26713.1 Introduction 26713.2 Forecasting US Export Growth 26913.3 Car Counts and Earnings Per Share for Retailers 27113.4 Measuring Chinese PMI Manufacturing with Satellite Data 27713.5 Summary 28014 Location Data 28314.1 Introduction 28314.2 Shipping Data to Track Crude Oil Supplies 28314.3 Mobile Phone Location Data to Understand Retail Activity 28714.3.1 Trading REIT ETF Using Mobile Phone Location Data 28814.3.2 Estimating Earnings per Share with Mobile Phone Location Data 29114.4 Taxi Ride Data and New York Fed Meetings 29514.5 Corporate Jet Location Data and M&A 29614.6 Summary 29815 Text Web Social Media and News 29915.1 Introduction 29915.2 Collecting Web Data 29915.3 Social Media 30015.3.1 Hedonometer Index 30215.3.2 Using Twitter Data to Help Forecast US Change in Nonfarm Payrolls 30515.3.3 Twitter Data to Forecast Stock Market Reaction to FOMC 30815.3.4 Liquidity and Sentiment from Social Media 30915.4 News 30915.4.1 Machine-Readable News to Trade FX and Understand FX Volatility 31015.4.2 Federal Reserve Communications and US Treasury Yields 31615.5 Other Web Sources 32015.5.1 Measuring Consumer Price Inflation 32115.6 Summary 32216 Investor Attention 32316.1 Introduction 32316.2 Readership of Payrolls to Measure Investor Attention 32316.3 Google Trends Data to Measure Market Themes 32516.4 Investopedia Search Data to Measure Investor Anxiety 32816.5 Using Wikipedia to Understand Price Action in Cryptocurrencies 33016.6 Online Attention for Countries to Inform EMFX Trading 33016.7 Summary 33317 Consumer Transactions 33517.1 Introduction 33517.2 Credit and Debit Card Transaction Data 33617.3 Consumer Receipts 33717.4 Summary 34018 Government, Industrial, and Corporate Data 34118.1 Introduction 34118.2 Using Innovation Measures to Trade Equities 34218.3 Quantifying Currency Crisis Risk 34418.4 Modeling Central Bank Intervention in Currency Markets 34618.5 Summary 34819 Market Data 35119.1 Introduction 35119.2 Relationship between Institutional FX Flow Data and FX Spot 35119.3 Understanding Liquidity Using High-Frequency FX Data 35519.4 Summary 35720 Alternative Data in Private Markets 35920.1 Introduction 35920.2 Defining Private Equity and Venture Capital Firms 36020.3 Private Equity Datasets 36220.4 Understanding the Performance of Private Firms 36320.5 Summary 364Conclusions 365Some Last Words 365References 367About the Authors 373Index 375