Data Mining the Web

Zdravko Markov, PhD, is Associate Professor of Computer Science at Central Connecticut State University. The author of three textbooks, Dr. Markov teaches undergraduate and graduate courses in computer science and artificial intelligence. He is currently a Principal Investigator (PI) in a National Science Foundation–funded project designed to introduce machine learning to undergraduates. Daniel T. Larose, PhD, is Professor of Statistics in the Department of Mathematical Sciences at Central Connecticut State University. He is the author of three data mining books and a forthcoming textbook in undergraduate statistics. He developed and directs CCSU's DataMining@CCSU programs.

"…it has to be noted that this book is an excellent resource for conducting Web mining lectures or single units within Data mining class. The data can be used for small as well as quite comprehensive business intelligence projects. The book's content is easy to access; even students with very basic statistical skills can get the flavor of the intriguing aspects of Web mining." (Journal of Statistical Software, April 2008) "…highlight[s] the exciting research related to data mining the Web…a detailed summary of the current state of the art." (CHOICE, December 2007)"I can say I really enjoyed reading this book…a great educational resource for students and teachers." (Information Retrieval, 2008)

• PREFACE. PART I: WEB STRUCTURE MINING.1 INFORMATION RETRIEVAL AND WEB SEARCH.Web Challenges.Web Search Engines.Topic Directories.Semantic Web.Crawling the Web.Web Basics.Web Crawlers.Indexing and Keyword Search.Document Representation.Implementation Considerations.Relevance Ranking.Advanced Text Search.Using the HTML Structure in Keyword Search.Evaluating Search Quality.Similarity Search.Cosine Similarity.Jaccard Similarity.Document Resemblance.References.Exercises.2 HYPERLINK-BASED RANKING.Introduction.Social Networks Analysis.PageRank.Authorities and Hubs.Link-Based Similarity Search.Enhanced Techniques for Page Ranking.References.Exercises.PART II: WEB CONTENT MINING.3 CLUSTERING.Introduction.Hierarchical Agglomerative Clustering.k-Means Clustering.Probabilty-Based Clustering.Finite Mixture Problem.Classification Problem.Clustering Problem.Collaborative Filtering (Recommender Systems).References.Exercises.4 EVALUATING CLUSTERING.Approaches to Evaluating Clustering.Similarity-Based Criterion Functions.Probabilistic Criterion Functions.MDL-Based Model and Feature Evaluation.Minimum Description Length Principle.MDL-Based Model Evaluation.Feature Selection.Classes-to-Clusters Evaluation.Precision, Recall, and F-Measure.Entropy.References.Exercises.5 CLASSIFICATION.General Setting and Evaluation Techniques.Nearest-Neighbor Algorithm.Feature Selection.Naive Bayes Algorithm.Numerical Approaches.Relational Learning.References.Exercises.PART III: WEB USAGE MINING.6 INTRODUCTION TO WEB USAGE MINING.Definition of Web Usage Mining.Cross-Industry Standard Process for Data Mining.Clickstream Analysis.Web Server Log Files.Remote Host Field.Date/Time Field.HTTP Request Field.Status Code Field.Transfer Volume (Bytes) Field.Common Log Format.Identification Field.Authuser Field.Extended Common Log Format.Referrer Field.User Agent Field.Example of a Web Log Record.Microsoft IIS Log Format.Auxiliary Information.References.Exercises.7 PREPROCESSING FOR WEB USAGE MINING.Need for Preprocessing the Data.Data Cleaning and Filtering.Page Extension Exploration and Filtering.De-Spidering the Web Log File.User Identification.Session Identification.Path Completion.Directories and the Basket Transformation.Further Data Preprocessing Steps.References.Exercises.8 EXPLORATORY DATA ANALYSIS FOR WEB USAGE MINING.Introduction.Number of Visit Actions.Session Duration.Relationship between Visit Actions and Session Duration.Average Time per Page.Duration for Individual Pages.References.Exercises.9 MODELING FOR WEB USAGE MINING: CLUSTERING, ASSOCIATION, AND CLASSIFICATION.Introduction.Modeling Methodology.Definition of Clustering.The BIRCH Clustering Algorithm.Affinity Analysis and the A Priori Algorithm.Discretizing the Numerical Variables: Binning.Applying the A Priori Algorithm to the CCSU Web Log Data.Classification and Regression Trees.The C4.5 Algorithm.References.Exercises.INDEX.