Michael W. Berry, Professor and Associate Department Head, Department of Electrical Engineering and Computer Science, University of Tennessee.Michael is on the Editorial board of Computing in Science and Engineering and Statistical Analysis and Data Mining Journals. Jacob Kogan, Department of Mathematics and Statistics, University of Maryland Baltimore County, USA.

"It is extremely useful for practitioners and students in computer science, natural language processing, bioinformatics and engineering who wish to use text mining techniques." (Journal of Information Retrieval, 1 April 2011)

• List of Contributors xiPreface xiiiPart I Text Extraction, Classification, and Clustering 11 Automatic keyword extraction from individual documents 31.1 Introduction 31.1.1 Keyword extraction methods 41.2 Rapid automatic keyword extraction 51.2.1 Candidate keywords 61.2.2 Keyword scores 71.2.3 Adjoining keywords 81.2.4 Extracted keywords 81.3 Benchmark evaluation 91.3.1 Evaluating precision and recall 91.3.2 Evaluating efficiency 101.4 Stoplist generation 111.5 Evaluation on news articles 151.5.1 The MPQA Corpus 151.5.2 Extracting keywords from news articles 151.6 Summary 181.7 Acknowledgements 19References 192 Algebraic techniques for multilingual document clustering 212.1 Introduction 212.2 Background 222.3 Experimental setup 232.4 Multilingual LSA 252.5 Tucker1 method 272.6 PARAFAC2 method 282.7 LSA with term alignments 292.8 Latent morpho-semantic analysis (LMSA) 322.9 LMSA with term alignments 332.10 Discussion of results and techniques 332.11 Acknowledgements 35References 353 Content-based spam email classification using machine-learning algorithms 373.1 Introduction 373.2 Machine-learning algorithms 393.2.1 Naive Bayes 393.2.2 LogitBoost 403.2.3 Support vector machines 413.2.4 Augmented latent semantic indexing spaces 433.2.5 Radial basis function networks 443.3 Data preprocessing 453.3.1 Feature selection 453.3.2 Message representation 473.4 Evaluation of email classification 483.5 Experiments 493.5.1 Experiments with PU 1 493.5.2 Experiments with ZH 1 513.6 Characteristics of classifiers 533.7 Concluding remarks 543.8 Acknowledgements 55References 554 Utilizing nonnegative matrix factorization for email classification problems 574.1 Introduction 574.1.1 Related work 594.1.2 Synopsis 604.2 Background 604.2.1 Nonnegative matrix factorization 604.2.2 Algorithms for computing NMF 614.2.3 Datasets 634.2.4 Interpretation 644.3 NMF initialization based on feature ranking 654.3.1 Feature subset selection 664.3.2 FS initialization 664.4 NMF-based classification methods 704.4.1 Classification using basis features 704.4.2 Generalizing LSI based on NMF 724.5 Conclusions 784.6 Acknowledgements 79References 795 Constrained clustering with k-means type algorithms 815.1 Introduction 815.2 Notations and classical k-means 825.3 Constrained k-means with Bregman divergences 845.3.1 Quadratic k-means with cannot-link constraints 845.3.2 Elimination of must-link constraints 875.3.3 Clustering with Bregman divergences 895.4 Constrained smoka type clustering 925.5 Constrained spherical k-means 955.5.1 Spherical k-means with cannot-link constraints only 965.5.2 Spherical k-means with cannot-link and must-link constraints 985.6 Numerical experiments 995.6.1 Quadratic k-means 1005.6.2 Spherical k-means 1005.7 Conclusion 101References 102Part II Anomaly and Trend Detection 1056 Survey of text visualization techniques 1076.1 Visualization in text analysis 1076.2 Tag clouds 1086.3 Authorship and change tracking 1106.4 Data exploration and the search for novel patterns 1116.5 Sentiment tracking 1116.6 Visual analytics and FutureLens 1136.7 Scenario discovery 1146.7.1 Scenarios 1156.7.2 Evaluating solutions 1156.8 Earlier prototype 1166.9 Features of FutureLens 1176.10 Scenario discovery example: bioterrorism 1196.11 Scenario discovery example: drug trafficking 1216.12 Future work 123References 1267 Adaptive threshold setting for novelty mining 1297.1 Introduction 1297.2 Adaptive threshold setting in novelty mining 1317.2.1 Background 1317.2.2 Motivation 1327.2.3 Gaussian-based adaptive threshold setting 1327.2.4 Implementation issues 1377.3 Experimental study 1387.3.1 Datasets 1387.3.2 Working example 1397.3.3 Experiments and results 1427.4 Conclusion 146References 1478 Text mining and cybercrime 1498.1 Introduction 1498.2 Current research in Internet predation and cyberbullying 1518.2.1 Capturing IM and IRC chat 1518.2.2 Current collections for use in analysis 1528.2.3 Analysis of IM and IRC chat 1538.2.4 Internet predation detection 1538.2.5 Cyberbullying detection 1588.2.6 Legal issues 1598.3 Commercial software for monitoring chat 1598.4 Conclusions and future directions 1618.5 Acknowledgements 162References 162Part III Text Streams 1659 Events and trends in text streams 1679.1 Introduction 1679.2 Text streams 1699.3 Feature extraction and data reduction 1709.4 Event detection 1719.5 Trend detection 1749.6 Event and trend descriptions 1769.7 Discussion 1809.8 Summary 1819.9 Acknowledgements 181References 18110 Embedding semantics in LDA topic models 18310.1 Introduction 18310.2 Background 18410.2.1 Vector space modeling 18410.2.2 Latent semantic analysis 18510.2.3 Probabilistic latent semantic analysis 18510.3 Latent Dirichlet allocation 18610.3.1 Graphical model and generative process 18710.3.2 Posterior inference 18710.3.3 Online latent Dirichlet allocation (OLDA) 18910.3.4 Illustrative example 19110.4 Embedding external semantics from Wikipedia 19310.4.1 Related Wikipedia articles 19410.4.2 Wikipedia-influenced topic model 19410.5 Data-driven semantic embedding 19410.5.1 Generative process with data-driven semantic embedding 19510.5.2 OLDA algorithm with data-driven semantic embedding 19610.5.3 Experimental design 19710.5.4 Experimental results 19910.6 Related work 20210.7 Conclusion and future work 202References 203Index 205

Michael W. Berry, Professor and Associate Department Head, Department of Electrical Engineering and Computer Science, University of Tennessee.Michael is on the Editorial board of Computing in Science and Engineering and Statistical Analysis and Data Mining Journals. Jacob Kogan, Department of Mathematics and Statistics, University of Maryland Baltimore County, USA.

"It is extremely useful for practitioners and students in computer science, natural language processing, bioinformatics and engineering who wish to use text mining techniques." (Journal of Information Retrieval, 1 April 2011)

• List of Contributors xiPreface xiiiPart I Text Extraction, Classification, and Clustering 11 Automatic keyword extraction from individual documents 31.1 Introduction 31.1.1 Keyword extraction methods 41.2 Rapid automatic keyword extraction 51.2.1 Candidate keywords 61.2.2 Keyword scores 71.2.3 Adjoining keywords 81.2.4 Extracted keywords 81.3 Benchmark evaluation 91.3.1 Evaluating precision and recall 91.3.2 Evaluating efficiency 101.4 Stoplist generation 111.5 Evaluation on news articles 151.5.1 The MPQA Corpus 151.5.2 Extracting keywords from news articles 151.6 Summary 181.7 Acknowledgements 19References 192 Algebraic techniques for multilingual document clustering 212.1 Introduction 212.2 Background 222.3 Experimental setup 232.4 Multilingual LSA 252.5 Tucker1 method 272.6 PARAFAC2 method 282.7 LSA with term alignments 292.8 Latent morpho-semantic analysis (LMSA) 322.9 LMSA with term alignments 332.10 Discussion of results and techniques 332.11 Acknowledgements 35References 353 Content-based spam email classification using machine-learning algorithms 373.1 Introduction 373.2 Machine-learning algorithms 393.2.1 Naive Bayes 393.2.2 LogitBoost 403.2.3 Support vector machines 413.2.4 Augmented latent semantic indexing spaces 433.2.5 Radial basis function networks 443.3 Data preprocessing 453.3.1 Feature selection 453.3.2 Message representation 473.4 Evaluation of email classification 483.5 Experiments 493.5.1 Experiments with PU 1 493.5.2 Experiments with ZH 1 513.6 Characteristics of classifiers 533.7 Concluding remarks 543.8 Acknowledgements 55References 554 Utilizing nonnegative matrix factorization for email classification problems 574.1 Introduction 574.1.1 Related work 594.1.2 Synopsis 604.2 Background 604.2.1 Nonnegative matrix factorization 604.2.2 Algorithms for computing NMF 614.2.3 Datasets 634.2.4 Interpretation 644.3 NMF initialization based on feature ranking 654.3.1 Feature subset selection 664.3.2 FS initialization 664.4 NMF-based classification methods 704.4.1 Classification using basis features 704.4.2 Generalizing LSI based on NMF 724.5 Conclusions 784.6 Acknowledgements 79References 795 Constrained clustering with k-means type algorithms 815.1 Introduction 815.2 Notations and classical k-means 825.3 Constrained k-means with Bregman divergences 845.3.1 Quadratic k-means with cannot-link constraints 845.3.2 Elimination of must-link constraints 875.3.3 Clustering with Bregman divergences 895.4 Constrained smoka type clustering 925.5 Constrained spherical k-means 955.5.1 Spherical k-means with cannot-link constraints only 965.5.2 Spherical k-means with cannot-link and must-link constraints 985.6 Numerical experiments 995.6.1 Quadratic k-means 1005.6.2 Spherical k-means 1005.7 Conclusion 101References 102Part II Anomaly and Trend Detection 1056 Survey of text visualization techniques 1076.1 Visualization in text analysis 1076.2 Tag clouds 1086.3 Authorship and change tracking 1106.4 Data exploration and the search for novel patterns 1116.5 Sentiment tracking 1116.6 Visual analytics and FutureLens 1136.7 Scenario discovery 1146.7.1 Scenarios 1156.7.2 Evaluating solutions 1156.8 Earlier prototype 1166.9 Features of FutureLens 1176.10 Scenario discovery example: bioterrorism 1196.11 Scenario discovery example: drug trafficking 1216.12 Future work 123References 1267 Adaptive threshold setting for novelty mining 1297.1 Introduction 1297.2 Adaptive threshold setting in novelty mining 1317.2.1 Background 1317.2.2 Motivation 1327.2.3 Gaussian-based adaptive threshold setting 1327.2.4 Implementation issues 1377.3 Experimental study 1387.3.1 Datasets 1387.3.2 Working example 1397.3.3 Experiments and results 1427.4 Conclusion 146References 1478 Text mining and cybercrime 1498.1 Introduction 1498.2 Current research in Internet predation and cyberbullying 1518.2.1 Capturing IM and IRC chat 1518.2.2 Current collections for use in analysis 1528.2.3 Analysis of IM and IRC chat 1538.2.4 Internet predation detection 1538.2.5 Cyberbullying detection 1588.2.6 Legal issues 1598.3 Commercial software for monitoring chat 1598.4 Conclusions and future directions 1618.5 Acknowledgements 162References 162Part III Text Streams 1659 Events and trends in text streams 1679.1 Introduction 1679.2 Text streams 1699.3 Feature extraction and data reduction 1709.4 Event detection 1719.5 Trend detection 1749.6 Event and trend descriptions 1769.7 Discussion 1809.8 Summary 1819.9 Acknowledgements 181References 18110 Embedding semantics in LDA topic models 18310.1 Introduction 18310.2 Background 18410.2.1 Vector space modeling 18410.2.2 Latent semantic analysis 18510.2.3 Probabilistic latent semantic analysis 18510.3 Latent Dirichlet allocation 18610.3.1 Graphical model and generative process 18710.3.2 Posterior inference 18710.3.3 Online latent Dirichlet allocation (OLDA) 18910.3.4 Illustrative example 19110.4 Embedding external semantics from Wikipedia 19310.4.1 Related Wikipedia articles 19410.4.2 Wikipedia-influenced topic model 19410.5 Data-driven semantic embedding 19410.5.1 Generative process with data-driven semantic embedding 19510.5.2 OLDA algorithm with data-driven semantic embedding 19610.5.3 Experimental design 19710.5.4 Experimental results 19910.6 Related work 20210.7 Conclusion and future work 202References 203Index 205

Michael W. Berry, Professor and Associate Department Head, Department of Electrical Engineering and Computer Science, University of Tennessee.Michael is on the Editorial board of Computing in Science and Engineering and Statistical Analysis and Data Mining Journals. Jacob Kogan, Department of Mathematics and Statistics, University of Maryland Baltimore County, USA.

"It is extremely useful for practitioners and students in computer science, natural language processing, bioinformatics and engineering who wish to use text mining techniques." (Journal of Information Retrieval, 1 April 2011)

• List of Contributors xiPreface xiiiPart I Text Extraction, Classification, and Clustering 11 Automatic keyword extraction from individual documents 31.1 Introduction 31.1.1 Keyword extraction methods 41.2 Rapid automatic keyword extraction 51.2.1 Candidate keywords 61.2.2 Keyword scores 71.2.3 Adjoining keywords 81.2.4 Extracted keywords 81.3 Benchmark evaluation 91.3.1 Evaluating precision and recall 91.3.2 Evaluating efficiency 101.4 Stoplist generation 111.5 Evaluation on news articles 151.5.1 The MPQA Corpus 151.5.2 Extracting keywords from news articles 151.6 Summary 181.7 Acknowledgements 19References 192 Algebraic techniques for multilingual document clustering 212.1 Introduction 212.2 Background 222.3 Experimental setup 232.4 Multilingual LSA 252.5 Tucker1 method 272.6 PARAFAC2 method 282.7 LSA with term alignments 292.8 Latent morpho-semantic analysis (LMSA) 322.9 LMSA with term alignments 332.10 Discussion of results and techniques 332.11 Acknowledgements 35References 353 Content-based spam email classification using machine-learning algorithms 373.1 Introduction 373.2 Machine-learning algorithms 393.2.1 Naive Bayes 393.2.2 LogitBoost 403.2.3 Support vector machines 413.2.4 Augmented latent semantic indexing spaces 433.2.5 Radial basis function networks 443.3 Data preprocessing 453.3.1 Feature selection 453.3.2 Message representation 473.4 Evaluation of email classification 483.5 Experiments 493.5.1 Experiments with PU 1 493.5.2 Experiments with ZH 1 513.6 Characteristics of classifiers 533.7 Concluding remarks 543.8 Acknowledgements 55References 554 Utilizing nonnegative matrix factorization for email classification problems 574.1 Introduction 574.1.1 Related work 594.1.2 Synopsis 604.2 Background 604.2.1 Nonnegative matrix factorization 604.2.2 Algorithms for computing NMF 614.2.3 Datasets 634.2.4 Interpretation 644.3 NMF initialization based on feature ranking 654.3.1 Feature subset selection 664.3.2 FS initialization 664.4 NMF-based classification methods 704.4.1 Classification using basis features 704.4.2 Generalizing LSI based on NMF 724.5 Conclusions 784.6 Acknowledgements 79References 795 Constrained clustering with k-means type algorithms 815.1 Introduction 815.2 Notations and classical k-means 825.3 Constrained k-means with Bregman divergences 845.3.1 Quadratic k-means with cannot-link constraints 845.3.2 Elimination of must-link constraints 875.3.3 Clustering with Bregman divergences 895.4 Constrained smoka type clustering 925.5 Constrained spherical k-means 955.5.1 Spherical k-means with cannot-link constraints only 965.5.2 Spherical k-means with cannot-link and must-link constraints 985.6 Numerical experiments 995.6.1 Quadratic k-means 1005.6.2 Spherical k-means 1005.7 Conclusion 101References 102Part II Anomaly and Trend Detection 1056 Survey of text visualization techniques 1076.1 Visualization in text analysis 1076.2 Tag clouds 1086.3 Authorship and change tracking 1106.4 Data exploration and the search for novel patterns 1116.5 Sentiment tracking 1116.6 Visual analytics and FutureLens 1136.7 Scenario discovery 1146.7.1 Scenarios 1156.7.2 Evaluating solutions 1156.8 Earlier prototype 1166.9 Features of FutureLens 1176.10 Scenario discovery example: bioterrorism 1196.11 Scenario discovery example: drug trafficking 1216.12 Future work 123References 1267 Adaptive threshold setting for novelty mining 1297.1 Introduction 1297.2 Adaptive threshold setting in novelty mining 1317.2.1 Background 1317.2.2 Motivation 1327.2.3 Gaussian-based adaptive threshold setting 1327.2.4 Implementation issues 1377.3 Experimental study 1387.3.1 Datasets 1387.3.2 Working example 1397.3.3 Experiments and results 1427.4 Conclusion 146References 1478 Text mining and cybercrime 1498.1 Introduction 1498.2 Current research in Internet predation and cyberbullying 1518.2.1 Capturing IM and IRC chat 1518.2.2 Current collections for use in analysis 1528.2.3 Analysis of IM and IRC chat 1538.2.4 Internet predation detection 1538.2.5 Cyberbullying detection 1588.2.6 Legal issues 1598.3 Commercial software for monitoring chat 1598.4 Conclusions and future directions 1618.5 Acknowledgements 162References 162Part III Text Streams 1659 Events and trends in text streams 1679.1 Introduction 1679.2 Text streams 1699.3 Feature extraction and data reduction 1709.4 Event detection 1719.5 Trend detection 1749.6 Event and trend descriptions 1769.7 Discussion 1809.8 Summary 1819.9 Acknowledgements 181References 18110 Embedding semantics in LDA topic models 18310.1 Introduction 18310.2 Background 18410.2.1 Vector space modeling 18410.2.2 Latent semantic analysis 18510.2.3 Probabilistic latent semantic analysis 18510.3 Latent Dirichlet allocation 18610.3.1 Graphical model and generative process 18710.3.2 Posterior inference 18710.3.3 Online latent Dirichlet allocation (OLDA) 18910.3.4 Illustrative example 19110.4 Embedding external semantics from Wikipedia 19310.4.1 Related Wikipedia articles 19410.4.2 Wikipedia-influenced topic model 19410.5 Data-driven semantic embedding 19410.5.1 Generative process with data-driven semantic embedding 19510.5.2 OLDA algorithm with data-driven semantic embedding 19610.5.3 Experimental design 19710.5.4 Experimental results 19910.6 Related work 20210.7 Conclusion and future work 202References 203Index 205