Oleg Okun - Böcker

The rapidly growing amount of data, available from di?erent technologies in the ?eld of bio-sciences, high-energy physics, economy, climate analysis, and in several other scienti?c disciplines, requires a new generation of machine learning and statistical methods to deal with their complexity and hete- geneity. As data collections becomes easier, data analysis is required to be more sophisticated in order to extract useful information from the available data. Even if data can be represented in several ways, according to their structural characteristics, ranging from strings, lists, trees to graphs and other more complex data structures, in most applications they are typically represented as a matrix whose rows correspond to measurable characteristics called f- tures, attributes, variables, depending on the considered discipline and whose columns correspond to examples (cases, samples, patterns). In order to avoid confusion,we will talk about features and examples.In real-worldtasks,there canbe manymorefeatures than examples(cancer classi?cationbasedongene expressionlevels in bioinformatics) or there can be many more examples than features(intrusion detection in computer/networksecurity).In addition, each example can be either labeled or not. Attaching labels allows to distinguish members of the same class or group from members of other classes or groups. Hence, one can talk about supervised and unsupervised tasks that can be solved by machine learning methods. Since it is widely accepted that no single classi?er or clustering algorithm canbesuperiortotheothers,ensemblesofsupervisedandunsupervisedme- ods are gaining popularity. A typical ensemble includes a number of clas- ?ers/clustererswhosepredictionsarecombinedtogetheraccordingtoacertain rule, e.g. majority vote.

This book contains the extended papers presented at the 2nd Workshop on Supervised and Unsupervised Ensemble Methods and their Applications (SUEMA)heldon21-22July,2008inPatras,Greece,inconjunctionwiththe 18thEuropeanConferenceon Arti?cial Intelligence(ECAI 2008). This wo- shop was a successor of the smaller event held in 2007 in conjunction with 3rd Iberian Conference on Pattern Recognition and Image Analysis, Girona, Spain. The success of that event as well as the publication of workshop - pers in the edited book "Supervised and Unsupervised Ensemble Methods and their Applications", published by Springer-Verlag in Studies in Com- tational Intelligence Series in volume 126, encouraged us to continue a good tradition. The scope of both SUEMA workshops (hence, the book as well) is the application of theoretical ideas in the ?eld of ensembles of classi?cation and clusteringalgorithmstoreal/lifeproblemsinscienceandindustry. Ensembles, which represent a number of algorithms whose class or cluster membership predictions are combined together to produce a single outcome value, have alreadyprovedto be a viable alternativeto a single best algorithmin various practical tasks under di?erent scenarios, from bioinformatics to biometrics, from medicine to network security.The ensemble approach is caused to life by the famous "no free lunch" theorem, stating that there is no absolutely best algorithm to solve all problems. Although ensembles cannot be cons- ered as absolute remedy of a single algorithm de?ciency, it is widely believed thatensemblesprovideabetteranswerto"nofreelunch"theoremthanas- glebestalgorithm. Statistical,algorithmical,representational,computational and practical reasons can explain the success of ensemble methods.

The rapidly growing amount of data, available from di?erent technologies in the ?eld of bio-sciences, high-energy physics, economy, climate analysis, and in several other scienti?c disciplines, requires a new generation of machine learning and statistical methods to deal with their complexity and hete- geneity. As data collections becomes easier, data analysis is required to be more sophisticated in order to extract useful information from the available data. Even if data can be represented in several ways, according to their structural characteristics, ranging from strings, lists, trees to graphs and other more complex data structures, in most applications they are typically represented as a matrix whose rows correspond to measurable characteristics called f- tures, attributes, variables, depending on the considered discipline and whose columns correspond to examples (cases, samples, patterns). In order to avoid confusion,we will talk about features and examples.In real-worldtasks,there canbe manymorefeatures than examples(cancer classi?cationbasedongene expressionlevels in bioinformatics) or there can be many more examples than features(intrusion detection in computer/networksecurity).In addition, each example can be either labeled or not. Attaching labels allows to distinguish members of the same class or group from members of other classes or groups. Hence, one can talk about supervised and unsupervised tasks that can be solved by machine learning methods. Since it is widely accepted that no single classi?er or clustering algorithm canbesuperiortotheothers,ensemblesofsupervisedandunsupervisedme- ods are gaining popularity. A typical ensemble includes a number of clas- ?ers/clustererswhosepredictionsarecombinedtogetheraccordingtoacertain rule, e.g. majority vote.

This book contains the extended papers presented at the 3rd Workshop on Supervised and Unsupervised Ensemble Methods and their Applications (SUEMA) that was held in conjunction with the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2010, Barcelona, Catalonia, Spain). As its two predecessors, its main theme was ensembles of supervised and unsupervised algorithms – advanced machinelearning and data mining technique. Unlike a single classification or clustering algorithm, an ensemble is a groupof algorithms, each of which first independently solves the task at hand by assigning a class or cluster label (voting) to instances in a dataset and after that all votes are combined together to produce the final class or cluster membership. As a result, ensembles often outperform best single algorithms in many real-world problems. This book consists of 14 chapters, each of which can be read independently of the others. In addition to two previous SUEMA editions, also published by Springer, many chapters in the current book include pseudo code and/or programming code of the algorithms described in them. This was done in order to facilitate ensemble adoption in practice and to help to both researchers and engineers developing ensemble applications.

This book contains the extended papers presented at the 2nd Workshop on Supervised and Unsupervised Ensemble Methods and their Applications (SUEMA)heldon21-22July,2008inPatras,Greece,inconjunctionwiththe 18thEuropeanConferenceon Arti?cial Intelligence(ECAI 2008). This wo- shop was a successor of the smaller event held in 2007 in conjunction with 3rd Iberian Conference on Pattern Recognition and Image Analysis, Girona, Spain. The success of that event as well as the publication of workshop - pers in the edited book "Supervised and Unsupervised Ensemble Methods and their Applications", published by Springer-Verlag in Studies in Com- tational Intelligence Series in volume 126, encouraged us to continue a good tradition. The scope of both SUEMA workshops (hence, the book as well) is the application of theoretical ideas in the ?eld of ensembles of classi?cation and clusteringalgorithmstoreal/lifeproblemsinscienceandindustry. Ensembles, which represent a number of algorithms whose class or cluster membership predictions are combined together to produce a single outcome value, have alreadyprovedto be a viable alternativeto a single best algorithmin various practical tasks under di?erent scenarios, from bioinformatics to biometrics, from medicine to network security.The ensemble approach is caused to life by the famous "no free lunch" theorem, stating that there is no absolutely best algorithm to solve all problems. Although ensembles cannot be cons- ered as absolute remedy of a single algorithm de?ciency, it is widely believed thatensemblesprovideabetteranswerto"nofreelunch"theoremthanas- glebestalgorithm. Statistical,algorithmical,representational,computational and practical reasons can explain the success of ensemble methods.