Joaquim P. Marques de Sá – författare

Four years have passed since the first edition of this book. During this time I have had the opportunity to apply it in classes obtaining feedback from students and inspiration for improvements. I have also benefited from many comments by users of the book. For the present second edition large parts of the book have undergone major revision, although the basic concept – concise but sufficiently rigorous mathematical treatment with emphasis on computer applications to real datasets –, has been retained. The second edition improvements are as follows: • Inclusion of R as an application tool. As a matter of fact, R is a free software product which has nowadays reached a high level of maturity and is being increasingly used by many people as a statistical analysis tool. • Chapter 3 has an added section on bootstrap estimation methods, which have gained a large popularity in practical applications. • A revised explanation and treatment of tree classifiers in Chapter 6 with the inclusion of the QUEST approach. • Several improvements of Chapter 7 (regression), namely: details concerning the meaning and computation of multiple and partial correlation coefficients, with examples; a more thorough treatment and exemplification of the ridge regression topic; more attention dedicated to model evaluation. • Inclusion in the book CD of additional MATLAB functions as well as a set of R functions. • Extra examples and exercises have been added in several chapters. • The bibliography has been revised and new references added.

Four years have passed since the first edition of this book. During this time I have had the opportunity to apply it in classes obtaining feedback from students and inspiration for improvements. I have also benefited from many comments by users of the book. For the present second edition large parts of the book have undergone major revision, although the basic concept – concise but sufficiently rigorous mathematical treatment with emphasis on computer applications to real datasets –, has been retained. The second edition improvements are as follows: • Inclusion of R as an application tool. As a matter of fact, R is a free software product which has nowadays reached a high level of maturity and is being increasingly used by many people as a statistical analysis tool. • Chapter 3 has an added section on bootstrap estimation methods, which have gained a large popularity in practical applications. • A revised explanation and treatment of tree classifiers in Chapter 6 with the inclusion of the QUEST approach. • Several improvements of Chapter 7 (regression), namely: details concerning the meaning and computation of multiple and partial correlation coefficients, with examples; a more thorough treatment and exemplification of the ridge regression topic; more attention dedicated to model evaluation. • Inclusion in the book CD of additional MATLAB functions as well as a set of R functions. • Extra examples and exercises have been added in several chapters. • The bibliography has been revised and new references added.

Our lives are immersed in a sea of chance. Everyone’s existence is a meeting point of a multitude of accidents. The origin of the word ‘chance’ is usually traced back to the vulgar Latin word ‘cadentia’, meaning a befalling by fortuitous circumstances, with no knowable or determinable causes. The Roman philosopher Cicero clearly expressed the idea of ‘chance’ in his work De Divinatione: For we do not apply the words ‘chance’, ‘luck’, ‘accident’ or ‘casualty’ except toanevent which hassooccurredorhappened that it either might not have occurred at all, or might have occurred in any other way. 2.VI.15. For if a thing that is going to happen, may happen in one way or another, indi?erently, chance is predominant; but things that happen by chance cannot be certain. 2.IX.24. Ina certain sensechance isthespiceoflife. Iftherewerenophenomena with unforeseeable outcomes, phenomena with an element of chance, all temporal cause–e?ect sequences would be completely deterministic.

Our lives are immersed in a sea of chance. Everyone’s existence is a meeting point of a multitude of accidents. The origin of the word ‘chance’ is usually traced back to the vulgar Latin word ‘cadentia’, meaning a befalling by fortuitous circumstances, with no knowable or determinable causes. The Roman philosopher Cicero clearly expressed the idea of ‘chance’ in his work De Divinatione: For we do not apply the words ‘chance’, ‘luck’, ‘accident’ or ‘casualty’ except toanevent which hassooccurredorhappened that it either might not have occurred at all, or might have occurred in any other way. 2.VI.15. For if a thing that is going to happen, may happen in one way or another, indi?erently, chance is predominant; but things that happen by chance cannot be certain. 2.IX.24. Ina certain sensechance isthespiceoflife. Iftherewerenophenomena with unforeseeable outcomes, phenomena with an element of chance, all temporal cause–e?ect sequences would be completely deterministic.

This book explains the minimum error entropy (MEE) concept applied to data classification machines. Theoretical results on the inner workings of the MEE concept, in its application to solving a variety of classification problems, are presented in the wider realm of risk functionals.Researchers and practitioners also find in the book a detailed presentation of practical data classifiers using MEE. These include multi‐layer perceptrons, recurrent neural networks, complexvalued neural networks, modular neural networks, and decision trees. A clustering algorithm using a MEE‐like concept is also presented. Examples, tests, evaluation experiments and comparison with similar machines using classic approaches, complement the descriptions.

This book explains the minimum error entropy (MEE) concept applied to data classification machines. Theoretical results on the inner workings of the MEE concept, in its application to solving a variety of classification problems, are presented in the wider realm of risk functionals.Researchers and practitioners also find in the book a detailed presentation of practical data classifiers using MEE. These include multi‐layer perceptrons, recurrent neural networks, complexvalued neural networks, modular neural networks, and decision trees. A clustering algorithm using a MEE‐like concept is also presented. Examples, tests, evaluation experiments and comparison with similar machines using classic approaches, complement the descriptions.

Four years have passed since the first edition of this book. During this time I have had the opportunity to apply it in classes obtaining feedback from students and inspiration for improvements. I have also benefited from many comments by users of the book. For the present second edition large parts of the book have undergone major revision, although the basic concept – concise but sufficiently rigorous mathematical treatment with emphasis on computer applications to real datasets –, has been retained. The second edition improvements are as follows: • Inclusion of R as an application tool. As a matter of fact, R is a free software product which has nowadays reached a high level of maturity and is being increasingly used by many people as a statistical analysis tool. • Chapter 3 has an added section on bootstrap estimation methods, which have gained a large popularity in practical applications. • A revised explanation and treatment of tree classifiers in Chapter 6 with the inclusion of the QUEST approach. • Several improvements of Chapter 7 (regression), namely: details concerning the meaning and computation of multiple and partial correlation coefficients, with examples; a more thorough treatment and exemplification of the ridge regression topic; more attention dedicated to model evaluation. • Inclusion in the book CD of additional MATLAB functions as well as a set of R functions. • Extra examples and exercises have been added in several chapters. • The bibliography has been revised and new references added.