Fabio Roli - Böcker

This book constitutes the refereed proceedings of the 12th International Workshop on Multiple Classifier Systems, MCS 2015, held in Günzburg, Germany, in June/July 2015. The papers address issues in multiple classifier systems and ensemble methods, including pattern recognition, machine learning, neural network, data mining and statistics.

This interdisciplinary volume presents a detailed overview of the latest advances and challenges remaining in the field of adaptive biometric systems. A broad range of techniques are provided from an international selection of pre-eminent authorities, collected together under a unified taxonomy and designed to be applicable to any pattern recognition system. Features: presents a thorough introduction to the concept of adaptive biometric systems; reviews systems for adaptive face recognition that perform self-updating of facial models using operational (unlabeled) data; describes a novel semi-supervised training strategy known as fusion-based co-training; examines the characterization and recognition of human gestures in videos; discusses a selection of learning techniques that can be applied to build an adaptive biometric system; investigates procedures for handling temporal variance in facial biometrics due to aging; proposes a score-level fusion scheme for an adaptive multimodal biometric system.

This interdisciplinary volume presents a detailed overview of the latest advances and challenges remaining in the field of adaptive biometric systems. reviews systems for adaptive face recognition that perform self-updating of facial models using operational (unlabeled) data;

This book constitutes the refereed proceedings of the 5th International Workshop on Multiple Classifier Systems, MCS 2004, held in Cagliari, Italy in June 2004.The 35 revised full papers presented together with 2 invited papers were carefully reviewed and selected from 50 submissions. The papers are organized in topical sections on bagging and boosting, combination methods, design methods, performance analysis, and applications.

The belief that a committee of people make better decisions than any individual is widely held and appreciated. We also understand that, for this to be true, the members of the committee have to be simultaneously competent and comp- mentary. This intuitive notion holds true for committees of data sources (such as sensors) and models (such as classi?ers). The substantial current research in the areas of data fusion and model fusion focuses on ensuring that the di?- ent sources provide useful information but nevertheless complement one another to yield better results than any source would on its own. During the 1990s, a variety of schemes in classi?er fusion, which is the focus of this workshop, were developed under many names in di?erent scienti?c communities such as machine learning, pattern recognition, neural networks, and statistics. The previous ?ve workshops on Multiple Classi?er Systems (MCS) were themselves exercises in information fusion, with the goal of bringing the di?erent scienti?c commu- ties together, providing each other with di?erent perspectives on this fascinating topic, and aiding cross-fertilization of ideas. These ?ve workshops achieved this goal, demonstrating signi?cant advances in the theory, algorithms, and appli- tions of multiple classi?er systems. Followingits?vepredecessorspublishedbySpringer,thisvolumecontainsthe proceedings of the 6th International Workshop on Multiple Classi?er Systems (MCS2005)heldattheEmbassySuitesinSeaside,California,USA,June13-15, 2005. Forty-two papers were selected by the Scienti?c Committee, and they were organized into the following sessions: Boosting, Combination Methods, Design of Ensembles, Performance Analysis, and Applications.

This volume contains the Proceedings of the 13th International Conference on Image Analysis and Processing (ICIAP 2005), held in Cagliari, Italy, at the conference centre “Centro della Cultura e dei Congressi”, on September 6–8, 2005. ICIAP 2005 was the thirteenth edition of a series of conferences organized every two years by the Italian group of researchersa?liated to the International Association for Pattern Recognition (GIRPR) with the aim to bring together researchers in image processing and pattern recognition from around the world. As for the previous editions, conference topics concerned the theory of image analysis and processing and its classical and Internet-driven applications. The central theme of ICIAP 2005 was “Pattern Recognition in the Internet and Mobile Communications Era”. The interest for such a theme was con?rmed by the large number of papers dealing with it, the special session devoted to pattern recognition for computer network security, and the emphasis of twoinvited talks on Internet and mobile communication issues. ICIAP 2005 received 217 paper submissions. Fifteen papers were collected into the two special sessions dealing with Pattern Recognition for Computer Network Security and Computer Vision for Augmented Reality and Augmented Environments.

This is the proceedings of the 11th International Workshop on Structural and Syntactic Pattern Recognition, SSPR 2006 and the 6th International Workshop on Statistical Techniques in Pattern Recognition, SPR 2006, held in Hong Kong, August 2006 alongside the Conference on Pattern Recognition, ICPR 2006. 38 revised full papers and 61 revised poster papers are included, together with 4 invited papers covering image analysis, character recognition, bayesian networks, graph-based methods and more.

This book constitutes the refereed proceedings of the 4th International Workshop on Multiple Classifier Systems, MCS 2003, held in Guildford, UK in June 2003.The 40 revised full papers presented with one invited paper were carefully reviewed and selected for presentation. The papers are organized in topical sections on boosting, combination rules, multi-class methods, fusion schemes and architectures, neural network ensembles, ensemble strategies, and applications

This book constitutes the refereed proceedings of the Second International Workshop on Multiple Classifier Systems, MCS 2001, held in Cambridge, UK in July 2001. The 44 revised papers presented were carefully reviewed and selected for presentation. The book offers topical sections on bagging and boosting, MCS design methodology, ensemble classifiers, feature spaces for MCS, MCS in remote sensing, one class MCS and clustering, and combination strategies.

Morethanadecadeago,combiningmultipleclassi?erswasproposedasap- siblesolutiontotheproblemsposedbythetraditionalpatternclassi?cation approachwhichinvolvedselectingthebestclassi?erfromasetofcandidates basedontheirexperimentalevaluation. Asnoclassi?erisknowntobethebest forallcasesandtheselectionofthebestclassi?erforagivenpracticaltaskis verydi?cult,diverseresearchcommunities,includingMachineLearning,N- ralNetworks,PatternRecognition,andStatistics,addressedtheengineering problemofhowtoexploitthestrengthswhileavoidingtheweaknessesofd- ferentdesigns. Thisambitiousresearchtrendwasalsomotivatedbyempirical observationsaboutthecomplementarityofdi?erentclassi?erdesigns,natural requirementsofinformationfusionapplications,andintrinsicdi?cultiesasso- atedwiththeoptimalchoiceofsomeclassi?erdesignparameters,suchasthe architectureandtheinitialweightsforaneuralnetwork. Afteryearsofresearch, thecombinationofmultipleclassi?ershasbecomeawellestablishedandexciting researcharea,whichprovidese?ectivesolutionstodi?cultpatternrecognition problems. Aconsiderablebodyofempiricalevidencesupportsthemeritof- signingcombinedsystemswhoseaccuracyishigherthanthatofeachindividual classi?er,andvariousmethodsforthegenerationandthecombinationofm- tipleclassi?ershavebecomeavailable. However,despitetheprovedutilityof multipleclassi?ersystems,nogeneralanswertotheoriginalquestionaboutthe possibilityofexploitingthestrengthswhileavoidingtheweaknessesofdi?erent classi?erdesignshasyetemerged. Otherfundamentalissuesarealsoamatterof on-goingresearchindi?erentresearchcommunities. Theresultsachievedd- ingthepastyearsarealsospreadoverdi?erentresearchcommunities,andthis makesitdi?culttoexchangesuchresultsandpromotetheircross-fertilization. Theacknowledgmentofthefundamentalrolethatthecreationofacommon internationalforumforresearchersofthediversecommunitiescouldplayfor theadvancementofthisresearch?eldmotivatedthepresentseriesofwo- shopsonmultipleclassi?ersystems. Followingitspredecessors,MultipleCl- si?erSystems2000(SpringerISBN3-540-67704-6)and2001(SpringerISBN 3-540-42284-6),thisvolumecontainstheproceedingsoftheThirdInternational WorkshoponMultipleClassi?erSystems(MCS2002),heldattheGrandHotel ChiaLaguna,Cagliari,Italy,onJune24-26,2002. The29papersselectedby thescienti?ccommitteehavebeenorganizedinsessionsdealingwithbagging andboosting,ensemblelearningandneuralnetworks,combinationstrategies, designmethodologies,analysisandperformanceevaluation,andapplications.Theworkshopprogramandthisvolumeareenrichedwiththreeinvitedtalks givenbyJoydeepGhosh(UniversityofTexas,USA),TrevorHastie(Stanford University,USA),andSarunasRaudys(VilniusGediminasTechnicalUniversity, Lithuania). Papersweresubmittedfromresearchersofthefourdiversecom- nities,socon?rmingthatthisseriesofworkshopscanbecomeacommonforum VI Foreword forexchangingviewsandreportinglatestresearchresults. Asfortheprevious editions,thesigni?cantnumberofpapersdealingwithrealpatternrecognition applicationsareproofofthepracticalutilityofmultipleclassi?ersystems. This workshopwassupportedbytheUniversityofCagliari,Italy,theUniversityof Surrey,Guildford,UnitedKingdom,andtheDepartmentofElectricalandEl- tronicEngineeringoftheUniversityofCagliari. Allthesesupportsaregratefully acknowledged. WealsothanktheInternationalAssociationforPatternRecog- tionanditsTechnicalCommitteeTC1onStatisticalPatternRecognitionTe- niquesforsponsoringMCS2002. Wewishtoexpressourappreciationtoallthose whohelpedtoorganizeMCS2002. Firstofall,wewouldliketothankallthe membersoftheScienti?cCommitteewhoseprofessionalismwasinstrumental increatingaveryinterestingtechnicalprogram.Specialthanksareduetothe membersoftheOrganizingCommittee,GiorgioFumera,GiorgioGiacinto,and GianLucaMarcialisfortheirindispensablecontributionstotheMCS2002web sitemanagement,localorganization,andproceedingspreparation. April2002 FabioRoliandJosefKittler WorkshopChairs F. Roli(Univ. ofCagliari,Italy) J. Kittler(Univ. ofSurrey,UnitedKingdom) Scienti?cCommittee J. A. Benediktsson(Iceland) M. Kamel(Canada) H. Bunke(Switzerland) L. I. Kuncheva(UK) L. P. Cordella(Italy) L. Lam(HongKong) B. V. Dasarathy(USA) D. Landgrebe(USA) R. P. W. Duin(TheNetherlands) Dar-ShyangLee(USA) C. Furlanello(Italy) D. Partridge(UK) J. Ghosh(USA) A. J. C. Sharkey(UK) T. K. Ho(USA) K. Tumer(USA) S. Impedovo(Italy) G. Vernazza(Italy) N. Intrator(Israel) T. Windeatt(UK) A. K. Jain(USA) LocalCommittee G. Fumera(Univ. ofCagliari,Italy) G. Giacinto(Univ. ofCagliari,Italy) G. L. Marcialis(Univ. ofCagliari,Italy) Organizedby Dept. ofElectricalandElectronicEngineeringoftheUniversityofCagliari UniversityofSurrey Sponsoredby UniversityofCagliari UniversityofSurrey Dept. ofElectricalandElectronicEngineeringoftheUniversityofCagliari TheInternationalAssociationforPatternRecognition Supportedby UniversityofCagliari Dept. ofElectricalandElectronicEngineeringoftheUniversityofCagliari UniversityofSurrey TableofContents InvitedPapers Multiclassi?erSystems:BacktotheFuture...1 J. Ghosh SupportVectorMachines,KernelLogisticRegressionandBoosting...16 J. Zhu,T. Hastie MultipleClassi?cationSystemsintheContextofFeatureExtractionand Selection...27 ? S. Raudys BaggingandBoosting BoostedTreeEnsemblesforSolvingMulticlassProblems...42 T. Windeatt,G. Ardeshir DistributedPastingofSmallVotes...52 N. V. Chawla,L. O. Hall,K. W. Bowyer,T. E. Moore,Jr. , W. P. Kegelmeyer BaggingandBoostingfortheNearestMeanClassi?er:E?ectsofSample SizeonDiversityandAccuracy...62 M. Skurichina,L. I. Kuncheva,R. P. W. Duin HighlightingHardPatternsviaAdaboostWeightsEvolution ...72 B. Caprile,C. Furlanello,S. Merler UsingDiversitywithThreeVariantsofBoosting:Aggressive,Conservative, andInverse ...81 L. I. Kuncheva,C. J. Whitaker EnsembleLearningandNeuralNetworks MultistageNeuralNetworkEnsembles...91 S. Yang,A. Browne,P. D. Picton ForwardandBackwardSelectioninRegressionHybridNetwork...98 S. Cohen,N. Intrator TypesofMultinetSystem...108 A. J. C.

This volume gives a state of the art view of recent developments in the use of artificial neural networks for the analysis of remotely sensed satellite data. Remote sensing has now become a discipline in which ever increasing volumes of data, gathered from space together with growing application needs for high precision spatial products, need to be interpreted in shorter times and with increasing accuracy. Neural networks, as a new form of computational paradigm, seem well suited to many of the tasks involved in remotely sensed image analysis. This book demonstrates a wide range of uses of neural networks for remote sensing applications and provides the views of a large number European experts brought together in the framework of a concerted action supported by the European Commission.

Many theoretical and experimental studies have shown that a multiple classi?er system is an e?ective technique for reducing prediction errors [9,10,11,20,19]. These studies identify mainly three elements that characterize a set of cl- si?ers: -Therepresentationoftheinput(whateachindividualclassi?erreceivesby wayofinput). -Thearchitectureoftheindividualclassi?ers(algorithmsandparametri- tion). - The way to cause these classi?ers to take a decision together. Itcanbeassumedthatacombinationmethodise?cientifeachindividualcl- si?ermakeserrors'inadi?erentway',sothatitcanbeexpectedthatmostofthe classi?ers can correct the mistakes that an individual one does [1,19]. The term 'weak classi?ers' refers to classi?ers whose capacity has been reduced in some way so as to increase their prediction diversity. Either their internal architecture issimple(e.g.,theyusemono-layerperceptronsinsteadofmoresophisticated neural networks), or they are prevented from using all the information available. Sinceeachclassi?erseesdi?erentsectionsofthelearningset,theerrorcorre- tion among them is reduced. It has been shown that the majority vote is the beststrategyiftheerrorsamongtheclassi?ersarenotcorrelated.Moreover, in real applications, the majority vote also appears to be as e?cient as more sophisticated decision rules [2,13]. Onemethodofgeneratingadiversesetofclassi?ersistoupsetsomeaspect ofthetraininginputofwhichtheclassi?erisrather unstable. In the present paper,westudytwodistinctwaystocreatesuchweakenedclassi?ers;i.e.learning set resampling (using the 'Bagging' approach [5]), and random feature subset selection (using 'MFS', a Multiple Feature Subsets approach [3]). Other recent and similar techniques are not discussed here but are also based on modi?cations to the training and/or the feature set [7,8,12,21].

These proceedings are a record of the Multiple Classi?er Systems Workshop, MCS 2007, held at the Institute of Information Theory and Automation, Czech Academy of Sciences, Prague in May 2007. Being the seventh in a well-established series of meetings providing an international forum for the discussion of issues in multiple classi?er system design, the workshop achieved its objective of bringing together researchers from diverse communities (neural networks, pattern rec- nition, machine learning and statistics) concerned with this research topic. From more than 80 submissions, the Programme Committee selected 49 - pers to create an interesting scienti?c programme. The special focus of MCS 2007 was on the application of multiple classi?er systems in biometrics. This part- ular application area exercises all aspects of multiple classi?er fusion, from - tramodal classi?er combination, through con?dence-based fusion, to multimodal biometric systems. The sponsorship of MCS 2007 by the European Union N- work of Excellence in Biometrics BioSecure and in Multimedia Understanding through Semantics, Computation and Learning MUSCLE and their assistance in selecting the contributions to the MCS 2007 programme consistent with this theme is gratefully acknowledged.

This volume in the Springer Lecture Notes in Computer Science (LNCS) series contains 98 papers presented at the S+SSPR 2008 workshops. S+SSPR 2008 was the sixth time that the SPR and SSPR workshops organized by Technical Committees, TC1 and TC2, of the International Association for Pattern Rec- nition (IAPR) wereheld as joint workshops. S+SSPR 2008was held in Orlando, Florida, the family entertainment capital of the world, on the beautiful campus of the University of Central Florida, one of the up and coming metropolitan universities in the USA. S+SSPR 2008 was held during December 4–6, 2008 only a few days before the 19th International Conference on Pattern Recog- tion(ICPR2008),whichwasheldin Tampa,onlytwo hoursawayfromOrlando, thus giving the opportunity of both conferences to attendees to enjoy the many attractions o?ered by two neighboring cities in the state of Florida. SPR 2008 and SSPR 2008 received a total of 175 paper submissions from many di?erent countries around the world,thus giving the workshop an int- national clout, as was the case for past workshops. This volume contains 98 accepted papers: 56 for oral presentations and 42 for poster presentations. In addition to parallel oral sessions for SPR and SSPR, there was also one joint oral session with papers of interest to both the SPR and SSPR communities. A recent trend that has emerged in the pattern recognition and machine lea- ing research communities is the study of graph-based methods that integrate statistical andstructural approaches.

These proceedings are a record of the Multiple Classi?er Systems Workshop, MCS 2009, held at the University of Iceland, Reykjavik, Iceland in June 2009. Being the eighth in a well-established series of meetings providing an inter- tional forum for the discussion of issues in multiple classi?er system design, the workshop achieved its objective of bringing together researchers from diverse communities (neural networks,pattern recognition,machine learning and stat- tics) concerned with this research topic. From more than 70 submissions, the Program Committee selected 54 papers to create an interesting scienti?c program. The special focus of MCS 2009 was on the application of multiple classi?er systems in remote sensing. This part- ular application uses multiple classi?ers for raw data fusion, feature level fusion and decision level fusion. In addition to the excellent regular submission in the technical program, outstanding contributions were made by invited speakers Melba Crawford from Purdue University and Zhi-Hua Zhou of Nanjing Univ- sity. Papers of these talks are included in these workshop proceedings. With the workshop’sapplicationfocusbeingonremotesensing,Prof.Crawford’sexpertise in the use of multiple classi?cation systems in this context made the discussions on this topic at MCS 2009 particularly fruitful.

These proceedings are a record of the Multiple Classi?er Systems Workshop, MCS 2010, held at the Nile University, Egypt in April 2010. Being the ninth in a well-established series of meetings providing an international forum for d- cussion of issues in multiple classi?er system design, the workshop achieved its objective of bringing together researchers from diverse communities (neural n- works, pattern recognition, machine learning and statistics) concerned with this researchtopic.Frommorethan50submissions,theProgramCommitteeselected 31 papers to create an interesting scienti?c program.Paperswere organizedinto sessionsdealingwithclassi?ercombinationandclassi?erselection,diversity,b- ging and boosting, combination of multiple kernels, and applications. The wo- shopprogramandthisvolumewereenrichedbytwoinvitedtalksgivenbyGavin Brown(University of Manchester,UK), and Friedhelm Schwenker(University of Ulm, Germany). As usual, the workshop would not have been possible without the help of many individuals and organizations.First of all, our thanks go to the members of the MCS 2010 Program Committee, whose expertise and dedication helped us create an interesting event that marks the progressmade in this ?eld overthe last year and aspire to chart its future research. The help of James Field from the University of Surrey, who administered the submitted paper reviews, and of Giorgio Fumera who managed the MCS website deserve a particular mention. Special thanks are due to the members of the Nile University Organizing C- mittee,AhmedSalah,AmiraElBaroudy,EsraaAly,HebaEzzat,NesrineSameh, Rana Salah and Mohamed Zahhar for their indispensable contributions to the registration management, local organization, and proceedings preparation.

This book constitutes the refereed proceedings of the 10th International Workshop on Multiple Classifier Systems, MCS 2011, held in Naples, Italy, in June 2011. The 36 revised papers presented together with two invited papers were carefully reviewed and selected from more than 50 submissions. The contributions are organized into sessions dealing with classifier ensembles; trees and forests; one-class classifiers; multiple kernels; classifier selection; sequential combination; ECOC; diversity; clustering; biometrics; and computer security.

This book constitutes the refereed proceedings of the 11th International Workshop on Multiple Classifier Systems, MCS 2013, held in Nanjing, China, in May 2013. The 34 revised papers presented together with two invited papers were carefully reviewed and selected from 59 submissions. The papers address issues in multiple classifier systems and ensemble methods, including pattern recognition, machine learning, neural network, data mining and statistics.