Josef Kittler – författare

This book constitutes the refereed proceedings of the 12th International Workshop on Multiple Classifier Systems, MCS 2015, held in Günzburg, Germany, in June/July 2015. The 19 revised papers presented were carefully reviewed and selected from 25 submissions. The papers address issues in multiple classifier systems and ensemble methods, including pattern recognition, machine learning, neural network, data mining and statistics. They are organized in topical sections on theory and algorithms and application and evaluation.

This book constitutes the refereed proceedings of the 12th International Workshop on Multiple Classifier Systems, MCS 2015, held in Günzburg, Germany, in June/July 2015. The 19 revised papers presented were carefully reviewed and selected from 25 submissions. The papers address issues in multiple classifier systems and ensemble methods, including pattern recognition, machine learning, neural network, data mining and statistics. They are organized in topical sections on theory and algorithms and application and evaluation.

This book constitutes the refereed proceedings of the 20th Iberoamerican Congress on Pattern Recognition, CIARP 2015, held in Montevideo, Uruguay, in November 2015.The 95 papers presented were carefully reviewed and selected from 185 submissions. The papers are organized in topical sections on applications on pattern recognition; biometrics; computer vision; gesture recognition; image classification and retrieval; image coding, processing and analysis; segmentation, analysis of shape and texture; signals analysis and processing; theory of pattern recognition; video analysis, segmentation and tracking.

This book constitutes the refereed proceedings of the 20th Iberoamerican Congress on Pattern Recognition, CIARP 2015, held in Montevideo, Uruguay, in November 2015.

The 95 papers presented were carefully reviewed and selected from 185 submissions. The papers are organized in topical sections on applications on pattern recognition; biometrics; computer vision; gesture recognition; image classification and retrieval; image coding, processing and analysis; segmentation, analysis of shape and texture; signals analysis and processing; theory of pattern recognition; video analysis, segmentation and tracking.

This book constitutes the refereed proceedings of the 9th International Conference on Articulated Motion and Deformable Objects, AMDO 2016, held in Palma de Mallorca, Spain, in July 2016.The 20 papers presented were carefully reviewed and selected from 34 submissions. The conference dealt with the following topics: advanced computer graphics and immersive videogames; human modeling and animation; human motion analysis and tracking; 3D human reconstruction and recognition; multimodal user interaction and applications; ubiquitous and social computing; designtools; input technology; programming user interfaces; 3D medical deformable models and visualization; deep learning methods for computer vision and graphics; multibiometric.

This book constitutes the refereed proceedings of the 9th International Conference on Articulated Motion and Deformable Objects, AMDO 2016, held in Palma de Mallorca, Spain, in July 2016.The 20 papers presented were carefully reviewed and selected from 34 submissions. The conference dealt with the following topics: advanced computer graphics and immersive videogames; human modeling and animation; human motion analysis and tracking; 3D human reconstruction and recognition; multimodal user interaction and applications; ubiquitous and social computing; designtools; input technology; programming user interfaces; 3D medical deformable models and visualization; deep learning methods for computer vision and graphics; multibiometric.

This book constitutes the refereed proceedings of the 10th International Conference on Articulated Motion and Deformable Objects, AMDO 2018, held in Palma de Mallorca, Spain, in July 2018.The 12 papers presented were carefully reviewed and selected from 26 submissions. The papers address the following topics: advanced computer graphics and immersive videogames; human modeling and animation; human motion analysis and tracking; 3D human reconstruction and recognition; multimodal user interaction and applications; ubiquitous and social computing; design tools; input technology; programming user interfaces; 3D medical deformable models and visualization; deep learning methods for computer vision and graphics; and multibiometric.

This book constitutes the refereed proceedings of the 10th International Conference on Articulated Motion and Deformable Objects, AMDO 2018, held in Palma de Mallorca, Spain, in July 2018.The 12 papers presented were carefully reviewed and selected from 26 submissions. The papers address the following topics: advanced computer graphics and immersive videogames; human modeling and animation; human motion analysis and tracking; 3D human reconstruction and recognition; multimodal user interaction and applications; ubiquitous and social computing; design tools; input technology; programming user interfaces; 3D medical deformable models and visualization; deep learning methods for computer vision and graphics; and multibiometric.

Pattern recognition is traditionally considered to cover all aspects of sensory data perception ranging from data acquisition, through preprocessing and low level analysis, to high level interpretation. Owing to its breadth and important application potential, the field of pattern recognition has been attracting considerable attention of researchers in academia and industry and consequently it has been witnessing a rapid growth and perpetual development. The need for dissemination of the latest results is being served by a host of international conferences on pattern recognition. One such series of meetings is regularly held in the United Kingdom under the auspices of the British Pattern Recognition Association. This volume contains papers presented at the BPRA 4th International Conference on Pattern Recognition held in Cambridge, March 28-30, 1988. Alongside the conventional topics of statistical and syntactic pattern recognition, contributions address issues in the hot subject areas of adaptive learning networks, computer vision, knowledge base methods and architectures for pattern processing, and among others, report progress in the application domains of document processing, speech and text recognition and shape analysis for industrial robotics. It is believed that the collection is not merely a report on current activities but that it will also be an important source of inspiration for future developments in the field of pattern recognition.

The fusion of di?erent information sourcesis a persistent and intriguing issue. It hasbeenaddressedforcenturiesinvariousdisciplines,includingpoliticalscience, probability and statistics, system reliability assessment, computer science, and distributed detection in communications. Early seminal work on fusion was c- ried out by pioneers such as Laplace and von Neumann. More recently, research activities in information fusion have focused on pattern recognition. During the 1990s,classi?erfusionschemes,especiallyattheso-calleddecision-level,emerged under a plethora of di?erent names in various scienti?c communities, including machine learning, neural networks, pattern recognition, and statistics. The d- ferent nomenclatures introduced by these communities re?ected their di?erent perspectives and cultural backgrounds as well as the absence of common forums and the poor dissemination of the most important results. In 1999, the ?rst workshop on multiple classi?er systems was organized with the main goal of creating a common international forum to promote the diss- ination of the results achieved in the diverse communities and the adoption of a common terminology, thus giving the di?erent perspectives and cultural ba- grounds some concrete added value. After ?ve meetings of this workshop, there is strong evidence that signi?cant steps have been made towards this goal. - searchers from these diverse communities successfully participated in the wo- shops, and world experts presented surveys of the state of the art from the perspectives of their communities to aid cross-fertilization.

The fusion of di?erent information sourcesis a persistent and intriguing issue. It hasbeenaddressedforcenturiesinvariousdisciplines,includingpoliticalscience, probability and statistics, system reliability assessment, computer science, and distributed detection in communications. Early seminal work on fusion was c- ried out by pioneers such as Laplace and von Neumann. More recently, research activities in information fusion have focused on pattern recognition. During the 1990s,classi?erfusionschemes,especiallyattheso-calleddecision-level,emerged under a plethora of di?erent names in various scienti?c communities, including machine learning, neural networks, pattern recognition, and statistics. The d- ferent nomenclatures introduced by these communities re?ected their di?erent perspectives and cultural backgrounds as well as the absence of common forums and the poor dissemination of the most important results. In 1999, the ?rst workshop on multiple classi?er systems was organized with the main goal of creating a common international forum to promote the diss- ination of the results achieved in the diverse communities and the adoption of a common terminology, thus giving the di?erent perspectives and cultural ba- grounds some concrete added value. After ?ve meetings of this workshop, there is strong evidence that signi?cant steps have been made towards this goal. - searchers from these diverse communities successfully participated in the wo- shops, and world experts presented surveys of the state of the art from the perspectives of their communities to aid cross-fertilization.

The belief that a committee of people make better decisions than any individual is widely held and appreciated. We also understand that, for this to be true, the members of the committee have to be simultaneously competent and comp- mentary. This intuitive notion holds true for committees of data sources (such as sensors) and models (such as classi?ers). The substantial current research in the areas of data fusion and model fusion focuses on ensuring that the di?- ent sources provide useful information but nevertheless complement one another to yield better results than any source would on its own. During the 1990s, a variety of schemes in classi?er fusion, which is the focus of this workshop, were developed under many names in di?erent scienti?c communities such as machine learning, pattern recognition, neural networks, and statistics. The previous ?ve workshops on Multiple Classi?er Systems (MCS) were themselves exercises in information fusion, with the goal of bringing the di?erent scienti?c commu- ties together, providing each other with di?erent perspectives on this fascinating topic, and aiding cross-fertilization of ideas. These ?ve workshops achieved this goal, demonstrating signi?cant advances in the theory, algorithms, and appli- tions of multiple classi?er systems. Followingits?vepredecessorspublishedbySpringer,thisvolumecontainsthe proceedings of the 6th International Workshop on Multiple Classi?er Systems (MCS2005)heldattheEmbassySuitesinSeaside,California,USA,June13-15, 2005. Forty-two papers were selected by the Scienti?c Committee, and they were organized into the following sessions: Boosting, Combination Methods, Design of Ensembles, Performance Analysis, and Applications.

This book constitutes the refereed proceedings of the 4th International Conference on Audio-and Video-Based Biometric Person Authentication, AVBPA 2003, held in Guilford, UK in June 2003.The 39 revised full plenary papers and 72 revised full poster papers were carefully reviewed and selected for presentation. There are topical sections on face; speech; fingerprint; image,video processing, and tracking; general issues; handwriting, signature, and palm; gait; and fusion

Driven by the requirements of a large number of practical and commercially - portant applications, the last decade has witnessed considerable advances in p- tern recognition. Better understanding of the design issues and new paradigms, such as the Support Vector Machine, have contributed to the development of - proved methods of pattern classi cation. However, while any performance gains are welcome, and often extremely signi cant from the practical point of view, it is increasingly more challenging to reach the point of perfection as de ned by the theoretical optimality of decision making in a given decision framework. The asymptoticity of gains that can be made for a single classi er is a re?- tion of the fact that any particular design, regardless of how good it is, simply provides just one estimate of the optimal decision rule. This observation has motivated the recent interest in Multiple Classi er Systems , which aim to make use of several designs jointly to obtain a better estimate of the optimal decision boundary and thus improve the system performance. This volume contains the proceedings of the international workshop on Multiple Classi er Systems held at Robinson College, Cambridge, United Kingdom (July 2{4, 2001), which was organized to provide a forum for researchers in this subject area to exchange views and report their latest results.

Morethanadecadeago,combiningmultipleclassi?erswasproposedasap- siblesolutiontotheproblemsposedbythetraditionalpatternclassi?cation approachwhichinvolvedselectingthebestclassi?erfromasetofcandidates basedontheirexperimentalevaluation. Asnoclassi?erisknowntobethebest forallcasesandtheselectionofthebestclassi?erforagivenpracticaltaskis verydi?cult,diverseresearchcommunities,includingMachineLearning,N- ralNetworks,PatternRecognition,andStatistics,addressedtheengineering problemofhowtoexploitthestrengthswhileavoidingtheweaknessesofd- ferentdesigns. Thisambitiousresearchtrendwasalsomotivatedbyempirical observationsaboutthecomplementarityofdi?erentclassi?erdesigns,natural requirementsofinformationfusionapplications,andintrinsicdi?cultiesasso- atedwiththeoptimalchoiceofsomeclassi?erdesignparameters,suchasthe architectureandtheinitialweightsforaneuralnetwork. Afteryearsofresearch, thecombinationofmultipleclassi?ershasbecomeawellestablishedandexciting researcharea,whichprovidese?ectivesolutionstodi?cultpatternrecognition problems. Aconsiderablebodyofempiricalevidencesupportsthemeritof- signingcombinedsystemswhoseaccuracyishigherthanthatofeachindividual classi?er,andvariousmethodsforthegenerationandthecombinationofm- tipleclassi?ershavebecomeavailable. However,despitetheprovedutilityof multipleclassi?ersystems,nogeneralanswertotheoriginalquestionaboutthe possibilityofexploitingthestrengthswhileavoidingtheweaknessesofdi?erent classi?erdesignshasyetemerged. Otherfundamentalissuesarealsoamatterof on-goingresearchindi?erentresearchcommunities. Theresultsachievedd- ingthepastyearsarealsospreadoverdi?erentresearchcommunities,andthis makesitdi?culttoexchangesuchresultsandpromotetheircross-fertilization. Theacknowledgmentofthefundamentalrolethatthecreationofacommon internationalforumforresearchersofthediversecommunitiescouldplayfor theadvancementofthisresearch?eldmotivatedthepresentseriesofwo- shopsonmultipleclassi?ersystems. Followingitspredecessors,MultipleCl- si?erSystems2000(SpringerISBN3-540-67704-6)and2001(SpringerISBN 3-540-42284-6),thisvolumecontainstheproceedingsoftheThirdInternational WorkshoponMultipleClassi?erSystems(MCS2002),heldattheGrandHotel ChiaLaguna,Cagliari,Italy,onJune24-26,2002. The29papersselectedby thescienti?ccommitteehavebeenorganizedinsessionsdealingwithbagging andboosting,ensemblelearningandneuralnetworks,combinationstrategies, designmethodologies,analysisandperformanceevaluation,andapplications.Theworkshopprogramandthisvolumeareenrichedwiththreeinvitedtalks givenbyJoydeepGhosh(UniversityofTexas,USA),TrevorHastie(Stanford University,USA),andSarunasRaudys(VilniusGediminasTechnicalUniversity, Lithuania). Papersweresubmittedfromresearchersofthefourdiversecom- nities,socon?rmingthatthisseriesofworkshopscanbecomeacommonforum VI Foreword forexchangingviewsandreportinglatestresearchresults. Asfortheprevious editions,thesigni?cantnumberofpapersdealingwithrealpatternrecognition applicationsareproofofthepracticalutilityofmultipleclassi?ersystems. This workshopwassupportedbytheUniversityofCagliari,Italy,theUniversityof Surrey,Guildford,UnitedKingdom,andtheDepartmentofElectricalandEl- tronicEngineeringoftheUniversityofCagliari. Allthesesupportsaregratefully acknowledged. WealsothanktheInternationalAssociationforPatternRecog- tionanditsTechnicalCommitteeTC1onStatisticalPatternRecognitionTe- niquesforsponsoringMCS2002. Wewishtoexpressourappreciationtoallthose whohelpedtoorganizeMCS2002. Firstofall,wewouldliketothankallthe membersoftheScienti?cCommitteewhoseprofessionalismwasinstrumental increatingaveryinterestingtechnicalprogram.Specialthanksareduetothe membersoftheOrganizingCommittee,GiorgioFumera,GiorgioGiacinto,and GianLucaMarcialisfortheirindispensablecontributionstotheMCS2002web sitemanagement,localorganization,andproceedingspreparation. April2002 FabioRoliandJosefKittler WorkshopChairs F. Roli(Univ. ofCagliari,Italy) J. Kittler(Univ. ofSurrey,UnitedKingdom) Scienti?cCommittee J. A. Benediktsson(Iceland) M. Kamel(Canada) H. Bunke(Switzerland) L. I. Kuncheva(UK) L. P. Cordella(Italy) L. Lam(HongKong) B. V. Dasarathy(USA) D. Landgrebe(USA) R. P. W. Duin(TheNetherlands) Dar-ShyangLee(USA) C. Furlanello(Italy) D. Partridge(UK) J. Ghosh(USA) A. J. C. Sharkey(UK) T. K. Ho(USA) K. Tumer(USA) S. Impedovo(Italy) G. Vernazza(Italy) N. Intrator(Israel) T. Windeatt(UK) A. K. Jain(USA) LocalCommittee G. Fumera(Univ. ofCagliari,Italy) G. Giacinto(Univ. ofCagliari,Italy) G. L. Marcialis(Univ. ofCagliari,Italy) Organizedby Dept. ofElectricalandElectronicEngineeringoftheUniversityofCagliari UniversityofSurrey Sponsoredby UniversityofCagliari UniversityofSurrey Dept. ofElectricalandElectronicEngineeringoftheUniversityofCagliari TheInternationalAssociationforPatternRecognition Supportedby UniversityofCagliari Dept. ofElectricalandElectronicEngineeringoftheUniversityofCagliari UniversityofSurrey TableofContents InvitedPapers Multiclassi?erSystems:BacktotheFuture...1 J. Ghosh SupportVectorMachines,KernelLogisticRegressionandBoosting...16 J. Zhu,T. Hastie MultipleClassi?cationSystemsintheContextofFeatureExtractionand Selection...27 ? S. Raudys BaggingandBoosting BoostedTreeEnsemblesforSolvingMulticlassProblems...42 T. Windeatt,G. Ardeshir DistributedPastingofSmallVotes...52 N. V. Chawla,L. O. Hall,K. W. Bowyer,T. E. Moore,Jr. , W. P. Kegelmeyer BaggingandBoostingfortheNearestMeanClassi?er:E?ectsofSample SizeonDiversityandAccuracy...62 M. Skurichina,L. I. Kuncheva,R. P. W. Duin HighlightingHardPatternsviaAdaboostWeightsEvolution ...72 B. Caprile,C. Furlanello,S. Merler UsingDiversitywithThreeVariantsofBoosting:Aggressive,Conservative, andInverse ...81 L. I. Kuncheva,C. J. Whitaker EnsembleLearningandNeuralNetworks MultistageNeuralNetworkEnsembles...91 S. Yang,A. Browne,P. D. Picton ForwardandBackwardSelectioninRegressionHybridNetwork...98 S. Cohen,N. Intrator TypesofMultinetSystem...108 A. J. C.

This book constitutes the refereed proceedings of the 11th Iberoamerican Congress on Pattern Recognition, CIARP 2006, held in Cancun, Mexico in November 2006. The 99 revised full papers presented together with three keynote articles were carefully reviewed and selected from 239 submissions. The papers cover ongoing research and mathematical methods.

Driven by the requirements of a large number of practical and commercially - portant applications, the last decade has witnessed considerable advances in p- tern recognition. Better understanding of the design issues and new paradigms, such as the Support Vector Machine, have contributed to the development of - proved methods of pattern classi cation. However, while any performance gains are welcome, and often extremely signi cant from the practical point of view, it is increasingly more challenging to reach the point of perfection as de ned by the theoretical optimality of decision making in a given decision framework. The asymptoticity of gains that can be made for a single classi er is a re?- tion of the fact that any particular design, regardless of how good it is, simply provides just one estimate of the optimal decision rule. This observation has motivated the recent interest in Multiple Classi er Systems , which aim to make use of several designs jointly to obtain a better estimate of the optimal decision boundary and thus improve the system performance. This volume contains the proceedings of the international workshop on Multiple Classi er Systems held at Robinson College, Cambridge, United Kingdom (July 2{4, 2001), which was organized to provide a forum for researchers in this subject area to exchange views and report their latest results.

Many theoretical and experimental studies have shown that a multiple classi?er system is an e?ective technique for reducing prediction errors [9,10,11,20,19]. These studies identify mainly three elements that characterize a set of cl- si?ers: -Therepresentationoftheinput(whateachindividualclassi?erreceivesby wayofinput). -Thearchitectureoftheindividualclassi?ers(algorithmsandparametri- tion). - The way to cause these classi?ers to take a decision together. Itcanbeassumedthatacombinationmethodise?cientifeachindividualcl- si?ermakeserrors'inadi?erentway',sothatitcanbeexpectedthatmostofthe classi?ers can correct the mistakes that an individual one does [1,19]. The term 'weak classi?ers' refers to classi?ers whose capacity has been reduced in some way so as to increase their prediction diversity. Either their internal architecture issimple(e.g.,theyusemono-layerperceptronsinsteadofmoresophisticated neural networks), or they are prevented from using all the information available. Sinceeachclassi?erseesdi?erentsectionsofthelearningset,theerrorcorre- tion among them is reduced. It has been shown that the majority vote is the beststrategyiftheerrorsamongtheclassi?ersarenotcorrelated.Moreover, in real applications, the majority vote also appears to be as e?cient as more sophisticated decision rules [2,13]. Onemethodofgeneratingadiversesetofclassi?ersistoupsetsomeaspect ofthetraininginputofwhichtheclassi?erisrather unstable. In the present paper,westudytwodistinctwaystocreatesuchweakenedclassi?ers;i.e.learning set resampling (using the 'Bagging' approach [5]), and random feature subset selection (using 'MFS', a Multiple Feature Subsets approach [3]). Other recent and similar techniques are not discussed here but are also based on modi?cations to the training and/or the feature set [7,8,12,21].

These proceedings are a record of the Multiple Classi?er Systems Workshop, MCS 2007, held at the Institute of Information Theory and Automation, Czech Academy of Sciences, Prague in May 2007. Being the seventh in a well-established series of meetings providing an international forum for the discussion of issues in multiple classi?er system design, the workshop achieved its objective of bringing together researchers from diverse communities (neural networks, pattern rec- nition, machine learning and statistics) concerned with this research topic. From more than 80 submissions, the Programme Committee selected 49 - pers to create an interesting scienti?c programme. The special focus of MCS 2007 was on the application of multiple classi?er systems in biometrics. This part- ular application area exercises all aspects of multiple classi?er fusion, from - tramodal classi?er combination, through con?dence-based fusion, to multimodal biometric systems. The sponsorship of MCS 2007 by the European Union N- work of Excellence in Biometrics BioSecure and in Multimedia Understanding through Semantics, Computation and Learning MUSCLE and their assistance in selecting the contributions to the MCS 2007 programme consistent with this theme is gratefully acknowledged.

These proceedings are a record of the Multiple Classi?er Systems Workshop, MCS 2007, held at the Institute of Information Theory and Automation, Czech Academy of Sciences, Prague in May 2007. Being the seventh in a well-established series of meetings providing an international forum for the discussion of issues in multiple classi?er system design, the workshop achieved its objective of bringing together researchers from diverse communities (neural networks, pattern rec- nition, machine learning and statistics) concerned with this research topic. From more than 80 submissions, the Programme Committee selected 49 - pers to create an interesting scienti?c programme. The special focus of MCS 2007 was on the application of multiple classi?er systems in biometrics. This part- ular application area exercises all aspects of multiple classi?er fusion, from - tramodal classi?er combination, through con?dence-based fusion, to multimodal biometric systems. The sponsorship of MCS 2007 by the European Union N- work of Excellence in Biometrics BioSecure and in Multimedia Understanding through Semantics, Computation and Learning MUSCLE and their assistance in selecting the contributions to the MCS 2007 programme consistent with this theme is gratefully acknowledged.

This book constitutes the refereed proceedings of the 12th Iberoamerican Congress on Pattern Recognition, CIARP 2007, held in Valparaiso, Chile, November 13-16, 2007. The 97 revised full papers presented together with four keynote articles were carefully reviewed and selected from 200 submissions. The papers cover ongoing research and mathematical methods for pattern recognition, image analysis, and applications in areas such as computer vision, robotics, industry and health.

These proceedings are a record of the Multiple Classi?er Systems Workshop, MCS 2009, held at the University of Iceland, Reykjavik, Iceland in June 2009. Being the eighth in a well-established series of meetings providing an inter- tional forum for the discussion of issues in multiple classi?er system design, the workshop achieved its objective of bringing together researchers from diverse communities (neural networks,pattern recognition,machine learning and stat- tics) concerned with this research topic. From more than 70 submissions, the Program Committee selected 54 papers to create an interesting scienti?c program. The special focus of MCS 2009 was on the application of multiple classi?er systems in remote sensing. This part- ular application uses multiple classi?ers for raw data fusion, feature level fusion and decision level fusion. In addition to the excellent regular submission in the technical program, outstanding contributions were made by invited speakers Melba Crawford from Purdue University and Zhi-Hua Zhou of Nanjing Univ- sity. Papers of these talks are included in these workshop proceedings. With the workshop’sapplicationfocusbeingonremotesensing,Prof.Crawford’sexpertise in the use of multiple classi?cation systems in this context made the discussions on this topic at MCS 2009 particularly fruitful.