David J. Hand - Böcker

Measurement is a fundamental concept that underpins almost every aspect of the modern world. It is central to the sciences, social sciences, medicine, and economics, but it affects everyday life. We measure everything - from the distance of far-off galaxies to the temperature of the air, levels of risk, political majorities, taxes, blood pressure, IQ, and weight. The history of measurement goes back to the ancient world, and its story has been one of gradual standardization. Today there are different types of measurement, levels of accuracy, and systems of units, applied in different contexts. Measurement involves notions of variability, accuracy, reliability, and error, and challenges such as the measurement of extreme values.In this Very Short Introduction, David Hand explains the common mathematical framework underlying all measurement, the main approaches to measurement, and the challenges involved. Following a brief historical account of measurement, he discusses measurement as used in the physical sciences and engineering, the life sciences and medicine, the social and behavioural sciences, economics, business, and public policy.ABOUT THE SERIES: The Very Short Introductions series from Oxford University Press contains hundreds of titles in almost every subject area. These pocket-sized books are the perfect way to get ahead in a new subject quickly. Our expert authors combine facts, analysis, perspective, new ideas, and enthusiasm to make interesting and challenging topics highly readable.

Modern statistics is very different from the dry and dusty discipline of the popular imagination. In its place is an exciting subject which uses deep theory and powerful software tools to shed light and enable understanding. And it sheds this light on all aspects of our lives, enabling astronomers to explore the origins of the universe, archaeologists to investigate ancient civilisations, governments to understand how to benefit and improve society, and businesses to learn how best to provide goods and services. Aimed at readers with no prior mathematical knowledge, this Very Short Introduction explores and explains how statistics work, and how we can decipher them. ABOUT THE SERIES: The Very Short Introductions series from Oxford University Press contains hundreds of titles in almost every subject area. These pocket-sized books are the perfect way to get ahead in a new subject quickly. Our expert authors combine facts, analysis, perspective, new ideas, and enthusiasm to make interesting and challenging topics highly readable.

This book should be of interest to statistics lecturers who want ready-made data sets complete with notes for teaching.

Repeated measures data arise when the same characteristic is measured on each case or subject at several times or under several conditions. There is a multitude of techniques available for analysing such data and in the past this has led to some confusion. This book describes the whole spectrum of approaches, beginning with very simple and crude methods, working through intermediate techniques commonly used by consultant statisticians, and concluding with more recent and advanced methods. Those covered include multiple testing, response feature analysis, univariate analysis of variance approaches, multivariate analysis of variance approaches, regression models, two-stage line models, approaches to categorical data and techniques for analysing crossover designs. The theory is illustrated with examples, using real data brought to the authors during their work as statistical consultants.

Repeated measures data arise when the same characteristic is measured on each case or subject at several times or under several conditions. There is a multitude of techniques available for analysing such data and in the past this has led to some confusion. This book describes the whole spectrum of approaches, beginning with very simple and crude methods, working through intermediate techniques commonly used by consultant statisticians, and concluding with more recent and advanced methods. Those covered include multiple testing, response feature analysis, univariate analysis of variance approaches, multivariate analysis of variance approaches, regression models, two-stage line models, approaches to categorical data and techniques for analysing crossover designs. The theory is illustrated with examples, using real data brought to the authors during their work as statistical consultants.

This book should be of interest to statistics lecturers who want ready-made data sets complete with notes for teaching.

This book presents a summary of recent work on the interface between artificial intelligence and statistics. It does this through a series of papers by different authors working in different areas of this interface. These papers are a selected and referenced subset of papers presented at the 3rd Interntional Workshop on Artificial Intelligence and Statistics, Florida, January 1991.

This text describes regression-based approaches to analyzing longitudinal and repeated measures data. It emphasizes statistical models, discusses the relationships between different approaches, and uses real data to illustrate practical applications. It uses commercially available software when it exists and illustrates the program code and output. The data appendix provides many real data sets-beyond those used for the examples-which can serve as the basis for exercises.

We live in a world of measurements. Measurements, be they of length, speed, weight, temperature, intelligence, income, endurance, greed, gross domestic product, quality of life, unemployment or skill at a job, are all numerical manifestations of the extent of some underlying attribute. They reflect the reality around us – length and weight provide examples of systems that represent clear physical attributes. At the same time, measurements also define the reality around us – psychometric tests and price inflation constitute both the definitions and the procedures for measuring these concepts. Altogether, measurements are central to our modern world and our view of it. This book explores the nature of measurement, investigating its different kinds, how these kinds should be interpreted, and the legitimacy of their statistical manipulation. The procedures through which numbers are assigned to objects are described, and measurement in psychology, medicine, the physical sciences, and the social sciences are examined in detail. The ideas of measurement are so ubiquitous that we often fail to notice them; they are concealed behind a veil of familiarity. This book lifts the corner of that veil and, in doing so, shows that there are aspects of the familiar world that are occasionally puzzling, sometimes downright extraordinary, and often more intriguing than is generally believed.

The chapters in this book describe various aspects of the application of statistical methods in finance. It will interest and attract statisticians to this area, illustrate some of the many ways that statistical tools are used in financial applications, and give some indication of problems which are still outstanding. The statisticians will be stimulated to learn more about the kinds of models and techniques outlined in the book - both the domain of finance and the science of statistics will benefit from increased awareness by statisticians of the problems, models, and techniques applied in financial applications. For this reason, extensive references are given. The level of technical detail varies between the chapters. Some present broad non-technical overviews of an area, while others describe the mathematical niceties. This illustrates both the range of possibilities available in the area for statisticians, while simultaneously giving a flavour of the different kinds of mathematical and statistical skills required. Whether you favour data analysis or mathematical manipulation, if you are a statistician there are problems in finance which are appropriate to your skills.

The author has written an accessible book presenting the central issues and placing particular emphasis on comparison, performance assessment and how to match method to application. He reviews the different approaches and guides researchers (and the huge class of users and potential users) to suitable classes of techniques. He points the way to more detailed study in "further reading" sections in each chapter.

A practical guide to making good decisions in a world of missing dataIn the era of big data, it is easy to imagine that we have all the information we need to make good decisions. But in fact the data we have are never complete, and may be only the tip of the iceberg. Just as much of the universe is composed of dark matter, invisible to us but nonetheless present, the universe of information is full of dark data that we overlook at our peril. In Dark Data, data expert David Hand takes us on a fascinating and enlightening journey into the world of the data we don't see.Dark Data explores the many ways in which we can be blind to missing data and how that can lead us to conclusions and actions that are mistaken, dangerous, or even disastrous. Examining a wealth of real-life examples, from the Challenger shuttle explosion to complex financial frauds, Hand gives us a practical taxonomy of the types of dark data that exist and the situations in which they can arise, so that we can learn to recognize and control for them. In doing so, he teaches us not only to be alert to the problems presented by the things we don’t know, but also shows how dark data can be used to our advantage, leading to greater understanding and better decisions.Today, we all make decisions using data. Dark Data shows us all how to reduce the risk of making bad ones.

A practical guide to making good decisions in a world of missing dataIn the era of big data, it is easy to imagine that we have all the information we need to make good decisions. But in fact the data we have are never complete, and may be only the tip of the iceberg. Just as much of the universe is composed of dark matter, invisible to us but nonetheless present, the universe of information is full of dark data that we overlook at our peril. In Dark Data, data expert David Hand takes us on a fascinating and enlightening journey into the world of the data we don't see.Dark Data explores the many ways in which we can be blind to missing data and how that can lead us to conclusions and actions that are mistaken, dangerous, or even disastrous. Examining a wealth of real-life examples, from the Challenger shuttle explosion to complex financial frauds, Hand gives us a practical taxonomy of the types of dark data that exist and the situations in which they can arise, so that we can learn to recognize and control for them. In doing so, he teaches us not only to be alert to the problems presented by the things we don’t know, but also shows how dark data can be used to our advantage, leading to greater understanding and better decisions.Today, we all make decisions using data. Dark Data shows us all how to reduce the risk of making bad ones.

Since ROC curves have become ubiquitous in many application areas, the various advances have been scattered across disparate articles and texts. ROC Curves for Continuous Data is the first book solely devoted to the subject, bringing together all the relevant material to provide a clear understanding of how to analyze ROC curves.The fundamental theory of ROC curvesThe book first discusses the relationship between the ROC curve and numerous performance measures and then extends the theory into practice by describing how ROC curves are estimated. Further building on the theory, the authors present statistical tests for ROC curves and their summary statistics. They consider the impact of covariates on ROC curves, examine the important special problem of comparing two ROC curves, and cover Bayesian methods for ROC analysis. Special topicsThe text then moves on to extensions of the basic analysis to cope with more complex situations, such as the combination of multiple ROC curves and problems induced by the presence of more than two classes. Focusing on design and interpretation issues, it covers missing data, verification bias, sample size determination, the design of ROC studies, and the choice of optimum threshold from the ROC curve. The final chapter explores applications that not only illustrate some of the techniques but also demonstrate the very wide applicability of these techniques across different disciplines.With nearly 5,000 articles published to date relating to ROC analysis, the explosive interest in ROC curves and their analysis will continue in the foreseeable future. Embracing this growth of interest, this timely book will undoubtedly guide present and future users of ROC analysis.

THE WELLBEING OF NATIONSMEANING, MOTIVE AND MEASUREMENT ‘A great book that adds much needed well-reasoned argument and weight to the global debate on how we better measure what is getting better and what is not. Hand and Allin chart a series of paths away from the beguiling simplicity of GDP, show where those routes began and to where they might lead. Slowly we are learning to better count what really matters in our lives. This book explains the international collaboration behind this new learning and moves it far forward.’Daniel Dorling, School of Geography and the Environment, University of Oxford How and why to use new measures of national wellbeing We have known for many years that standard measures of a country’s economic performance, like GDP and GNP, fail to tell the whole story about progress and wellbeing. The Wellbeing of Nations explores national and cross-national initiatives to build wider measures. It covers economic performance, quality of life, the state of the environment, progress, development and sustainability. The authors take the view that national wellbeing — how a country is doing — embraces all these aspects and so measures of real progress need these dimensions. This book: Presents insights drawn from a wide range of national and international developments and provides an extensive appendix of resource materials.Provides an in-depth study of the UK measuring national wellbeing programme, with which both authors continue to be involved.Explores all the main approaches to wellbeing, such as developments to the national economic accounts, Sen’s capabilities approach and the concept and measurement of sustainable development.Offers a historical perspective on progress and examines current developments in this field.The authors note that it is still early days in the practical application of these wider measures by government and business, but that use should be the main factor driving their development and production. Aspirations are high and there is much to be gained by the use of such wider measures of national wellbeing, progress and development around the world. The Wellbeing of Nations is aimed at statisticians, economists, social researchers and policy makers in government, including national and international statistics offices. Academics in development studies, economics, environmental studies, positive psychology, health research, anthropology, political studies and sociology will also benefit from this book.

Since ROC curves have become ubiquitous in many application areas, the various advances have been scattered across disparate articles and texts. ROC Curves for Continuous Data is the first book solely devoted to the subject, bringing together all the relevant material to provide a clear understanding of how to analyze ROC curves.The fundamental theory of ROC curvesThe book first discusses the relationship between the ROC curve and numerous performance measures and then extends the theory into practice by describing how ROC curves are estimated. Further building on the theory, the authors present statistical tests for ROC curves and their summary statistics. They consider the impact of covariates on ROC curves, examine the important special problem of comparing two ROC curves, and cover Bayesian methods for ROC analysis. Special topicsThe text then moves on to extensions of the basic analysis to cope with more complex situations, such as the combination of multiple ROC curves and problems induced by the presence of more than two classes. Focusing on design and interpretation issues, it covers missing data, verification bias, sample size determination, the design of ROC studies, and the choice of optimum threshold from the ROC curve. The final chapter explores applications that not only illustrate some of the techniques but also demonstrate the very wide applicability of these techniques across different disciplines.With nearly 5,000 articles published to date relating to ROC analysis, the explosive interest in ROC curves and their analysis will continue in the foreseeable future. Embracing this growth of interest, this timely book will undoubtedly guide present and future users of ROC analysis.

John Nelder was one of the most influential statisticians of his generation, having made an impact on many parts of the discipline. This book contains reviews of some of those areas, written by top researchers. It is accessible to non-specialists, and is noteworthy for its breadth of coverage.

This book is about the function and use of official statistics. While thousands of official statistics are published every year, and some are quoted by politicians, used by policy-makers or reported in the media, the authors observe that, in the main, official statistics do not feature much in everyday lives of people and businesses.

ThesearetheproceedingsofthefourthbiennialconferenceintheIntelligentData Analysisseries. TheconferencetookplaceinCascais,Portugal,13-15September 2001. Thethemeofthisconferenceseriesistheuseofcomputersinintelligent waysindataanalysis,includingtheexplorationofintelligentprogramsfordata analysis. Dataanalytictoolscontinuetodevelop,drivenbythecomputerrevo- tion. Methodswhichwouldhaverequiredunimaginableamountsofcomputing power,andwhichwouldhavetakenyearstoreachaconclusion,cannowbe appliedwitheaseandvirtuallyinstantly. Suchmethodsarebeingdevelopedby avarietyofintellectualcommunities,includingstatistics,arti?cialintelligence, neuralnetworks,machinelearning,datamining,andinteractivedynamicdata visualization. Thisconferenceseriesseekstobringtogetherresearchersstudying theuseofintelligentdataanalysisinthesevariousdisciplines,tostimulate- teractionsothateachdisciplinemaylearnfromtheothers. Soastoencourage suchinteraction,wedeliberatelykepttheconferencetoasingletrackmeeting. Thismeantthat,ofthealmost150submissionswereceived,wewereableto selectonly23fororalpresentationand16forposterpresentation.Inaddition tothesecontributedpapers,therewasakeynoteaddressfromDarylPregibon, invitedpresentationsfromKatharinaMorik,RolfBackhofen,andSunilRao,and aspecial'datachallenge'session,whereresearchersdescribedtheirattemptsto analyseachallengingdatasetprovidedbyPaulCohen. Thisacceptancerate enabledustoensureahighqualityconference,whilealsopermittingustop- videgoodcoverageofthevarioustopicssubsumedwithinthegeneralheading ofintelligentdataanalysis. Wewouldliketoexpressourthanksandappreciationtoeveryoneinvolved intheorganizationofthemeetingandtheselectionofthepapers. Itisthe behind-the-scenese?ortswhichensurethesmoothrunningandsuccessofany conference. Wewouldalsoliketoexpressourgratitudetothesponsors:Fundac" ,ao paraaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia,Faculdade deCienciaseTecnologia,UniversidadeNovadeLisboa,Funda,c"aoCalousteG- benkianandIPEInvestimentoseParticipac" ,oesEmpresariais,S. A. September2001 FrankHo?mann DavidJ. Hand NiallAdams GabrielaGuimaraes DougFisher Organization IDA2001wasorganizedbythedepartmentofComputerScience,NewUniversity ofLisbon. ConferenceCommittee GeneralChair: DouglasFisher(VanderbiltUniversity,USA) ProgramChairs: DavidJ.Hand(ImperialCollege,UK) NiallAdams(ImperialCollege,UK) ConferenceChair: GabrielaGuimaraes(NewUniversityofLisbon,Portugal) PublicityChair: FrankHoppner(Univ. ofAppl. SciencesEmden,Germany) PublicationChair: FrankHo?mann(RoyalInstituteofTechnology,Sweden) LocalChair: FernandoMoura-Pires(UniversityofEvora,Portugal) AreaChairs: RobertaSiciliano(UniversityofNaples,Italy) ArnoSiebes(CWI,TheNetherlands) PavelBrazdil(UniversityofPorto,Portugal) ProgramCommittee NiallAdams(ImperialCollege,UK) PieterAdriaans(Syllogic,TheNetherlands) RussellAlmond(EducationalTestingService,USA) ThomasBack(InformatikCentrumDortmund,Germany) RiccardoBellazzi(UniversityofPavia,Italy) MichaelBerthold(Tripos,USA) LiuBing(NationalUniversityofSingapore) PaulCohen(UniversityofMassachusetts,USA) PaulDarius(LeuvenUniversity,Belgium) FazelFamili(NationalResearchCouncil,Canada) DouglasFisher(VanderbiltUniversity,USA) KarlFroeschl(UniversityofVienna,Austria) AlexGammerman(RoyalHolloway,UK) AdolfGrauel(UniversityofPaderborn,Germany) GabrielaGuimaraes(NewUniversityofLisbon,Portugal) LawrenceO. Hall(UniversityofSouthFlorida,USA) FrankHo?mann(RoyalInstituteofTechnology,Sweden) AdeleHowe(ColoradoStateUniversity,USA) Klaus-PeterHuber(SASInstitute,Germany) DavidJensen(UniversityofMassachusetts,USA) JoostKok(LeidenUniversity,TheNetherlands) RudolfKruse(UniversityofMagdeburg,Germany) FrankKlawonn(UniversityofAppliedSciencesEmden,Germany) VIII Organization HansLenz(FreeUniversityofBerlin,Germany) DavidMadigan(Soliloquy,USA) RainerMalaka(EuropeanMediaLaboratory,Germany) HeikkiMannila(Nokia,Finland) FernandoMouraPires(UniversityofEvora,Portugal) SusanaNascimento(UniversityofLisbon,Portugal) WayneOldford(UniversityofWaterloo,Canada) AlbertPrat(TechnicalUniversityofCatalunya,Spain) PeterProtzel(TechnicalUniversityChemnitz,Germany) GiacomodellaRiccia(UniversityofUdine,Italy) RosannaSchiavo(UniversityofVenice,Italy) KaisaSere(AboAkademiUniversity,Finland) RobertaSiciliano(UniversityofNaples,Italy) RosariaSilipo(Nuance,USA) FloorVerdenius(ATO-DLO,TheNetherlands) StefanWrobel(UniversityofMagdeburg,Germany) HuiXiaoLiu(BrunelUniversity,UK) NevinZhang(HongKongUniversityofScienceandTechnology,HongKong) SponsoringInstitutions Fundac" ,aoparaaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia FaculdadedeCienciaseTecnologia,UniversidadeNovadeLisboa Fundac" ,aoCalousteGulbenkian IPEInvestimentoseParticipac" ,15September 2001.Thethemeofthisconferenceseriesistheuseofcomputersinintelligent waysindataanalysis,includingtheexplorationofintelligentprogramsfordata analysis. Dataanalytictoolscontinuetodevelop,drivenbythecomputerrevo- tion. Methodswhichwouldhaverequiredunimaginableamountsofcomputing power,andwhichwouldhavetakenyearstoreachaconclusion,cannowbe appliedwitheaseandvirtuallyinstantly. Suchmethodsarebeingdevelopedby avarietyofintellectualcommunities,includingstatistics,arti?cialintelligence, neuralnetworks,machinelearning,datamining,andinteractivedynamicdata visualization. Thisconferenceseriesseekstobringtogetherresearchersstudying theuseofintelligentdataanalysisinthesevariousdisciplines,tostimulate- teractionsothateachdisciplinemaylearnfromtheothers. Soastoencourage suchinteraction,wedeliberatelykepttheconferencetoasingletrackmeeting. Thismeantthat,ofthealmost150submissionswereceived,wewereableto selectonly23fororalpresentationand16forposterpresentation.Inaddition tothesecontributedpapers,therewasakeynoteaddressfromDarylPregibon, invitedpresentationsfromKatharinaMorik,RolfBackhofen,andSunilRao,and aspecial'datachallenge'session,whereresearchersdescribedtheirattemptsto analyseachallengingdatasetprovidedbyPaulCohen. Thisacceptancerate enabledustoensureahighqualityconference,whilealsopermittingustop- videgoodcoverageofthevarioustopicssubsumedwithinthegeneralheading ofintelligentdataanalysis. Wewouldliketoexpressourthanksandappreciationtoeveryoneinvolved intheorganizationofthemeetingandtheselectionofthepapers. Itisthe behind-the-scenese?ortswhichensurethesmoothrunningandsuccessofany conference. Wewouldalsoliketoexpressourgratitudetothesponsors:Fundac" cao paraaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia,Faculdade deCienciaseTecnologia,UniversidadeNovadeLisboa,Fundacc"aoCalousteG- benkianandIPEInvestimentoseParticipac" coesEmpresariais,S. A. September2001 FrankHo?mann DavidJ. Hand NiallAdams GabrielaGuimaraes DougFisher Organization IDA2001wasorganizedbythedepartmentofComputerScience,NewUniversity ofLisbon. ConferenceCommittee GeneralChair: DouglasFisher(VanderbiltUniversity,USA) ProgramChairs: DavidJ.Hand(ImperialCollege,UK) NiallAdams(ImperialCollege,UK) ConferenceChair: GabrielaGuimaraes(NewUniversityofLisbon,Portugal) PublicityChair: FrankHoppner(Univ. ofAppl. SciencesEmden,Germany) PublicationChair: FrankHo?mann(RoyalInstituteofTechnology,Sweden) LocalChair: FernandoMoura-Pires(UniversityofEvora,Portugal) AreaChairs: RobertaSiciliano(UniversityofNaples,Italy) ArnoSiebes(CWI,TheNetherlands) PavelBrazdil(UniversityofPorto,Portugal) ProgramCommittee NiallAdams(ImperialCollege,UK) PieterAdriaans(Syllogic,TheNetherlands) RussellAlmond(EducationalTestingService,USA) ThomasBack(InformatikCentrumDortmund,Germany) RiccardoBellazzi(UniversityofPavia,Italy) MichaelBerthold(Tripos,USA) LiuBing(NationalUniversityofSingapore) PaulCohen(UniversityofMassachusetts,USA) PaulDarius(LeuvenUniversity,Belgium) FazelFamili(NationalResearchCouncil,Canada) DouglasFisher(VanderbiltUniversity,USA) KarlFroeschl(UniversityofVienna,Austria) AlexGammerman(RoyalHolloway,UK) AdolfGrauel(UniversityofPaderborn,Germany) GabrielaGuimaraes(NewUniversityofLisbon,Portugal) LawrenceO. Hall(UniversityofSouthFlorida,USA) FrankHo?mann(RoyalInstituteofTechnology,Sweden) AdeleHowe(ColoradoStateUniversity,USA) Klaus-PeterHuber(SASInstitute,Germany) DavidJensen(UniversityofMassachusetts,USA) JoostKok(LeidenUniversity,TheNetherlands) RudolfKruse(UniversityofMagdeburg,Germany) FrankKlawonn(UniversityofAppliedSciencesEmden,Germany) VIII Organization HansLenz(FreeUniversityofBerlin,Germany) DavidMadigan(Soliloquy,USA) RainerMalaka(EuropeanMediaLaboratory,Germany) HeikkiMannila(Nokia,Finland) FernandoMouraPires(UniversityofEvora,Portugal) SusanaNascimento(UniversityofLisbon,Portugal) WayneOldford(UniversityofWaterloo,Canada) AlbertPrat(TechnicalUniversityofCatalunya,Spain) PeterProtzel(TechnicalUniversityChemnitz,Germany) GiacomodellaRiccia(UniversityofUdine,Italy) RosannaSchiavo(UniversityofVenice,Italy) KaisaSere(AboAkademiUniversity,Finland) RobertaSiciliano(UniversityofNaples,Italy) RosariaSilipo(Nuance,USA) FloorVerdenius(ATO-DLO,TheNetherlands) StefanWrobel(UniversityofMagdeburg,Germany) HuiXiaoLiu(BrunelUniversity,UK) NevinZhang(HongKongUniversityofScienceandTechnology,HongKong) SponsoringInstitutions Fundac" caoparaaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia FaculdadedeCienciaseTecnologia,UniversidadeNovadeLisboa Fundac" caoCalousteGulbenkian IPEInvestimentoseParticipac" coesEmpresariais,S.A. TableofContents TheFourthInternationalSymposiumonIntelligentData Analysis FeatureCharacterizationinScienti?cDatasets...1 ElizabethBradley(UniversityofColorado),NancyCollins(University ofColorado),W. PhilipKegelmeyer(SandiaNationalLaboratories) RelevanceFeedbackintheBayesianNetworkRetrievalModel: AnApproachBasedonTermInstantiation...13 LuisM. deCampos(UniversityofGranada),JuanM. Fernan ' dez-Luna (UniversityofJa'en),JuanF. Huete(UniversityofGranada) GeneratingFuzzySummariesfromFuzzyMultidimensionalDatabases...24 AnneLaurent(Universit'ePierreetMarieCurie) AMixture-of-ExpertsFrameworkforLearningfromImbalancedData Sets...34 AndrewEstabrooks(IBM),NathalieJapkowicz(UniversityofOttawa) PredictingTime-VaryingFunctionswithLocalModels...44 AchimLewandowski(ChemnitzUniversity),PeterProtzel(Chemnitz University) BuildingModelsofEcologicalDynamicsUsingHMMBasedTemporal DataClustering-APreliminaryStudy...53 CenLi(TennesseeStateUniversity),GautamBiswas(Vanderbilt University),MikeDale(Gri?thUniversity),PatDale(Gri?th University) TaggingwithSmallTrainingCorpora...63 NunoC. Marques(UniversidadeAberta),GabrielPereiraLopes (Centria) ASearchEngineforMorphologicallyComplexLanguages...73 UdoHahn(UniversitatFreiburg),MartinHoneck(Universitat sklinikum Freiburg),StefanSchulz(Universitat oppner(Univ. ofAppl. SciencesEmden,Germany) PublicationChair: FrankHo?mann(RoyalInstituteofTechnology,Sweden) LocalChair: FernandoMoura-Pires(UniversityofEvora,Portugal) AreaChairs: RobertaSiciliano(UniversityofNaples,Italy) ArnoSiebes(CWI,TheNetherlands) PavelBrazdil(UniversityofPorto,Portugal) ProgramCommittee NiallAdams(ImperialCollege,UK) PieterAdriaans(Syllogic,TheNetherlands) RussellAlmond(EducationalTestingService,USA) ThomasBack(InformatikCentrumDortmund,Germany) RiccardoBellazzi(UniversityofPavia,Italy) MichaelBerthold(Tripos,USA) LiuBing(NationalUniversityofSingapore) PaulCohen(UniversityofMassachusetts,USA) PaulDarius(LeuvenUniversity,Belgium) FazelFamili(NationalResearchCouncil,Canada) DouglasFisher(VanderbiltUniversity,USA) KarlFroeschl(UniversityofVienna,Austria) AlexGammerman(RoyalHolloway,UK) AdolfGrauel(UniversityofPaderborn,Germany) GabrielaGuimaraes(NewUniversityofLisbon,Portugal) LawrenceO. Hall(UniversityofSouthFlorida,USA) FrankHo?mann(RoyalInstituteofTechnology,Sweden) AdeleHowe(ColoradoStateUniversity,USA) Klaus-PeterHuber(SASInstitute,Germany) DavidJensen(UniversityofMassachusetts,USA) JoostKok(LeidenUniversity,TheNetherlands) RudolfKruse(UniversityofMagdeburg,Germany) FrankKlawonn(UniversityofAppliedSciencesEmden,Germany) VIII Organization HansLenz(FreeUniversityofBerlin,Germany) DavidMadigan(Soliloquy,USA) RainerMalaka(EuropeanMediaLaboratory,Germany) HeikkiMannila(Nokia,Finland) FernandoMouraPires(UniversityofEvora,Portugal) SusanaNascimento(UniversityofLisbon,Portugal) WayneOldford(UniversityofWaterloo,Canada) AlbertPrat(TechnicalUniversityofCatalunya,Spain) PeterProtzel(TechnicalUniversityChemnitz,Germany) GiacomodellaRiccia(UniversityofUdine,Italy) RosannaSchiavo(UniversityofVenice,Italy) KaisaSere(AboAkademiUniversity,Finland) RobertaSiciliano(UniversityofNaples,Italy) RosariaSilipo(Nuance,USA) FloorVerdenius(ATO-DLO,TheNetherlands) StefanWrobel(UniversityofMagdeburg,Germany) HuiXiaoLiu(BrunelUniversity,UK) NevinZhang(HongKongUniversityofScienceandTechnology,HongKong) SponsoringInstitutions Fundac" caoparaaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia FaculdadedeCienciaseTecnologia,UniversidadeNovadeLisboa Fundac" caoCalousteGulbenkian IPEInvestimentoseParticipac" coesEmpresariais,S.A. TableofContents TheFourthInternationalSymposiumonIntelligentData Analysis FeatureCharacterizationinScienti?cDatasets...1 ElizabethBradley(UniversityofColorado),NancyCollins(University ofColorado),W. PhilipKegelmeyer(SandiaNationalLaboratories) RelevanceFeedbackintheBayesianNetworkRetrievalModel: AnApproachBasedonTermInstantiation...13 LuisM. deCampos(UniversityofGranada),JuanM. Fernan ' dez-Luna (UniversityofJa'en),JuanF. Huete(UniversityofGranada) GeneratingFuzzySummariesfromFuzzyMultidimensionalDatabases...24 AnneLaurent(Universit'ePierreetMarieCurie) AMixture-of-ExpertsFrameworkforLearningfromImbalancedData Sets...34 AndrewEstabrooks(IBM),NathalieJapkowicz(UniversityofOttawa) PredictingTime-VaryingFunctionswithLocalModels...44 AchimLewandowski(ChemnitzUniversity),PeterProtzel(Chemnitz University) BuildingModelsofEcologicalDynamicsUsingHMMBasedTemporal DataClustering-APreliminaryStudy...53 CenLi(TennesseeStateUniversity),GautamBiswas(Vanderbilt University),MikeDale(Gri?thUniversity),PatDale(Gri?th University) TaggingwithSmallTrainingCorpora...63 NunoC. Marques(UniversidadeAberta),GabrielPereiraLopes (Centria) ASearchEngineforMorphologicallyComplexLanguages...73 UdoHahn(UniversitatFreiburg),MartinHoneck(Universitat sklinikum Freiburg),StefanSchulz(Universitat sklinikumFreiburg) ErrorsDetectionandCorrectioninLargeScaleDataCollecting...84 RenatoBruni(Universit'adiRoma),AntonioSassano(Universit'adi Roma) X TableofContents ANewFrameworktoAssessAssociationRules ...95 FernandoBerzal(UniversityofGranada),IgnacioBlanco(University ofGranada),DanielS'anchez(UniversityofGranada), Mar'?a-AmparoVila(UniversityofGranada) CommunitiesofInterest ...105 CorinnaCortes(AT&TShannonResearchLabs),DarylPregibon (AT&TShannonResearchLabs),ChrisVolinsky(AT&TShannon ResearchLabs) AnEvaluationofGradingClassi?ers ...115 AlexanderK. Seewald(AustrianResearchInstituteforArti?cial Intelligence),JohannesFur nkranz(AustrianResearchInstitutefor Arti?cialIntelligence) FindingInformativeRulesinIntervalSequences ...125 FrankHoppner(UniversityofAppliedSciencesEmden), FrankKlawonn(UniversityofAppliedSciencesEmden) Correlation-BasedandContextualMerit-BasedEnsembleFeature Selection...135 SeppoPuuronen(UniversityofJyvaskyla),AlexeyTsymbal(University ofJyvaskyla),IrynaSkrypnyk(UniversityofJyvas kyla) NonmetricMultidimensionalScalingwithNeuralNetworks...145 MichielC. vanWezel(UniversiteitLeiden),WalterA.Kosters (UniversiteitLeiden),PetervanderPutten(UniversiteitLeiden), JoostN. Kok(UniversiteitLeiden) FunctionalTreesforRegression...156 Joao " Gama(UniversityofPorto) DataMiningwithProductsofTrees...167 Jos'eTom'eA. S. Ferreira(ImperialCollege),DavidG. T. Denison (ImperialCollege),DavidJ. Hand(ImperialCollege) 3 SBagging:FastClassi?erInductionMethodwithSubsamplingand Bagging ...177 MasahiroTerabe(MitshibishiResearchInstitute,Inc. ),TakashiWashio (I. S. I. R. ,OsakaUniversity),HiroshiMotoda(I. S. I. R.

This monograph is a detailed introductory presentation of the key classes of intelligent data analysis methods. The twelve coherently written chapters by leading experts provide complete coverage of the core issues. The first half of the book is devoted to the discussion of classical statistical issues, ranging from the basic concepts of probability, through general notions of inference, to advanced multivariate and time series methods, as well as a detailed discussion of the increasingly important Bayesian approaches and Support Vector Machines. The following chapters then concentrate on the area of machine learning and artificial intelligence and provide introductions into the topics of rule induction methods, neural networks, fuzzy logic, and stochastic search methods. The book concludes with a chapter on Visualization and a higher-level overview of the IDA processes, which illustrates the breadth of application of the presented ideas.

This book constitutes the refereed proceedings of an international workshop on Pattern Detection and Discovery organized by the European Science Foundation in London, UK in September 2002. The 17 revised full papers presented were carefully selected and reviewed for inclusion in this state-of-the-art book. Six papers present an introduction and general issues in the emerging field. Four papers are devoted to association rules. Four papers deal with various aspects of text mining and Web mining, and three papers explore advanced applications.

Formanyyearstheintersectionofcomputing anddataanalysiscontainedme- based statistics packages and not much else. Recently, statisticians have - braced computing, computer scientists have started using statistical theories and methods, and researchers in all corners have invented algorithms to nd structure in vast online datasets. Data analysts now have access to tools for exploratory data analysis, decision tree induction, causal induction, function - timation,constructingcustomizedreferencedistributions,andvisualization,and thereareintelligentassistantsto adviseonmatters ofdesignandanalysis.There aretoolsfortraditional,relativelysmallsamples,andalsoforenormousdatasets. In all, the scope for probing data in new and penetrating ways has never been so exciting. The IDA-99 conference brings together a wide variety of researchers c- cerned with extracting knowledge from data, including people from statistics, machine learning, neural networks, computer science, pattern recognition, da- base management, and other areas.The strategiesadopted by people from these areas are often di erent, and a synergy results if this is recognized.The IDA series of conferences is intended to stimulate interaction between these di erent areas,sothatmorepowerfultoolsemergeforextractingknowledgefromdataand a better understanding is developed of the process of intelligent data analysis. The result is a conference that has a clear focus (one application area:intelligent data analysis) and a broad scope (many di erent methods and techniques).