Perspectives in Neural Computing – serie

Neural networks provide a powerful new technology to model and control nonlinear and complex systems. In this book, the authors present a detailed formulation of neural networks from the information-theoretic viewpoint. They show how this perspective provides new insights into the design theory of neural networks. In particular they show how these methods may be applied to the topics of supervised and unsupervised learning including feature extraction, linear and non-linear independent component analysis, and Boltzmann machines. Readers are assumed to have a basic understanding of neural networks, but all the relevant concepts from information theory are carefully introduced and explained. Consequently, readers from several different scientific disciplines, notably cognitive scientists, engineers, physicists, statisticians, and computer scientists, will find this to be a very valuable introduction to this topic.

This volume contains the proceedings of the 12th Italian Workshop on Neural Nets WIRN VIETRI-Ol, jointly organized by the International Institute for Advanced Scientific Studies "Eduardo R. Caianiello" (IIASS), the Societa Italiana Reti Neuroniche (SIREN), the IEEE NNC Italian RIG and the Italian SIG of the INNS. Following the tradition of previous years, we invited three foreign scientists to the workshop, Dr. G. Indiveri and Professors A. Roy and R. Sun, who respectively presented the lectures "Computation in Neuromorphic Analog VLSI Systems", "On Connectionism and Rule Extraction", "Beyond Simple Rule Extraction: Acquiring Planning Knowledge from Neural Networks" (the last two papers being part of the special session mentioned below). In addition, a review talk was presented, dealing with a very up-to-date topic: "NeuroJuzzy Approximator based on Mamdani's Model". A large part of the book contains original contributions approved by referees as oral or poster presentations, which have been assembled for reading convenience into three sections: Architectures and Algorithms, Image and Signal Processing, and Applications. The last part of the books contains the papers of the special Session "From Synapses to Rules". Our thanks go to Prof. B. Apolloni, who organized this section. Furthermore, two sections are dedicated to the memory of two great scientists who were friends in life, Professors Mark Aizerman and Eduardo R. Caianiello. The editors would like to thank the invited speakers, the review lecturers and all the contributors whose highly qualified papers helped with the success of the workshop.

From its early beginnings in the fifties and sixties, the field of neural networks has been steadily developing to become one of the most interdisciplinary areas of research within computer science. This volume contains selected papers from WIRN Vietri-98, the 10th Italian Workshop on Neural Nets, 21-23 May 1998, Vietri sul Mare, Salerno, Italy. This annual event, sponsored amongst others by the IEEE Neural Network Council and the INNS/SIG Italy, brings together the best of research from all over the world. The papers cover a range of key topics within neural networks, including pattern recognition, signal processing, hybrid systems, mathematical models, hardware and software design, and fuzzy techniques. It also includes two review talks on a Morpho-Functional Model to Describe Variability Found at Hippocampal Synapses and Neural Networks and Speech Processing. By providing the reader with a comprehensive overview of recent research in this area, the volume makes a valuable contribution to the Perspectives in Neural Computing Series.

From its early beginnings in the fifties and sixties, the field of neural networks has been steadily developing to become one of the most interdisciplinary areas of research within computer science. This volume contains a selection of papers from WIRN Vietri-99, the 11th Italian Workshop on Neural Nets. This annual event, sponsored, amongst others, by the IEEE Neural Networks Council and the INNS/SIG Italy, brings together the best of research from all over the world. The papers cover a range of topics within neural networks, including pattern recognition, signal and image processing, mathematical models, neuro-fuzzy models and economics applications.

This volume contains the Proceedings of the 8th Italian Workshop on Neural Nets WIRN VIETRI-96, organized by the International Institute for Advanced Scientific Studies "Eduardo R. Caianiello" and by the Societa Italiana Reti Neuroniche (SIREN). We would like to thank Professor L.B. Almeida who agreed to deliver an invited lecture which is included in these Proceedings. The spectrum of contributors and participants covers the activity of Italian research in the field and of some European groups. The highly qualified and motivated attendance is*proof of the interest with which this annual event has met in the Italian scientific community. Maria Marinaro Roberto Tagliaferri Organizing - Scientific Committee: B. Apolloni (Univ. Milano), A. Bertoni (Univ. Milano), D.o. Caviglia (Univ. Genova), P. Campadelli (Univ. Milano), M. Ceccarelli (CNR NapolD, A. Colla (ELSAG - Genova), M. Frixione (/lASS), C. Furlanello (IRST - Trento), G.M. Guazzo (!lASS), M. Gori (Univ. Firenze), F. Lauria (Univ. Napo/D, M. Marinaro (Univ. Salerno - flASS), F. Masulli (Univ. Genova), P. Morasso (Univ. Genova), G. Orlandi (Univ. Roma), E. Pasero (Politecnico Torino), A. Petrosino (CNR Napoli), M. Protasi (Univ.Roma Ilj, S. Rampone (/lASS), R. Serra (Gruppo Ferruzzi Ravenna), F. Sorbello (Univ. Palermo), R. Stefanel.1i (Politecnico Milano), R. Tagliaferri (Univ. Salerno), R. Vaccaro (CNR Napo/D.

This volume contains the Proceedings of the 9th Italian Workshop on Neural Nets WIRN VIETRI-97, organised by the International Institute for Advanced Scientific Studies ((Eduardo R. Caianiello" (IIASS), the Societa Italiana Reti Neuroniche (SIREN) and the IEEE NNC Italian RIG. As in the previous editions some invited and reviewed talks on updated subjects are presented in addition to the original contributions selected by the Refereeing Committee. Also included is Professor C.M. Bishop's Invited paper on: * Latent Variables, Topographic Mappings and Data Visualization; and two review talks that deal with updated topics: * Fuzzy Neural Networks for Pattern Recognition; * A Unifying View of Gradient Calculations and Learning for Locally Recurrent Neural Networks; For publication the original contributions have been assembled into 4 sections: Applications, Architectures and Algorithms, Mathematical Models, Pattern Recognition and Robotics.The Editors thank the invited speaker and all the participants for having contributed to the success of the Workshop by submitting high quality manuscripts, and also express gratitude to the Refereeing Committee for the high quality of the selection process. Maria Marinaro Roberto Tagliaferri VI Organizing -Scientific Committee: B. Apolloni (Univ. Milano), A. Bertoni (Univ. Milano), D.O. Caviglia (Univ. Genova), P. Campadelli (Univ. Milano), A. Colla (ELSAG BAILEY -Genova), M. Frixione (1IASS), C. Furlanello (IRST -Trento), A. Esposito (1IASS), G.M. Guazzo (1IASS), M. Gori (Univ. Firenze), F. Lauria (Univ. Napoli), M. Marinaro (Univ. Salerno - IIASS), F.

Neural networks provide a powerful new technology to model and control nonlinear and complex systems. In this book, the authors present a detailed formulation of neural networks from the information-theoretic viewpoint. They show how this perspective provides new insights into the design theory of neural networks. In particular they show how these methods may be applied to the topics of supervised and unsupervised learning including feature extraction, linear and non-linear independent component analysis, and Boltzmann machines. Readers are assumed to have a basic understanding of neural networks, but all the relevant concepts from information theory are carefully introduced and explained. Consequently, readers from several different scientific disciplines, notably cognitive scientists, engineers, physicists, statisticians, and computer scientists, will find this to be a very valuable introduction to this topic.

The past decade could be seen as the heyday of neurocomputing: in which the capabilities of monolithic nets have been well explored and exploited. The question then is where do we go from here? A logical next step is to examine the potential offered by combinations of artificial neural nets, and it is that step that the chapters in this volume represent. Intuitively, it makes sense to look at combining ANNs. Clearly complex biological systems and brains rely on modularity. Similarly the principles of modularity, and of reliability through redundancy, can be found in many disparate areas, from the idea of decision by jury, through to hardware re­ dundancy in aeroplanes, and the advantages of modular design and reuse advocated by object-oriented programmers. And it is not surprising to find that the same principles can be usefully applied in the field of neurocomput­ ing as well, although finding the best way of adapting them is a subject of on-going research.

Following the intense research activIties of the last decade, artificial neural networks have emerged as one of the most promising new technologies for improving the quality of healthcare. Many successful applications of neural networks to biomedical problems have been reported which demonstrate, convincingly, the distinct benefits of neural networks, although many ofthese have only undergone a limited clinical evaluation. Healthcare providers and developers alike have discovered that medicine and healthcare are fertile areas for neural networks: the problems here require expertise and often involve non-trivial pattern recognition tasks - there are genuine difficulties with conventional methods, and data can be plentiful. The intense research activities in medical neural networks, and allied areas of artificial intelligence, have led to a substantial body of knowledge and the introduction of some neural systems into clinical practice. An aim of this book is to provide a coherent framework for some of the most experienced users and developers of medical neural networks in the world to share their knowledge and expertise with readers.

1. Introdudion This volume collects together the refereed versions of 25 papers presented at the 5th Neural Computation and Psychology Workshop (NCPW5), held at the University of Birmingham from the 8th until the lOth of September 1998. The NCPW is a well-established, lively forum, which brings together researchers from a range of disciplines (artificial intelligence, mathematics, cognitive science, computer science, neurobiology, philosophy and psychology), all of whom are interested in the application of neurally-inspired (connectionist) models to topics in psychology. The theme of the 5th workshop in the series was Connectionist models in cognitive neuroscience', and the workshop aimed to bring together papers focused on the inter-relations between functional (psychological) accounts of cognition and neural accounts of underlying brain processes, linked by connectionist models. From the very beginnings of modern psychology, with the work of William James and his contemporaries, researchers have believed it important to relate behavioural analyses to neurological underpinnings. However, with the advent of connectionist modelling, where models are at least inspired by neuronal processes, this enterprise has received a new boost. With this volume, we hope that this volume adds one further mosaic stone to this ambitious objective, of unifying functional and neuronal accounts of performance.

The conception of fresh ideas and the development of new techniques for Blind Source Separation and Independent Component Analysis have been rapid in recent years. It is also encouraging, from the perspective of the many scientists involved in this fascinating area of research, to witness the growing list of successful applications of these methods to a diverse range of practical everyday problems. This growth has been due, in part, to the number of promising young and enthusiastic researchers who have committed their efforts to expanding the current body of knowledge within this field of research. The author of this book is among one of their number. I trust that the present book by Dr. Mark Girolami will provide a rapid and effective means of communicating some of these new ideas to a wide international audience and that in turn this will expand further the growth of knowledge. In my opinion this book makes an important contribution to the theory of Independent Component Analysis and Blind Source Separation. This opens a range of exciting methods, techniques and algorithms for applied researchers and practitioner engineers, especially from the perspective of artificial neural networks and information theory. It has been interesting to see how rapidly the scientific literature in this area has grown.

Conventional applications of neural networks usually predict a single value as a function of given inputs. In forecasting, for example, a standard objective is to predict the future value of some entity of interest on the basis of a time series of past measurements or observations. Typical training schemes aim to minimise the sum of squared deviations between predicted and actual values (the 'targets'), by which, ideally, the network learns the conditional mean of the target given the input. If the underlying conditional distribution is Gaus­ sian or at least unimodal, this may be a satisfactory approach. However, for a multimodal distribution, the conditional mean does not capture the relevant features of the system, and the prediction performance will, in general, be very poor. This calls for a more powerful and sophisticated model, which can learn the whole conditional probability distribution. Chapter 1 demonstrates that even for a deterministic system and 'be­ nign' Gaussian observational noise, the conditional distribution of a future observation, conditional on a set of past observations, can become strongly skewed and multimodal. In Chapter 2, a general neural network structure for modelling conditional probability densities is derived, and it is shown that a universal approximator for this extended task requires at least two hidden layers. A training scheme is developed from a maximum likelihood approach in Chapter 3, and the performance ofthis method is demonstrated on three stochastic time series in chapters 4 and 5.

Neural networks have had considerable success in a variety of disciplines including engineering, control, and financial modelling. However a major weakness is the lack of established procedures for testing mis-specified models and the statistical significance of the various parameters which have been estimated. This is particularly important in the majority of financial applications where the data generating processes are dominantly stochastic and only partially deterministic. Based on the latest, most significant developments in estimation theory, model selection and the theory of mis-specified models, this volume develops neural networks into an advanced financial econometrics tool for non-parametric modelling. It provides the theoretical framework required, and displays the efficient use of neural networks for modelling complex financial phenomena. Unlike most other books in this area, this one treats neural networks as statistical devices for non-linear, non-parametric regression analysis.

Independent Component Analysis (ICA) is a fast developing area of intense research interest. Following on from Self-Organising Neural Networks: Independent Component Analysis and Blind Signal Separation, this book reviews the significant developments of the past year.It covers topics such as the use of hidden Markov methods, the independence assumption, and topographic ICA, and includes tutorial chapters on Bayesian and variational approaches. It also provides the latest approaches to ICA problems, including an investigation into certain "hard problems" for the very first time.Comprising contributions from the most respected and innovative researchers in the field, this volume will be of interest to students and researchers in computer science and electrical engineering; research and development personnel in disciplines such as statistical modelling and data analysis; bio-informatic workers; and physicists and chemists requiring novel data analysis methods.

This book contains the proceedings of the conference ANNIMAB-l, held 13-16 May 2000 in Goteborg, Sweden. The conference was organized by the Society for Artificial Neural Networks in Medicine and Biology (ANNIMAB-S), which was established to promote research within a new and genuinely cross-disciplinary field. Forty-two contributions were accepted for presentation; in addition to these, S invited papers are also included. Research within medicine and biology has often been characterised by application of statistical methods for evaluating domain specific data. The growing interest in Artificial Neural Networks has not only introduced new methods for data analysis, but also opened up for development of new models of biological and ecological systems. The ANNIMAB-l conference is focusing on some of the many uses of artificial neural networks with relevance for medicine and biology, specifically: • Medical applications of artificial neural networks: for better diagnoses and outcome predictions from clinical and laboratory data, in the processing of ECG and EEG signals, in medical image analysis, etc. More than half of the contributions address such clinically oriented issues. • Uses of ANNs in biology outside clinical medicine: for example, in models of ecology and evolution, for data analysis in molecular biology, and (of course) in models of animal and human nervous systems and their capabilities. • Theoretical aspects: recent developments in learning algorithms, ANNs in relation to expert systems and to traditional statistical procedures, hybrid systems and integrative approaches.

Connectionist Models of Learning, Development and Evolution comprises a selection of papers presented at the Sixth Neural Computation and Psychology Workshop - the only international workshop devoted to connectionist models of psychological phenomena. With a main theme of neural network modelling in the areas of evolution, learning, and development, the papers are organized into six sections: The neural basis of cognition Development and category learning Implicit learning Social cognition Evolution Semantics Covering artificial intelligence, mathematics, psychology, neurobiology, and philosophy, it will be an invaluable reference work for researchers and students working on connectionist modelling in computer science and psychology, or in any area related to cognitive science.

Artificial Intelligence is concerned with producing devices that help or replace human beings in their daily activities. Neural-symbolic learning systems play a central role in this task by combining, and trying to benefit from, the advantages of both the neural and symbolic paradigms of artificial intelligence. This book provides a comprehensive introduction to the field of neural-symbolic learning systems, and an invaluable overview of the latest research issues in this area. It is divided into three sections, covering the main topics of neural-symbolic integration - theoretical advances in knowledge representation and learning, knowledge extraction from trained neural networks, and inconsistency handling in neural-symbolic systems. Each section provides a balance of theory and practice, giving the results of applications using real-world problems in areas such as DNA sequence analysis, power systems fault diagnosis, and software requirements specifications. Neural-Symbolic Learning Systems will be invaluable reading for researchers and graduate students in Engineering, Computing Science, Artificial Intelligence, Machine Learning and Neurocomputing. It will also be of interest to Intelligent Systems practitioners and anyone interested in applications of hybrid artificial intelligence systems.

This book looks at how research into predicting the financial markets has progressed in recent years. The first section of the book describes the financial markets and asks whether they are indeed predictable, given the number of possible economic and financial variables. The second section surveys existing prediction models and looks at how these can be refined so as to provide the best prediction of the market's value at the next time step i.e. in one month's time. The third and forth sections look at the theory of specific prediction models and their applications, whilst the final section discusses possible future developments i.e. so that prediction models can be made more immune to sudden global changes.

Cortical evoked potentials are of interest primarily as tests of changing neuronal excitabilities accompanying normal brain function. The first three steps in the anal­ ysis of these complex waveforms are proper placement of electrodes for recording, the proper choice of electrical or sensory stimulus parameters, and the establish­ ment of behavioral control. The fourth is development of techniques for reliable measurement. Measurement consists of comparison of an unknown entity with a set of standard scales or dimensions having numerical attributes in preassigned degree. A physical object can be described by the dimensions of size, mass, density, etc. In addition there are dimensions such as location, velocity, weight, hardness, etc. Some of these dimensions can be complex (e. g. size depends on three or more subsidiary coordi­ nates), and some can be interdependent or nonorthogonal (e. g. specification of size and mass may determine density). In each dimension the unit is defined with refer­ ence to a standard physical entity, e. g. a unit of mass or length, and the result of measurement is expressed as an equivalence between the unknown and the sum of a specified number of units of that entity. The dimensions of a complex waveform are elementary waveforms from which that waveform can be built by simple addition. Any finite single-valued function of time is admissible. They are called basis functions (lO, 15), and they can be expressed in numeric as well as geometric form.

This volume contains the papers from the first British Neural Network Society meeting held at Queen Elizabeth Hall, King's College, London on 18--20 April 1990. The meeting was sponsored by the London Mathemati­ cal Society. The papers include introductory tutorial lectures, invited, and contributed papers. The invited contributions were given by experts from the United States, Finland, Denmark, Germany and the United Kingdom. The majority of the contributed papers came from workers in the United Kingdom. The first day was devoted to tutorials. Professor Stephen Grossberg was a guest speaker on the first day giving a thorough introduction to his Adaptive Resonance Theory of neural networks. Subsequent tutorials on the first day covered dynamical systems and neural networks, realistic neural modelling, pattern recognition using neural networks, and a review of hardware for neural network simulations. The contributed papers, given on the second day, demonstrated the breadth of interests of workers in the field. They covered topics in pattern recognition, multi-layer feedforward neural networks, network dynamics, memory and learning. The ordering of the papers in this volume is as they were given at the meeting. On the final day talks were given by Professor Kohonen (on self organising maps), Professor Kurten (on the dynamics of random and structured nets) and Professor Cotterill (on modelling the visual cortex). Dr A. Mayes presented a paper on various models for amnesia. The editors have taken the opportunity to include a paper of their own which was not presented at the meeting.

This volume consists of proceedings of the one-day conference on "Coupled Oscillating Neurons" held at King's College, London on December 13th, 1990. The subject is currently of increasing interest to neurophysiologists, neural network researchers, applied mathematicians and physicists. The papers attempt to cover the major areas of the subject, as the titles indicate. It is hoped that the appearance of the papers (some of which have been updated since their original presentation) indicates why the subject is becoming of great excitement. A better understanding of coupled oscillating neurons may well hold the key to a clearer appreciation of the manner in which neural networks composed of such elements can control complex behaviour from the heart to consciousness. December 1991 J.G. Taylor King's College, London C.L.T. Mannion CONTENTS Contributors....... ..................... .......... .......................... .................... ix Introduction to Nonlinear Oscillators I. Stewart ....................................................................................... . Identical Oscillator Networks with Symmetry P.B. Ashwin .................................................................................... 21 Bifurcating Neurones AV. Holden, J. Hyde, M.A Muhamad, H.G. Zhang....................... 41 A Model for Low Threshold Oscillations in Neurons J.L. Hindmarsh, R.M. Rose ............................................................ 81 Information Processing by Oscillating Neurons C.L. T. Mannion, J.G. Taylor ........................................................... 100 Gamma Oscillations, Association and Consciousness R.M.J. Cotterill, C. Nielsen ............................................................. 117 Modelling of Cardiac Rhythm: From Single Celis toMassive Networks D. Noble, J. C. Denyer, H.F. Brown, R. Winslow, A Kimball .......... 1 32 CONTRIBUTORS Ashwin, P.B. Mathematics Institute, University of Warwick, Coventry, CV4 7 AL, UK Brown, H.F.

Neural Network Dynamics is the latest volume in the Perspectives in Neural Computing series. It contains papers presented at the 1991 Workshop on Complex Dynamics in Neural Networks, held at IIASS in Vietri, Italy. The workshop encompassed a wide range of topics in which neural networks play a fundamental role, and aimed to bridge the gap between neural computation and computational neuroscience. The papers - which have been updated where necessary to include new results - are divided into four sections, covering the foundations of neural network dynamics, oscillatory neural networks, as well as scientific and biological applications of neural networks. Among the topics discussed are: A general analysis of neural network activity; Descriptions of various network architectures and nodes; Correlated neuronal firing; A theoretical framework for analyzing the behaviour of real and simulated neuronal networks; The structural properties of proteins; Nuclear phenomenology; Resonance searches in high energy physics; The investigation of information storage; Visual cortical architecture; Visual processing. Neural Network Dynamics is the first volume to cover neural networks and computational neuroscience in such detail. Although it is primarily aimed at researchers and postgraduate students in the above disciplines, it will also be of interest to researchers in electrical engineering, medicine, psychology and philosophy.

Neural Network Applications contains the 12 papers presented at the second British Neural Network Society Meeting (NCM '91) held at King's College London on 1st October 1991. The meeting was sponsored by the Centre for Neural Networks, King's College, and the British Neural Network Society, and was also part of the DEANNA ESPRIT programme. The papers reflect the wide spectrum of neural network applications that are currently being attempted in industry and medicine. They cover medical diagnosis, robotics, plant control, machine learning, and visual inspection, as well as more general discussions on net learning and knowledge representation. The breadth and depth of coverage is a sign of the health of the subject, as well as indicating the importance of neural network developments in industry and the manner in which the applications are progressing. Among the actual topics covered are: Learning algorithms - theory and practice; A review of medical diagnostic applications of neural networks; Simulated ultrasound tomographic imaging of defects; Linear quadtrees for neural network based position invariant pattern recognition; The pRTAM as a hardware-realisable neuron; The cognitive modalities ("CM") system of knowledge representation - the DNA of neural networks? This volume provides valuable reading for all those attempting to apply neural networks, as well as those entering the field, including researchers and postgraduate students in computational neuroscience, neurobiology, electrical engineering, computer science, mathematics, and medicine.

This book is the product of a 15-month intensive investigation of the European artificial network scene, together with a view of the broader framework of the subject in a world context. It could not have been completed in such a remarkably short time, and so effectively, without the dedicated efforts of Louise Turner, the DEANNA secretary, and Geoff Chappell, the DEANNA researcher, at the Centre for Neural Networks, King's College, London. I would like to take this opportunity to thank them for their heroic efforts. I would also like to thank my colleagues in the Centre and in the Mathematics Department, especially Mark Plumbley, Michael Reiss and Trevor Clarkson for all their help and encouragement, Denise Gorse of University College London, for allowing use of her lecture notes as a basis for the tutorial and the DEANNA partners for the part they played. Finally I would like to acknowledge the European Community support, and especially Mike Coyle for his trenchant comments during the carrying out of the work. March 1993 J. G. Taylor CONTENTS PART I: SETTING THE SCENE Chapter 1: DEANNA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 . 1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 . 2 The Geographical Dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1 1. 3 The Industrial Dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 . 4 The Plan for Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 6 Chapter 2: Neural Net Demonstrators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2. 1 The Status of Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2. 2 Reasons for the Employment of Neural Networks . . . . . . . . . . . . . . . . . . . 9 2. 3 Neural Network Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2. 4 Areas of Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2. 5 Typical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

A fundamental objective of Artificial Intelligence (AI) is the creation of in­ telligent computer programs. In more modest terms AI is simply con­ cerned with expanding the repertoire of computer applications into new domains and to new levels of efficiency. The motivation for this effort comes from many sources. At a practical level there is always a demand for achieving things in more efficient ways. Equally, there is the technical challenge of building programs that allow a machine to do something a machine has never done before. Both of these desires are contained within AI and both provide the inspirational force behind its development. In terms of satisfying both of these desires there can be no better example than machine learning. Machines that can learn have an in-built effi­ ciency. The same software can be applied in many applications and in many circumstances. The machine can adapt its behaviour so as to meet the demands of new, or changing, environments without the need for costly re-programming. In addition, a machine that can learn can be ap­ plied in new domains with the genuine potential for innovation. In this sense a machine that can learn can be applied in areas where little is known about possible causal relationships, and even in circumstances where causal relationships are judged not to exist. This last aspect is of major significance when considering machine learning as applied to fi­ nancial forecasting.

About This Book This book is about training methods - in particular, fast second-order training methods - for multi-layer perceptrons (MLPs). MLPs (also known as feed-forward neural networks) are the most widely-used class of neural network. Over the past decade MLPs have achieved increasing popularity among scientists, engineers and other professionals as tools for tackling a wide variety of information processing tasks. In common with all neural networks, MLPsare trained (rather than programmed) to carryout the chosen information processing function. Unfortunately, the (traditional' method for trainingMLPs- the well-knownbackpropagation method - is notoriously slow and unreliable when applied to many prac­ tical tasks. The development of fast and reliable training algorithms for MLPsis one of the most important areas ofresearch within the entire field of neural computing. The main purpose of this book is to bring to a wider audience a range of alternative methods for training MLPs, methods which have proved orders of magnitude faster than backpropagation when applied to many training tasks. The book also addresses the well-known (local minima' problem, and explains ways in which fast training methods can be com­ bined with strategies for avoiding (or escaping from) local minima. All the methods described in this book have a strong theoretical foundation, drawing on such diverse mathematical fields as classical optimisation theory, homotopic theory and stochastic approximation theory.

In almost all areas of science and engineering, the use of computers and microcomputers has, in recent years, transformed entire subject areas. What was not even considered possible a decade or two ago is now not only possible but is also part of everyday practice. As a result, a new approach usually needs to be taken (in order) to get the best out of a situation. What is required is now a computer's eye view of the world. However, all is not rosy in this new world. Humans tend to think in two or three dimensions at most, whereas computers can, without complaint, work in n­ dimensions, where n, in practice, gets bigger and bigger each year. As a result of this, more complex problem solutions are being attempted, whether or not the problems themselves are inherently complex. If information is available, it might as well be used, but what can be done with it? Straightforward, traditional computational solutions to this new problem of complexity can, and usually do, produce very unsatisfactory, unreliable and even unworkable results. Recently however, artificial neural networks, which have been found to be very versatile and powerful when dealing with difficulties such as nonlinearities, multivariate systems and high data content, have shown their strengths in general in dealing with complex problems. This volume brings together a collection of top researchers from around the world, in the field of artificial neural networks.

Evolutionary Learning Algorithms for Neural Adaptive Control is an advanced textbook, which investigates how neural networks and genetic algorithms can be applied to difficult adaptive control problems which conventional results are either unable to solve , or for which they can not provide satisfactory results. It focuses on the principles involved, rather than on the modelling of the applications themselves, and therefore provides the reader with a good introduction to the fundamental issues involved.

Concepts for Neural Networks - A Survey provides a wide-ranging survey of concepts relating to the study of neural networks. It includes chapters explaining the basics of both artificial neural networks and the mathematics of neural networks, as well as chapters covering the more philosophical background to the topic and consciousness. There is also significant emphasis on the practical use of the techniques described in the area of robotics. Containing contributions from some of the world's leading specialists in their fields (including Dr. Ton Coolen and Professor Igor Aleksander), this volume will provide the reader with a good, general introduction to the basic concepts needed to understan d and use neural network technology.