Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen...

14
Neural Networks: Artificial Intelligence and Industrial Applications

Transcript of Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen...

Page 1: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Neural Networks: Artificial Intelligence and Industrial Applications

Page 2: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Springer Berlin Heidelberg New York Barcelona Budapest Hong Kong London Milan Paris

Tokyo

Page 3: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Bert Kappen and Stan Gielen (Eds)

Neural Networks: Artificial Intelligence and Industrial Applications Proceedings of the Third Annual SNN Symposium on Neural Networks, Nijmegen, The Netherlands, 14-15 September 1995

, Springer

Page 4: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Bert Kappen and Stan Gielen Dutch Foundation for Neural Networks (SNN) Geert Grooteplein Noord 21 6525 EZ Nijmegen The Netherlands

ISBN-13:978-3-540-19992-2 e-ISBN-13:978-1-4471-3087-1 DOl: 10.10071978-1-4471-3087-1

British Library Cataloguing in Publication Data Neural Networks: Artificial Intelligence and Industrial Applications - Proceedings of the Third Annual SNN Symposium on Neural Networks. Nijmegen. The Netherlands. 14-15 September 1995

l. Kappen. Bert II. GieIen. Stan 006.3

ISBN-13: 978-3-540-19992-2

Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress

Apart from any fair dealing for the purposes of research or private study. or criticism or review. as permitted under the Copyright. Designs and Patents Act 1988. this publication may only be reproduced. stored or transmitted. in any form or by any means. with the prior permission in writing of the publishers. or in the case ofreprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers.

© in individual papers held by the authors unless indicated otherwise

© Springer-Verlag London Limited 1995

The use of registered names. trademarks. etc. in this publication does not imply. even in the absence of a specific statement. that such names are exempt from the relevant laws and regulations and therefore free for general use.

The publisher makes no representation. express or implied. with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made.

Typesetting: Camera ready by contributors

34/3830-543210 Printed on acid-free paper

Page 5: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Preface

This volume contains a collection of papers that were presented during the Third Annual Symposium on Neural Networks in Nijmegen, The Netherlands. The symposium was held on September 14 and 15 1995 and was organised by the Dutch Foundation for Neural Networks (SNN). The symposium consisted of two parallel tracks.

The scientific track is entitled Neural Networks and Artificial Intelligence. The term "Artificial Intelligence" is often associated with "traditional AI" methodology. Here it is used in its literal sense, indicating the problem to create intelligence by artificial means.

When considering the possibility of artificial intelligence, it is important to realise that we would never consider such a thing without the example of natural intelligence. Therefore, design of artificial intelligence should take advantage of the biological solutions. This is the research field called neural networks.

The aim of the scientific track is two-fold: to give an overview of new developments in neuro-biology and the cognitive sciences. These insights may lead to novel computational paradigms for artificial intelligence. Secondly, to give an overview of recent technical and theoretical achievements in robotics, vision and data modeling.

Until recently the use of neural networks has been restricted to the academic world. However, over the last few years industries and businesses have started to become aware of the benefits of neural networks for commercial use. Their ability to learn by examples, to deal with noisy data and with nonlinear structures makes neural networks suitable to deal with problems where conventional computing fails. These abilities of neural networks result in various commercial advantages: such as for instance improved accuracy and system performance, the ability to automate previously manual processes, and reduced development effort. The ultimate proof of the commercial benefits of neural computing is given by the rapidly increasing number of applications.

The industrial track is entitled Neural Networks in Practice. As is emphasised by this title, the industrial track presents working neural network solutions to real industrial problems. The aim of the track is to convince industry and business that neural networks provide solutions where other methods fail. To illustrate that neural computation is not restricted to any specific industrial area, the presentations have been selected from a broad range of application areas, varying from industrial process control to marketing and finance. Finally we would like to stress that most presentations concern applications which are at this moment in commercial use.

This industrial track is one of the activities of the Stimulation Initiative for European Neural Applications (SIENA). SIENA is an Esprit project (EP 9811) whose objective is to accelerate the take-up of Neural Network technology in Europe through a program of activities with a specifically industrial orientation.

Page 6: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

vi

SNN is a partner in SIENA for the Benelux, in a multinational consortium with other partners in the United Kingdom, Spain, France and Germany.

We would like to thank the industrial program committee and our colleagues in the SIENA project for their invaluable assistance in the design of the industrial program. We would like to thank the scientific program committee for their help in the review of the scientific program. Finally, we would like to thank Elma Burg for her great help in the preparation of these proceedings.

Bert Kappen Stan Gielen

Page 7: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Organisation

Scientific Program Committee:

A. Aertsen (Jerusalem), S.I. Amari (Tokyo), J. Buhmann (Bonn), B. van Dijk (Amsterdam), R. Eckhorn (Marburg), S. Gielen (Nijmegen), T. Heskes (Nijmegen), A. Herz (Oxford), B. Kappen (Nijmegen), B. Krose (Amsterdam), V. Lopez (Madrid), D. Mackay (Cambridge), T. Martinetz (MUnchen), J. Taylor (London), W. von Seelen (Bochum).

Industrial Program Committee:

E. Auee (Arnhem), S. Hafner (Stuttgart), T. Harris (Egham, Surrey), B. Kappen (Nijmegen), W. Wiegerinck (Nijmegen), T. Willems (Oisterwijk).

This symposium is organised by the Foundation for Neural Networks (SNN). InnovatieCentrum Midden-en Zuid-Gelderland (IC) and Vereniging Artificiele Neurale Netwerken (V ANN) collaborated in the organisation of the industrial track.

The Foundation for Neural Networks (SNN) is a university-based non-profit­making organisation that stimulates fundamental and applied research on neural networks in the Netherlands. The research program investigates neural information processing strategies for artificial behaviour, vision, pattern recognition and cognitive systems and various application areas. SNN collaborates with industry on neural solutions for their specific industrial applications. Currently, several groups from the universities of Nijmegen, Utrecht, Amsterdam, Delft and Groningen are participating. (Contact address: Geert Grooteplein 21,6525 EZ Nijmegen, The Netherlands, tel +31 80 614245, fax +31 80 541435, e-mail [email protected]. After 10 October 1995: tel + 31 243614245, fax +31 243541435.)

InnovatieCentrum Midden-en Zuid-Gelderland (IC) is an initiative of the Dutch Ministry of Economic Affairs to make knowledge accessible and applicable for small and medium-sized enterprises. IC operates on the basis of independent and individual advice. Also in the field of neural networks IC helps entrepreneurs of SMEs to investigate whether this technology can be useful in their specific situation. (Contact address: E. Auee, Bergstraat 35-4, 6811 LC Arnhem, The Netherlands, tel +31 85 458948, fax +31 85 459311, e-mail Ed_Auee.ICNN.ICNN­[email protected]. After 10 October 1995: tel +31 264458948, fax +31 26 4459311.)

De Vereniging Artificiele Neurale Netwerken (V ANN) is an independent association. It offers its members (academics and industrials) an informal forum to discuss neural networks and their applications. The V ANN is the Dutch Special

Page 8: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

viii

Interest Group of the International Neural Network Society and yearly organises 5 mini-symposia on practical applications of neural networks. To keep its members and others interested a year-book is published. (Contact address: D.J.N. Egberts, Julianalaan 35, 6721 ED Bennekom, The Netherlands, e-mail [email protected]).

Page 9: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Contents

NEURAL NETWORKS AND ARTIFICIAL INTELLIGENCE

Neurobiology - Orals

Segmentation Coding by the Visual System - Neural Signals that Possibly Support Scene Segmentation R. Eckhorn ................................................................................................................. 5

Synchrony and Fast Plasticity in the Visual Cortex B. W. van Dijk ............................................................................................................ 13

Rapid Neural Synchronization: From Spiking Cells to Synfire Webs A. V.M. Herz.................................................. ............................................................. 21

Dynamic Representations Provide the Gradual Specification of Movement Parameters K. Kopecz. W. Erlhagen and G. Schoner ................................................................. 29

Recording from Foveal Striate Cortex While the Monkey is Looking at the Stimulus J. Kruger.!. Bondar and H. Haan.................................................. .......................... 33

Propagation of Synfire Activity in Cortical Networks - A Statistical Approach M.-O. Gewaltig, M. Diesmann and A. Aertsen....................................................... 37

Propagation of Synfire Activity in Cortical Networks - A Dynamical Systems Approach M. Arndt. W. Erlhagen and A. Aertsen................................................................... 41

Neurobiology - Posters

Aspects of Spatiotemporal Learning in Artificial Neural Networks: Modelling Synaptic Membrane Currents Using SPICE Simulations J. Hoekstra and P. Mantel................................................ ........................................ 47

Possible Functional Roles of the Bipartite Dendrites of Pyramidal Cells R. Moller and H.-M. Grop........................................................................................ 51

The Role of Cerebellum in Motor Control F. Mechsner ............................................................................................................... 55

Page 10: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

x

A Point Process Approach to Cortical Networks s. Rotter and A. Aertsen........................................................................................... 59

Stochastic Resonance and Multi-Modal Firing Patterns in Single­Neuron Models D. Linders and B. Kappen ........................................................................................ 63

Cognitive Modelling and Rule Extraction - Orals

The BB Neural Network Rule Extraction Method F.R. Wiersma, M. Poel and A.M. Oudshoff................................................ ............. 69

Implications of Hadley's Definition of Systematicity N. Bakker ................................................................................................................... 73

The Binding Problem in Distributed Systems P.H. de Vries and G.]. Dalenoort............................................................................. 77

Integrating Symbolic and Sub symbolic Architectures for Parsing Arithmetic Expressions and Natural Language Sentences ].A. Tepper, H. Powell and D. Palmer-Brown.......................................... ............... 81

Bayesian Strategies for Machine Learning: Rule Extraction and Concept Detection L. Martignon and K.B. Laskey................................................................................. 85

Cognitive Modelling and Rule Extraction - Posters

Learning at Subsymbolic and Symbolic Levels A. Grurnbach ............................................................................................................. 91

The Construction of Evaluative Maps: Affective Computations in the Amygdala E.P. Fulcher ............................................................................................................... 95

Computation, Cognition and Neural Networks F. van der Velde................................................. ....................................................... 99

Automatic Speech Recognition Systems and Models of Human Word Recognition: A Comparative Analysis P. Wittenburg, D. van Kuijk and K. Behnke........................................................... 103

Language Acquisition and the Necessity of a New Neural Network Approach K. Behnke and P. Wittenburg .................................................................................. 107

A Neural Model of Visual Attention P. van de Laar, T. Heskes and S. Gielen................................................ .................. 111

Page 11: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

xi

The Combination of Knowledge in Fuzzy Cognitive Maps P. Camargo Silva ...................................................................................................... 115

Robotics and Vision - Orals

A Visually Guided Robot and a Neural Network Join to Grasp Slanted Objects P. van der Smagt, A. Dev and F.C.A. Groen ........................................................... 121

The Mobile Robot Rhino f. Buhmann, W. Burgard, A.B. Cremers, D. Fox, T. Hofmann, F. Schneider, J. Strikos and S. Thrun....................................................................... 129

Background Invariant Face Recognition R.P. Wurtz.................................................................................. ............................... 140

Learning Structure from Motion: How to Represent Two-Valued Functions A. Dev, B.f.A. Krose and F.C.A. Groen .................................................................... 144

A Natural Object Recognition System Using Self-Organising Translation-Invariant Maps D. Roobaert and M.M. Van Hulle ........................................................................... 151

Affine Scale-Space for Discrete Pointsets R. Geraets, A.H. Salden, B.M. ter Haar Romeny and M.A. Viergever.................. 155

Robotics and Vision - Posters

Dual Processing of Visual Motion Reduces Smearing, Delay and Noise, but Yields the 'Wavy Edge' and 'Window-Shift' lllusions A.f. Noest ................................................................................................................... 161

The Dynamics of the Perceptual Organisation in Apparent Motion M.A. Giese, G. Schaner and H.S. Hock .................................................................... 165

Multiscale Image Segmentation Based on a Receptive Field Model K.L. Vincken, A.S.E. Koster and M.A. Viergever.................................................... 169

A Neural Network Architecture for Scene Interpretation H.-f. Boehme, U.-D. Braumann, D. Heinke and H.-M. Gross ............................... 173

Neural Dynamics Parametrically Controlled by Image Correlations Organize Robot Navigation H. Neven and G. Schaner ......................................................................................... 177

Page 12: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

xii

Statistical Pattern Recognition - Orals

Statistical Ideas for Selecting Network Architectures B.D. Ripley................................................................................................................. 183

Developments in Probabilistic Modelling with Neural Networks­Ensemble Learning D./.C. MacKay..................................................... ...................................................... 191

A Constructive Algorithm for Building a Feed-Forward Neural Network C. Campbell. S. Coombes and A. Surkan ................................................................ 199

Density Estimation Using SOFM and Adaptive Kernels S.H. Lokerse. L.P./. Veelenturf and /.G. Beltman................................................... 203

Maximum Likelihood Estimates for Markov Networks Using Inhomogeneous Markov Chains H. von Hasseln .......................................................................................................... 207

Statistical Pattern Recognition - Posters

Density Estimation as a Preprocessing Step for Constructive Algorithms /.C. Lemm. V. Beiu and/.G. Taylor......................................................................... 213

Mutual Information Neural Networks: A New Connectionist Paradigm for Dynamic Pattern Recognition Tasks G. Rigoll and /. Rottland .......................................................................................... 217

Output Coding and Modularity for Multi-Class Problems A. Pastoors and T. Heskes........................................................................................ 221

Applications of Neural Networks - Orals

Optimal Training for Neural Network Applied to Nuclear Technology O. Ciftcioglu and E. Tilrkcan................................................................................... 227

Nonlinear Predictive Control with Neural Models H.A.B. te Braake. H.B. Verbruggen and H./.L. van Can........................................ 231

System Identification with Orthogonal Basis Functions and Neural Networks G. Schram. M.H.G. Verhaegen. A. Krijgsman and P. Djavdan............................. 235

Applications of Neural Networks - Posters

Applications of Neural Networks to pH Control J.M. Aragon. M.C. Palancar and /.S. Torrecilla ..................................................... 241

Page 13: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

Appropriate Context Association and Learning Parameters for Word Spotting with Partially Recurrent Neural Networks

xiii

D. van Leeuwen. P. Wittenburg and M. Poel.................................................. ....... 245

EEG Signal Analysis Using Dynamic Time Warp Transformation and Kohonen's Neural Network F. Cremer and L.P.I. Veelenturf............................................ .................................. 249

Learning the Equations of Data C.M. Roadknight. D. Palmer-Brown and G.E. Sanders......................................... 253

New Digital Hardware Concept for Self-Organising Feature Maps D. Ruwisch. H. Rahmel and M. Bode ...................................................................... 257

NEURAL NETWORKS IN PRACTICE

Applications of Neural Networks - Orals

Applicability of Artificial Neural Networks in Small and Medium-Sized Businesses E. T. Auee ................................................................................................................... 265

Calculation for Client Specific Transformers H. Brockmeyer........................................................................................................... 269

Modelling of Industrial Processes Using Natural Computation A.P. de W eijer................................................. ........................................................... 271

Neural Network Control for Steel Rolling Mills T. Martinetz. P. Protzel. O. Gramckow and G. Sorgel........................................... 280

Condition Monitoring with National Power I. MacIntyre and P. Stnith................................................ ........................................ 287

The Electronic Nose for Process Control M.A. Collins and L. Moy .......................................................................................... 297

Automatic Quality Control of Roofing Tiles H.-I. Kolb andl. Wagner................................................. ......................................... 303

Automatic Sorting of Pot Plants with a Neural Network Classifier T. Timmennans ........................................................................................................ 314

Neural Networks Applied to Direct Marketing R.P. fer Heide ............................................................................................................ 322

Modelling Market Dynamics in Food - and Durables Markets R·I· SchiJ:ring.............................................. ................................................................ 330

Handwritten Character Recognition Using Neural Networks A.C.R. Hogervorst. M.K. van Dijk. P.C.M. Verbakel and C. Krijgstnan............................................ .................................................................. 337

Page 14: Neural Networks: Artificial Intelligence and Industrial ...978-1-4471-3087-1/1.pdf · Bert Kappen and Stan Gielen (Eds) Neural Networks: Artificial Intelligence and Industrial Applications

xiv

Neural Networks - The Future of Forecasting in Finance? H.G. Zimmermann ................................................................................................... 344

COUNTERMATCH: A Neural Network Approach to Automatic Signature Verification G. Hesketh ................................................................................................................. 349

ZN-Face: A System for Access Control Using Automated Face Recognition J. Kopecz, W. Konen and E. Schulze-Kruger......................................... .................. 3.56

Current Prediction for Shipping Guidance I.C. Wust .................................................................................................................... 366

Applications of Neural Networks - Posters

Adaptive Nonlinear Control- Linearised Models with Neural Networks M.A. Hussain, I.C. Allwright and L.S. Kershenbaum ............................................ 377

A Software Engineering Approach to Neural Network Specification E. Schikuta.............................................. ................................................................... 381

A Neural Network-Based Software Tool for Number-Plate Recognition A. Frosini, M. Gori and L. Pistolesi ......................................................................... 38.5

Parallel Cross-Validation of Artificial Neural Networks E.P.P.A. Derks, W. Melssen and L.M.C. Buydens................................................... 389

Neural Networks in Large Scale Bank Effect Recognition A. Sie"a, C. Santa Cruz, V. Lopez, G. Fractman, J. Dorronsoro, C. Aguirre, I.M. Soto, A. Medina and R. Lopez ...................................................... 393

Author Index............................................................................................................ 397