Ciencias de Datos y Deep Learning: Neuronas artificiales ... · Ciencia de Datos y Deep Learning:...

CienciasdeDatosyDeepLearning:Neuronasartificialesparaaprender

Francisco Herrera

Ciencia deDatosyDeepLearning:Neuronasartificialesparaaprender

Historyof

DataScience

BigDataDeepLearning

En1943WarrenMcCullochyWalterPitts presentaronsumodelodeneuronaartificial,ydescribieronlosprimerosfundamentosdeloquesellamaríaposteriormenteredesneuronales.

W.McCullochandW.Pitts(1943).ALogicalCalculusofideasImmanentinNervousActivity.BulletinofMathematicalBiophysics5:115-133.

RedesNeuronales

Unaredneuronalseproponecomounsistemainteligentequeimitaalsistemanerviosoyalaestructuradelcerebro,peromuydiferenteentérminosdesuestructurayescala.Aligualquelasneuronasbiológicas,lasneuronasartificialesseinterconectanparaformarredesdeneuronasartificiales.Cadaneuronaartificialutilizaunafunciónprocesamiento queagregalainformacióndeconexionesdeentradaconotrasneuronalesartificiales,unafuncióndeactivaciónyunafuncióndetransferenciaparadarunasalidadelaneuronaensusconexionesdesalida.

yj å=

×=×++×+×=n

iijinjnjjj xwxwxwxwNet

12211 !

÷÷ø

öççè

æ-×=-= å qq ijijjj xwfNetfy )(

RedNeuronalClásica:Backpropagation

Credits:TheEvolutionofNeuralLearningSystems:ANovelArchitectureCombiningtheStrengthsofNTs,CNNs,andELMs.NMartinel, CMicheloni…- IEEESMCMagazine,2015- ieeexplore.ieee.org

• Múltiples capasocultas• Aprendizajejerárquico

• Característicascadavezmáscomplejas• Muybuencomportamiento enmútiplesdominios:Vision,Audio,…

DeepArchitecture(Redes Neuronales conmuchas capas)

UnsupervisedDeepArchitecture: Autoencoder

An autoencoder neuralnetworkisanunsupervisedlearningalgorithmthatappliesbackpropagation,settingthetargetvaluestobeequaltotheinputs.

Theaimofanautoencoder istolearnarepresentation(encoding)forasetofdata,typicallyforthepurposeofdimensionalityreduction.

Andrew Ng

Unsupervised feature learning with a neural network

Layer1

Layer2

Layer3

Andrew Ng

Layer1

Layer2

New representation for input.

Unsupervised feature learning with a neural network

DeepArchitecture: Autoencodern BreastCancerWisconsin(Diagnostic)DataSet

Autoencoder (f(x)=x) (unasolacapainternade3neuronasy1000"epochs".WDBC(569instanciascon32atributosdeentrada)

Credito:D.Charte

http://arxiv.org/abs/1312.5602

DeepMind:Start up-2011Demis Hassabis,Shane Legg yMustafa Suleyman

JuegosArcade(Breakout)

DeepLearningRetos enla“pintura”

http://www.deepart.io/

DeepLearning: DeepART

Ejemplos delresultado deDeepART

vanGoth

Ejemplos delresultado deDeepART

ModelodeDLutilizadoydescripcióndelametodología

http://arxiv.org/abs/1508.06576

Supervised DeepLearning:Convolutional NeuralNetworks

http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg

Supervised DeepLearning:Convolutional NeuralNetworks

Eachmoduleconsistsofaconvolutional layerandapoolinglayer.

Typicallytriestocompresslargedata(images)intoasmallersetofrobustfeatures,basedonlocalvariations.

Basicconvolutioncanstillcreatemanyfeatures.

CNNshavebeenfoundhighlyeffectiveandbeencommonlyusedincomputervisionandimagerecognition.

DeepLearning

Convolutional steps

https://github.com/rasbt/python-machine-learning-book/blob/master/faq/difference-deep-and-normal-learning/convolution.png

http://neuralnetworksanddeeplearning.com/chap6.html

DeepLearningDigitRecognizerandConvolutional NN

Convolutional neuralnetworks usethreebasicideas:localreceptivefields,sharedweights,andpooling.

Localreceptive fields:Tobemoreprecise,eachneuroninthefirsthiddenlayerwillbeconnectedtoasmallregionoftheinputneurons,say,forexample,a5×5region,correspondingto25inputpixels.So,foraparticularhiddenneuron,wemighthaveconnectionsthatlooklikethis:

Localreceptive fields:

24×24 neurons

28×28 input image

Shared weights andbiases:thesame weightsandbiasforeachofthe24×24hiddenneurons(sigmoide function)

Themapfromtheinputlayertothehiddenlayerafeaturemap.

Shared weights andbiases:

In the example shown, there are 3 feature maps.

If we have 20 feature maps that's a total of 20×26=520 parameters defining the convolutional layer. By comparison, suppose we had a fully connected first layer, with 784=28×28 input neurons, 30 hidden neurons, 23,550 parameters.

Pooling layers: thepoolinglayersdoissimplifytheinformationintheoutputfromtheconvolutional layer,onecommonprocedureforpoolingisknownasmax-pooling,inthe2x2regioninput.

Pooling layers:

The20imagescorrespondto20differentfeaturemaps

Credits:https://www.datarobot.com/blog/a-primer-on-deep-learning/

Alaizquierda,losdígitosdeentradasinprocesar.Aladerecha,representacionesgráficasdelascaracterísticasaprendidas.Enesencia,laredaprendea"ver"líneasybucles.

DeepLearning:MNISTdata

Asnapshotofimagepre-processingforconvolutional neuralnetworks:casestudyofMNISTSiham Tabik,DanielPeralta,AndrésHerrera-Poyatos,FranciscoHerreraInternationalJournal ofComputational Intelligence Systems,Vol.10(2017)555–56899.72accuracy

DeepLearningDigitRecognizerandpreprocessing

CNNmodels:LeNet,Network3,DropConnet

Preprocessing andaugmentation

Ensembles

CNNmodels:LeNet [4]

[4]Yann LeCun,Leon Bottou,Yoshua Bengio,andPatrickHaffner.Gradient-basedlearningappliedtodocumentrecognition.Proc.IEEE,86(11):2278– 2324,1998.

CNNmodels:Network3[13]

[13]MichaelANielsen.Neuralnetworksanddeep learning.URL:http://neuralnetworksanddeeplearning.com,2015.

CNNmodels:DropConnet [14]

[14]LiWan,Matthew Zeiler,Sixin Zhang,Yann LCun,andRobFergus.Regularizationofneuralnetworksusingdropconnect.InProceedingsofthe30thInternationalConferenceonMachineLearning(ICML- 13),pages 1058–1066,2013.

Translation.Theimageistranslatedanumberofpixelstowardagivendirection.

Centering.Toeliminatewhitecolumns/rows,andtoresize by scaling.

Rotation.Theimageisrotatedtoagivenangleθ.

Elastic deformation.Image pixels areslightly movedinrandomdirections,keepingtheimage’stopology.

Ensembles

The28handwrittendigitsmisclassifiedbyensemble-5ofNetwork3Thedigitbetween()representsthecorrectclass.The13digitslabeledwithasterisksarealsomisclassifiedbyDropConnect

Deep Learning: MNIST data Asnapshotofimagepre-processingforconvolutional neuralnetworks:casestudy ofMNISTSiham Tabik,DanielPeralta,AndrésHerrera-Poyatos,FranciscoHerreraInternationalJournal ofComputational Intelligence Systems,Vol.10(2017)555–56899.72accuracy

The28handwrittendigitsmisclassifiedbyensemble-5ofDropConnetThedigitbetween()representsthecorrectclass.The13digitslabeledwithasterisksarealsomisclassifiedbyNetwork3

The13handwrittendigitsmisclassifiedbyensemble-5ofDropConnetandNetwork3

DeepLearningMXNet

http://mxnet.io/get_started/index.html

DeepLearningMXNet

http://mxnet.io/how_to/finetune.html

DeepLearningLearningfromscratchvs fine-tuning(GoogLeNet yVGG-16)

VGG16http://www.robots.ox.ac.uk/~vgg/research/very_deep/

https://gist.github.com/ksimonyan/211839e770f7b538e2d8#file-readme-md

##Informationname:16-layermodelfromthearXiv paper:"VeryDeepConvolutional NetworksforLarge-ScaleImageRecognition"caffemodel:VGG_ILSVRC_16_layerscaffemodel_url: http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodellicense:see http://www.robots.ox.ac.uk/~vgg/research/very_deep/caffe_version:trainedusingacustomCaffe-basedframework

VGGisaconvolutional neuralnetworkmodelproposedbyK.Simonyan andA.ZissermanfromtheUniversityofOxfordinthepaper“VeryDeepConvolutional NetworksforLarge-ScaleImageRecognition”.Themodelachieves92.7%top-5testaccuracyinImageNet ,whichisadatasetofover14millionimagesbelongingto1000classes.

K.Simonyan,A.ZissermanVeryDeepConvolutional NetworksforLarge-ScaleImageRecognitionarXiv technicalreport,2014https://arxiv.org/pdf/1409.1556.pdf

Source:https://www.cs.toronto.edu/~frossard/post/vgg16/

Modelandpre-trainedparametersforVGG16inTensorFlowhttps://www.cs.toronto.edu/~frossard/post/vgg16/

Keras:https://keras.io/https://keras.io/applications/DeepLearninglibraryforTheano andTensorFlowKeras isahigh-levelneuralnetworksAPI,writteninPythonandcapableofrunningontopofeither TensorFlow or Theano.Itwasdevelopedwithafocusonenablingfastexperimentation. Beingabletogofromideatoresultwiththeleastpossibledelayiskeytodoinggoodresearch.

Models for image classificationwith weights trained on ImageNet

GoogLeNethttps://research.google.com/pubs/pub43022.html

Weproposeadeepconvolutional neuralnetworkarchitecturecodenamedInceptionthatachievesthenewstateoftheartforclassificationanddetectionintheImageNet Large-ScaleVisualRecognitionChallenge2014(ILSVRC2014).Themainhallmarkofthisarchitectureistheimprovedutilizationofthecomputingresourcesinsidethenetwork.Byacarefullycrafteddesign,weincreasedthedepthandwidthofthenetworkwhilekeepingthecomputationalbudgetconstant.Tooptimizequality,thearchitecturaldecisionswerebasedontheHebbian principleandtheintuitionofmulti-scaleprocessing.Oneparticularincarnationofthisarchitecture,GoogLeNet,a22layersdeepnetwork,wasusedtoassessitsqualityinthecontextofobjectdetectionandclassification.

Going Deeper with ConvolutionsChristianSzegedy, Wei Liu, Yangqing Jia, PierreSermanet, ScottReed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, AndrewRabinovicharXiv technicalreport,2014https://arxiv.org/pdf/1409.4842.pdf

GoogLeNethttp://www.robots.ox.ac.uk/~vgg/research/very_deep/

GooglLeNet inCaffehttps://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet

GooglLeNet inKerashttps://gist.github.com/joelouismarino/a2ede9ab3928f999575423b9887abd14http://joelouismarino.github.io/blog_posts/blog_googlenet_keras.html

Deep Learning: Detección de Armas en Video

R. Olmos, S. Tabik, F. Herrera (UGR) Video:Skyfall,201252

Deep Learning: Detección de Armas en Video

R. Olmos, S. Tabik, F. Herrera (UGR) Video:Skyfall,2012

Underwaterimagepatchesofcorals

Cellclassificationfromcervixsmears Fingerprintidentification/classificationEjemplosdeaplicaciones

DeepLearningLectura:RecentOverview

http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html

LeCum,Bengio yHinton,2015:“Ultimately,majorprogressinartificialintelligencewillcomeaboutthroughsystemsthatcombinerepresentationlearningwithcomplexreasoning.Althoughdeeplearningandsimplereasoninghavebeenusedforspeechandhandwritingrecognitionforalongtime,newparadigmsareneededtoreplacerule-basedmanipulationofsymbolicexpressionsbyoperationsonlargevectors”

Losavancesenelconocimientodelcerebroyelrazonamientohumanopermitirándiseñarnuevosparadigmasderepresentaciónyrazonamientocomplejo.

Ciencias de Datos y Deep Learning: Neuronas artificiales ... · Ciencia de Datos y Deep Learning:...

Documents

Transcript of Ciencias de Datos y Deep Learning: Neuronas artificiales ... · Ciencia de Datos y Deep Learning:...

PETREOS ARTIFICIALES

Neuronas espejo en la rehabilitación

Trabajo Islas Artificiales

REDES NEURONALES ARTIFICIALES

Satelites artificiales definitiva

materiales petreos artificiales

Tarea 9 Neuronas yNneurotransmisores. Fund. Neurociencias

musculos artificiales

CEREBRO Y NEURONAS

Satelites artificiales de venezuela

LA NEURONA CANDELABRO - zaguan.unizar.es · El neocórtex contiene dos clases de neuronas: las neuronas piramidales y las neuronas no piramidales . Neuronas piramidales . ... representan

Satelites Artificiales

planta artificiales

APLICACION NEURONAS ARTIFICIALES

Exositemas artificiales

Redes de neuronas artificiales isasi, galvan

Alergias por colorantes artificiales

Neuronas y neurotransmisores tarea 9

Avances en neurociencias. Regeneración de Neuronas

IA-redes Neuronales Artificiales