Ciencias de Datos y Deep Learning: Neuronas artificiales ... · Ciencia de Datos y Deep Learning:...

CienciasdeDatosyDeepLearning:Neuronasartificialesparaaprender

Francisco Herrera

Ciencia deDatosyDeepLearning:Neuronasartificialesparaaprender

2

Historyof

DataScience

BigDataDeepLearning


3

En1943WarrenMcCullochyWalterPitts presentaronsumodelodeneuronaartificial,ydescribieronlosprimerosfundamentosdeloquesellamaríaposteriormenteredesneuronales.

W.McCullochandW.Pitts(1943).ALogicalCalculusofideasImmanentinNervousActivity.BulletinofMathematicalBiophysics5:115-133.

RedesNeuronales


4

RedesNeuronales

Unaredneuronalseproponecomounsistemainteligentequeimitaalsistemanerviosoyalaestructuradelcerebro,peromuydiferenteentérminosdesuestructurayescala.Aligualquelasneuronasbiológicas,lasneuronasartificialesseinterconectanparaformarredesdeneuronasartificiales.Cadaneuronaartificialutilizaunafunciónprocesamiento queagregalainformacióndeconexionesdeentradaconotrasneuronalesartificiales,unafuncióndeactivaciónyunafuncióndetransferenciaparadarunasalidadelaneuronaensusconexionesdesalida.

Netj

x1

x2

xn

wj1

wj2

wjn

yj å=

×=×++×+×=n

iijinjnjjj xwxwxwxwNet

12211 !

÷÷ø

öççè

æ-×=-= å qq ijijjj xwfNetfy )(


5

RedNeuronalClásica:Backpropagation

Credits:TheEvolutionofNeuralLearningSystems:ANovelArchitectureCombiningtheStrengthsofNTs,CNNs,andELMs.NMartinel, CMicheloni…- IEEESMCMagazine,2015- ieeexplore.ieee.org


6

• Múltiples capasocultas• Aprendizajejerárquico

• Característicascadavezmáscomplejas• Muybuencomportamiento enmútiplesdominios:Vision,Audio,…

DeepArchitecture(Redes Neuronales conmuchas capas)


7

UnsupervisedDeepArchitecture: Autoencoder

An autoencoder neuralnetworkisanunsupervisedlearningalgorithmthatappliesbackpropagation,settingthetargetvaluestobeequaltotheinputs.

Theaimofanautoencoder istolearnarepresentation(encoding)forasetofdata,typicallyforthepurposeofdimensionalityreduction.

Andrew Ng

Unsupervised feature learning with a neural network

x4

x5

x6

+1

Layer1

Layer2

x1

x2

x3

x4

x5

x6

x1

x2

x3

+1

Layer3

a1

a2

a3

Andrew Ng

x4

x5

x6

+1

Layer1

Layer2

x1

x2

x3

+1

a1

a2

a3

New representation for input.

Unsupervised feature learning with a neural network


10

DeepArchitecture: Autoencodern BreastCancerWisconsin(Diagnostic)DataSet

Autoencoder (f(x)=x) (unasolacapainternade3neuronasy1000"epochs".WDBC(569instanciascon32atributosdeentrada)

Credito:D.Charte


11

http://arxiv.org/abs/1312.5602

DeepMind:Start up-2011Demis Hassabis,Shane Legg yMustafa Suleyman


12

JuegosArcade(Breakout)


13

DeepLearningRetos enla“pintura”


14

http://www.deepart.io/

DeepLearning: DeepART


15

Ejemplos delresultado deDeepART


vanGoth


16

Ejemplos delresultado deDeepART



17

ModelodeDLutilizadoydescripcióndelametodología

http://arxiv.org/abs/1508.06576


18

Supervised DeepLearning:Convolutional NeuralNetworks

http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg


19

Supervised DeepLearning:Convolutional NeuralNetworks

Eachmoduleconsistsofaconvolutional layerandapoolinglayer.

Typicallytriestocompresslargedata(images)intoasmallersetofrobustfeatures,basedonlocalvariations.

Basicconvolutioncanstillcreatemanyfeatures.

CNNshavebeenfoundhighlyeffectiveandbeencommonlyusedincomputervisionandimagerecognition.

DeepLearning

Convolutional steps

https://github.com/rasbt/python-machine-learning-book/blob/master/faq/difference-deep-and-normal-learning/convolution.png

http://neuralnetworksanddeeplearning.com/chap6.html

DeepLearningDigitRecognizerandConvolutional NN

Convolutional neuralnetworks usethreebasicideas:localreceptivefields,sharedweights,andpooling.

Localreceptive fields:Tobemoreprecise,eachneuroninthefirsthiddenlayerwillbeconnectedtoasmallregionoftheinputneurons,say,forexample,a5×5region,correspondingto25inputpixels.So,foraparticularhiddenneuron,wemighthaveconnectionsthatlooklikethis:



Localreceptive fields:

24×24 neurons

28×28 input image



Shared weights andbiases:thesame weightsandbiasforeachofthe24×24hiddenneurons(sigmoide function)

Themapfromtheinputlayertothehiddenlayerafeaturemap.



Shared weights andbiases:

In the example shown, there are 3 feature maps.

If we have 20 feature maps that's a total of 20×26=520 parameters defining the convolutional layer. By comparison, suppose we had a fully connected first layer, with 784=28×28 input neurons, 30 hidden neurons, 23,550 parameters.



Pooling layers: thepoolinglayersdoissimplifytheinformationintheoutputfromtheconvolutional layer,onecommonprocedureforpoolingisknownasmax-pooling,inthe2x2regioninput.


Pooling layers:



The20imagescorrespondto20differentfeaturemaps


28

Credits:https://www.datarobot.com/blog/a-primer-on-deep-learning/

Alaizquierda,losdígitosdeentradasinprocesar.Aladerecha,representacionesgráficasdelascaracterísticasaprendidas.Enesencia,laredaprendea"ver"líneasybucles.

DeepLearning:MNISTdata

Asnapshotofimagepre-processingforconvolutional neuralnetworks:casestudyofMNISTSiham Tabik,DanielPeralta,AndrésHerrera-Poyatos,FranciscoHerreraInternationalJournal ofComputational Intelligence Systems,Vol.10(2017)555–56899.72accuracy

DeepLearningDigitRecognizerandpreprocessing

CNNmodels:LeNet,Network3,DropConnet

Preprocessing andaugmentation

Ensembles


CNNmodels:LeNet [4]

[4]Yann LeCun,Leon Bottou,Yoshua Bengio,andPatrickHaffner.Gradient-basedlearningappliedtodocumentrecognition.Proc.IEEE,86(11):2278– 2324,1998.


CNNmodels:Network3[13]

[13]MichaelANielsen.Neuralnetworksanddeep learning.URL:http://neuralnetworksanddeeplearning.com,2015.


CNNmodels:DropConnet [14]

[14]LiWan,Matthew Zeiler,Sixin Zhang,Yann LCun,andRobFergus.Regularizationofneuralnetworksusingdropconnect.InProceedingsofthe30thInternationalConferenceonMachineLearning(ICML- 13),pages 1058–1066,2013.



Translation.Theimageistranslatedanumberofpixelstowardagivendirection.

Centering.Toeliminatewhitecolumns/rows,andtoresize by scaling.

Rotation.Theimageisrotatedtoagivenangleθ.

Elastic deformation.Image pixels areslightly movedinrandomdirections,keepingtheimage’stopology.



Ensembles



The28handwrittendigitsmisclassifiedbyensemble-5ofNetwork3Thedigitbetween()representsthecorrectclass.The13digitslabeledwithasterisksarealsomisclassifiedbyDropConnect

Deep Learning: MNIST data Asnapshotofimagepre-processingforconvolutional neuralnetworks:casestudy ofMNISTSiham Tabik,DanielPeralta,AndrésHerrera-Poyatos,FranciscoHerreraInternationalJournal ofComputational Intelligence Systems,Vol.10(2017)555–56899.72accuracy

The28handwrittendigitsmisclassifiedbyensemble-5ofDropConnetThedigitbetween()representsthecorrectclass.The13digitslabeledwithasterisksarealsomisclassifiedbyNetwork3



The13handwrittendigitsmisclassifiedbyensemble-5ofDropConnetandNetwork3

DeepLearningMXNet

http://mxnet.io/get_started/index.html

DeepLearningMXNet

http://mxnet.io/how_to/finetune.html

DeepLearningLearningfromscratchvs fine-tuning(GoogLeNet yVGG-16)

VGG16http://www.robots.ox.ac.uk/~vgg/research/very_deep/

https://gist.github.com/ksimonyan/211839e770f7b538e2d8#file-readme-md

##Informationname:16-layermodelfromthearXiv paper:"VeryDeepConvolutional NetworksforLarge-ScaleImageRecognition"caffemodel:VGG_ILSVRC_16_layerscaffemodel_url: http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodellicense:see http://www.robots.ox.ac.uk/~vgg/research/very_deep/caffe_version:trainedusingacustomCaffe-basedframework

VGGisaconvolutional neuralnetworkmodelproposedbyK.Simonyan andA.ZissermanfromtheUniversityofOxfordinthepaper“VeryDeepConvolutional NetworksforLarge-ScaleImageRecognition”.Themodelachieves92.7%top-5testaccuracyinImageNet ,whichisadatasetofover14millionimagesbelongingto1000classes.

K.Simonyan,A.ZissermanVeryDeepConvolutional NetworksforLarge-ScaleImageRecognitionarXiv technicalreport,2014https://arxiv.org/pdf/1409.1556.pdf


VGG16

Source:https://www.cs.toronto.edu/~frossard/post/vgg16/

Modelandpre-trainedparametersforVGG16inTensorFlowhttps://www.cs.toronto.edu/~frossard/post/vgg16/

Keras:https://keras.io/https://keras.io/applications/DeepLearninglibraryforTheano andTensorFlowKeras isahigh-levelneuralnetworksAPI,writteninPythonandcapableofrunningontopofeither TensorFlow or Theano.Itwasdevelopedwithafocusonenablingfastexperimentation. Beingabletogofromideatoresultwiththeleastpossibledelayiskeytodoinggoodresearch.

Models for image classificationwith weights trained on ImageNet


VGG16


GoogLeNethttps://research.google.com/pubs/pub43022.html

Weproposeadeepconvolutional neuralnetworkarchitecturecodenamedInceptionthatachievesthenewstateoftheartforclassificationanddetectionintheImageNet Large-ScaleVisualRecognitionChallenge2014(ILSVRC2014).Themainhallmarkofthisarchitectureistheimprovedutilizationofthecomputingresourcesinsidethenetwork.Byacarefullycrafteddesign,weincreasedthedepthandwidthofthenetworkwhilekeepingthecomputationalbudgetconstant.Tooptimizequality,thearchitecturaldecisionswerebasedontheHebbian principleandtheintuitionofmulti-scaleprocessing.Oneparticularincarnationofthisarchitecture,GoogLeNet,a22layersdeepnetwork,wasusedtoassessitsqualityinthecontextofobjectdetectionandclassification.

Going Deeper with ConvolutionsChristianSzegedy, Wei Liu, Yangqing Jia, PierreSermanet, ScottReed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, AndrewRabinovicharXiv technicalreport,2014https://arxiv.org/pdf/1409.4842.pdf


GoogLeNethttp://www.robots.ox.ac.uk/~vgg/research/very_deep/

GooglLeNet inCaffehttps://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet

GooglLeNet inKerashttps://gist.github.com/joelouismarino/a2ede9ab3928f999575423b9887abd14http://joelouismarino.github.io/blog_posts/blog_googlenet_keras.html


52

Deep Learning: Detección de Armas en Video

R. Olmos, S. Tabik, F. Herrera (UGR) Video:Skyfall,201252


53

Deep Learning: Detección de Armas en Video

R. Olmos, S. Tabik, F. Herrera (UGR) Video:Skyfall,2012


54

Underwaterimagepatchesofcorals

Cellclassificationfromcervixsmears Fingerprintidentification/classificationEjemplosdeaplicaciones


55

DeepLearningLectura:RecentOverview

http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html


56

LeCum,Bengio yHinton,2015:“Ultimately,majorprogressinartificialintelligencewillcomeaboutthroughsystemsthatcombinerepresentationlearningwithcomplexreasoning.Althoughdeeplearningandsimplereasoninghavebeenusedforspeechandhandwritingrecognitionforalongtime,newparadigmsareneededtoreplacerule-basedmanipulationofsymbolicexpressionsbyoperationsonlargevectors”

Losavancesenelconocimientodelcerebroyelrazonamientohumanopermitirándiseñarnuevosparadigmasderepresentaciónyrazonamientocomplejo.

Ciencias de Datos y Deep Learning: Neuronas artificiales ... · Ciencia de Datos y Deep Learning:...

Documents

Transcript of Ciencias de Datos y Deep Learning: Neuronas artificiales ... · Ciencia de Datos y Deep Learning:...