UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1:...
Transcript of UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1:...
![Page 1: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/1.jpg)
UVACS6316/CS4501–Fall2015:
MachineLearning
Lecture1:IntroducBon
Dr.YanjunQi
UniversityofVirginiaDepartmentof
ComputerScience
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
1
![Page 2: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/2.jpg)
Welcome
• CS6316/4501MachineLearning– MoWe3:30pm-4:45pm,– OlssonHall120
• hOp://www.cs.virginia.edu/yanjun/teach/2016f
• YourUVAcollab:Course6316-4501page
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
2
![Page 3: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/3.jpg)
Today
q CourseLogisBcsq Mybackgroundq Basicsandroughcontentplanq ApplicaUonandHistory
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
3
![Page 4: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/4.jpg)
CourseStaff
• Instructor:Prof.YanjunQi– QI:/chee/– Youcancallme“professor”,“professorJane”,“professorQi”;
• TAofficehours:Mon&Wed5:30pmpm-6:30pm@Rice504
• Myofficehours:Mon5pm-6pm@Rice503
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
4
![Page 5: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/5.jpg)
CourseLogisUcs
• Courseemaillisthasbeensetup.Youshouldhavereceivedemailsalready!
• Policy,thegradewillbecalculatedasfollows:– Assignments(55%,Sixtotal,each~9%)– Quizzes/ExamSamplePracUces(5%)– Midtermexam(20%)– Finalexam(20%)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
5
![Page 6: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/6.jpg)
CourseLogisUcs• Midterm:Oct,75minsinclass• Final:Dec,75minsinclass
• Sixassignments(each9%to10%)– Threeextensiondayspolicy(checkcoursewebsite)
• In-classquizzes/ExamsamplepracUce(total5%)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
6
![Page 7: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/7.jpg)
CourseLogisUcs• Policy,
– HomeworkshouldbesubmiOedelectronicallythroughUVaCollab
– Homeworkshouldbefinishedindividually– Dueatmidnightontheduedate
– Inordertopassthecourse,theaverageofyourmidtermandfinalmustalsobe"pass".
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
7
![Page 8: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/8.jpg)
LateHomeworkPolicy
• EachstudenthasthreeextensiondaystobeusedathisorherowndiscreUonthroughouttheenUrecourse.Yourgradeswouldbediscountedby15%perdaywhenyouusethese3latedays.Youcouldusethe3daysinwhatevercombinaUonyoulike.Forexample,all3dayson1assignment(foramaximumgradeof55%)or1eachdayover3assignments(foramaximumgradeof85%oneach).Aperyou'veusedall3days,youcannotgetcreditforanythingturnedinlate.
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
8
![Page 9: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/9.jpg)
CourseLogisUcs
• Textbooksforthisclassis:– NONE
• Myslides–ifitisnotmenBonedinmyslides,itisnotanofficialtopicofthecourse
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
9
![Page 10: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/10.jpg)
CourseLogisUcs
• BackgroundNeeded– Calculus,Basiclinearalgebra,BasicprobabilityandBasicAlgorithm
– StaUsUcsisrecommended.– Studentsshouldalreadyhavegoodprogrammingskills,i.e.pythonisrequiredforallprogrammingassignments
– Wewillreview“linearalgebra”and“probability”inclass
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
10
![Page 11: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/11.jpg)
Today
q CourseLogisUcsq Mybackground
q Basicsandroughcontentplanq ApplicaUonandHistory
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
11
![Page 12: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/12.jpg)
AboutMe• EducaUon:
– PhDfromSchoolofComputerScience,CarnegieMellonUniversity(@PiOsburgh,PA)in2008
– BSfromDepartmentofComputerScience,TsinghuaUniv.(@Beijing,China)
• MyaccentPATTERN:/l/,/n/,/ou/,/m/• Researchinterests:
– MachineLearning,BiomedicalapplicaBons
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
12
![Page 13: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/13.jpg)
AboutMe
• FiveYears’ofIndustryResearchLabinthepast:– 2008summer–2013summer,ResearchScienBstinITindustry(MachineLearningDepartment,NECLabsAmerica@Princeton,NJ)
– 2013Fall–Present,AssistantProfessor,ComputerScience,UVA
9/1/16
Industry + Academia
Dr.YanjunQi/UVACS6316-4501/f16
13
![Page 14: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/14.jpg)
Today
q CourseLogisUcsq Mybackgroundq BasicsandRoughcontentplanq ApplicaUonandHistory
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
14
![Page 15: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/15.jpg)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
15
• Biomedicine – Patient records, brain imaging, MRI & CT scans, … – Genomic sequences, bio-structure, drug effect info, …
• Science
– Historical documents, scanned books, databases from astronomy, environmental data, climate records, …
• Social media
– Social interactions data, twitter, facebook records, online reviews, …
• Business – Stock market transactions, corporate sales, airline traffic, …
• Entertainment – Internet images, Hollywood movies, music audio files, …
OUR DATA-RICH WORLD
![Page 16: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/16.jpg)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
16
• Data capturing (sensor, smart devices, medical instruments, et al.)
• Data transmission • Data storage • Data management • High performance data processing • Data visualization • Data security & privacy (e.g. multiple
individuals) • …… • Data analytics
¢ How can we analyze this big data wealth ? ¢ E.g. Machine learning and data mining
BIG DATA CHALLENGES
this course
e.g.HCI
e.g.cloudcompuUng
![Page 17: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/17.jpg)
BASICS OF MACHINE LEARNING
• �The goal of machine learning is to build computer systems that can learn and adapt from their experience.� – Tom Dietterich
• �Experience��in the form of available data examples (also called as instances, samples)
• Available examples are described with properties (data points in feature space X)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
17
![Page 18: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/18.jpg)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
18
e.g. SUPERVISED LEARNING
• Find function to map input space X to output space Y
• So that the difference between y and f(x)
of each example x is small.
Ibelievethatthisbookisnotatallhelpfulsinceitdoesnotexplainthoroughlythematerial.itjustprovidesthereaderwithtablesandcalculaUonsthatsomeUmesarenoteasilyunderstood…
x
y-1
InputX:e.g.apieceofEnglishtext
OutputY:{1/Yes,-1/No}e.g.IsthisaposiUveproductreview?
e.g.
![Page 19: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/19.jpg)
e.g. SUPERVISED Linear Binary Classifier
f x y
f(x,w,b) = sign(w x + b)
wx+b<0
CourtesyslidefromProf.AndrewMoore’stutorial
?
?
wx+b>0
denotes +1 point
denotes -1 point
denotes future points
?
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
19
![Page 20: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/20.jpg)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
20
• Training (i.e. learning parameters w,b ) – Training set includes
• available examples x1,…,xL • available corresponding labels y1,…,yL
– Find (w,b) by minimizing loss (i.e. difference between y and f(x) on available examples in training set)
(W, b) = argminW, b
Basic Concepts
![Page 21: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/21.jpg)
• Testing (i.e. evaluating performance on �future��points) – Difference between true y? and the predicted f(x?) on a
set of testing examples (i.e. testing set)
– Key: example x? not in the training set
• Generalisation:learnfuncUon/hypothesisfrompastdatainorderto“explain”,“predict”,“model”or“control”newdataexamples
9/1/16
Basic Concepts Dr.YanjunQi/UVACS6316-4501/f16
21
![Page 22: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/22.jpg)
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
22
• Loss function – e.g. hinge loss for binary
classification task
– e.g. pairwise ranking loss for
ranking task (i.e. ordering examples by preference)
• Regularization – E.g. additional information added on loss function to control model
Basic Concepts
![Page 23: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/23.jpg)
TYPICAL MACHINE LEARNING SYSTEM
9/1/16
Low-level sensing
Pre-processing
Feature Extract
Feature Select
Inference, Prediction, Recognition
Label Collection
Dr.YanjunQi/UVACS6316-4501/f16
23
Evaluation
![Page 24: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/24.jpg)
“BigData”ChallengesforMachineLearning
9/1/16
ü Largesizeofsamplesü Highdimensionalfeatures
Not the focus, will be covered in advanced-level course
Dr.YanjunQi/UVACS6316-4501/f16
24
![Page 25: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/25.jpg)
Large-Scale Machine Learning: SIZE MATTERS
25
9/1/16
• One thousand data instances
• One million data instances
• One billion data instances
• One trillion data instances
Thosearenotdifferentnumbers,thosearedifferentmindsets!!!
Dr.YanjunQi/UVACS6316-4501/f16
![Page 26: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/26.jpg)
BIG DATA CHALLENGES FOR MACHINE LEARNING
9/1/16
ThevariaUonsofbothX(feature,representaUon)andY(labels)arecomplex!
Most of this
course ü ComplexityofXü ComplexityofY
Dr.YanjunQi/UVACS6316-4501/f16
26
![Page 27: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/27.jpg)
TYPICAL MACHINE LEARNING SYSTEM
9/1/16
Low-level sensing
Pre-processing
Feature Extract
Feature Select
Inference, Prediction, Recognition
Label Collection
Data Complexity of X
Data Complexity
of Y
Dr.YanjunQi/UVACS6316-4501/f16
27
Evaluation
![Page 28: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/28.jpg)
UNSUPERVISED LEARNING :
[ COMPLEXITY OF Y ]
• No labels are provided (e.g. No Y provided)
• Find patterns from unlabeled data, e.g. clustering
9/1/16
e.g.clustering=>tofind�natural�groupingofinstancesgivenun-labeleddata
Dr.YanjunQi/UVACS6316-4501/f16
28
![Page 29: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/29.jpg)
STRUCTURAL OUTPUT LEARNING : [ COMPLEXITY OF Y ]
• Many prediction tasks involve output labels having structured correlations or constraints among instances
9/1/16
Manymorepossiblestructuresbetweeny_i,e.g.spaUal,temporal,relaUonal…
Thedogchasedthecat
APAFSVSPASGACGPECA…
TreeSequence GridStructured Dependency between Examples
Input
Output
CCEEEEECCCCCHHHCCC…
Dr.YanjunQi/UVACS6316-4501/f16
29
![Page 30: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/30.jpg)
Original Space Feature Space
�
�
�
STRUCTURAL INPUT : Kernel Methods [ COMPLEXITY OF X ]
Vectorvs.RelaUonaldata
e.g.Graphs,Sequences,3Dstructures,
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
30
![Page 31: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/31.jpg)
MORE RECENT: FEATURE LEARNING [ COMPLEXITY OF X ]
Deep Learning Supervised Embedding
9/1/16
Layer-wise Pretraining
Dr.YanjunQi/UVACS6316-4501/f16
31
![Page 32: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/32.jpg)
DEEP LEARNING / FEATURE LEARNING : [ COMPLEXITY OF X ]
Feature Engineering ü Most critical for accuracy ü Account for most of the computation for testing ü Most time-consuming in development cycle ü Often hand-craft and task dependent in practice
Feature Learning ü Easily adaptable to new similar tasks ü Layerwise representation ü Layer-by-layer unsupervised training ü Layer-by-layer supervised training 329/1/16
Dr.YanjunQi/UVACS6316-4501/f16
![Page 33: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/33.jpg)
339/1/16
Dr.YanjunQi/UVACS6316-4501/f16
![Page 34: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/34.jpg)
CourseContentPlanèFivemajorsecUonsofthiscourse
q Regression(supervised)q ClassificaUon(supervised)q Unsupervisedmodelsq Learningtheoryq Graphicalmodels
9/1/16 34
Dr.YanjunQi/UVACS6316-4501/f16
![Page 35: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/35.jpg)
9/1/16 35
hOp://scikit-learn.org/Dr.YanjunQi/UVACS6316-4501/f16
![Page 36: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/36.jpg)
Scikit-learnalgorithmcheat-sheet
9/1/16 36
hOp://scikit-learn.org/stable/tutorial/machine_learning_map/Dr.YanjunQi/UVACS6316-4501/f16
![Page 37: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/37.jpg)
9/1/16 37
Scikit-learn:Regression
LinearmodelfiOedbyminimizingaregularizedempiricallosswithSGD
Dr.YanjunQi/UVACS6316-4501/f16
LinearRegression+VariaUons
![Page 38: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/38.jpg)
Scikit-learn:ClassificaUon
9/1/16 38
Linearclassifiers(SVM,logisUcregression…)withSGDtraining.
approximatetheexplicitfeaturemappingsthatcorrespondtocertainkernelsTocombinethe
predicUonsofseveralbaseesUmatorsbuiltwithagivenlearningalgorithminordertoimprovegeneralizability/robustnessoverasingleesUmator.(1)averaging/bagging(2)boosUng
Dr.YanjunQi/UVACS6316-4501/f16
![Page 39: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/39.jpg)
9/1/16 39
BasicPCA
Bayes-NetHMM
Kmeans+GMM
UnsupervisedModelsDr.YanjunQi/UVACS6316-4501/f16
![Page 40: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/40.jpg)
Summary
• Thisisnotacourseaboutlearningtousetoolbox
• Wefocusonlearningprinciples,mathemaUcalformulaUon,algorithmdesignandlearningtheory.
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
40
![Page 41: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/41.jpg)
Today
q CourseLogisUcsq Mybackgroundq Basicsandroughcontentplanq ApplicaBonandHistory
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
41
![Page 42: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/42.jpg)
Whatcanwedowiththedatawealth?èREAL-WORLDIMPACT
§ Businessefficiencies§ ScienUficbreakthroughs§ Improvequality-of-life:§ healthcare,§ energysaving/generaUon,§ environmentaldisasters,§ nursinghome,§ transportaUon,§ …
9/1/16
MedicalImages
GenomicData
TransportaUonData
BraincomputerinteracUon(BCI)
Devicesensordata
Dr.YanjunQi/UVACS6316-4501/f16
42
![Page 43: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/43.jpg)
When to use Machine Learning (Adaptto/learnfromdata) ?
– 1. Extract knowledge from data – Relationships and correlations can be hidden within large
amounts of data – The amount of knowledge available about certain tasks is
simply too large for explicit encoding (e.g. rules) by humans
– 2. Learn tasks that are difficult to formalise – Hard to be defined well, except by examples, e.g., face
recognition
– 3. Create software that improves over time – New knowledge is constantly being discovered. – Rule or human encoding-based system is difficult to
continuously re-design �by hand�.
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
43
![Page 44: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/44.jpg)
MACHINE LEARNING IS CHANGING THE WORLD
9/1/16 Manymore!
Dr.YanjunQi/UVACS6316-4501/f16
44
![Page 45: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/45.jpg)
MACHINE LEARNING IN COMPUTER SCIENCE
• Machine learning is already the preferred approach for
– Speech recognition, natural language processing – Computer vision – Medical outcome analysis – Robot control …
• Why growing ? – Improved machine learning algorithms – Increased data capture, new sensors, networking – Systems/Software too complex to control manually – Demand to self-customization for user, environment, ….
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
45
![Page 46: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/46.jpg)
46
RELATED DISCIPLINES
• Artificial Intelligence • Data Mining • Probability and Statistics • Information theory • Numerical optimization • Computational complexity theory • Control theory (adaptive) • Psychology (developmental, cognitive) • Neurobiology • Linguistics • Philosophy
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
![Page 47: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/47.jpg)
WhatarethegoalsofAIresearch?
ArUfactsthatACTlikeHUMANS
ArUfactsthatTHINKlikeHUMANS
ArUfactsthatTHINKRATIONALLY
ArUfactsthatACTRATIONALLY
47From:M.A.Papalaskar
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
![Page 48: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/48.jpg)
Howcanwebuildmoreintelligentcomputer/machine?
• Ableto– perceivetheworld– understandtheworld
• Thisneeds– BasicspeechcapabiliUes– BasicvisioncapabiliUes– Language/semanUcunderstanding– Userbehavior/emoUonunderstanding– Abletothink??
9/1/16 48
Dr.YanjunQi/UVACS6316-4501/f16
![Page 49: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/49.jpg)
9/1/16 49
Howcanwebuildmoreintelligentcomputer/machine?
toservehumanbeings,and
fluentin"oversixmillionformsofcommunicaUon"
R2-D2andC-3PO
@StarWars–1977
Dr.YanjunQi/UVACS6316-4501/f16
![Page 50: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/50.jpg)
Howcanwebuildmoreintelligentcomputer/machine?
9/1/16 50
Jeopardy Game è Requires a Broad Knowledge Base
IBM Watson è an artificial intelligence computer system capable of answering questions posed in natural language developed in IBM's DeepQA project.
Dr.YanjunQi/UVACS6316-4501/f16
![Page 51: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/51.jpg)
Howcanwebuildmoreintelligentcomputer/machine?
51
Apple Siri / Amazon Echo è an intelligent personal assistant and knowledge navigator
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
![Page 52: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/52.jpg)
Howcanwebuildmoreintelligentcomputer/machine?: Objective Recognition / Image Labeling
Deep Convolution Neural Network (CNN) won (as Best systems) on �very large-scale��ImageNet competition 2012 / 2013 / 2014
89%, 2013
529/1/16
Dr.YanjunQi/UVACS6316-4501/f16
ImageNet:animagedatabaseorganizedaccordingtotheWordNetLSVRC:LargeScaleVisualRecogniUonChallengebasedonImageNet.
93%, 2014 [ training on 1.2 million images [X] vs. 1000 different word labels [Y] ]
![Page 53: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/53.jpg)
Howcanwebuildmoreintelligentcomputer/machine?: Objective Recognition / Image Labeling
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
53
- 2013,GoogleAcquiredDeepNeuralNetworksCompanyheadedbyUtoronto“DeepLearning”ProfessorHinton
- 2013,FacebookBuiltNewArUficialIntelligenceLabheadedbyNYU“DeepLearning”ProfessorLeCun
- 2016,Google'sDeepMinddefeatslegendaryGoplayerLeeSe-dolinhistoricvictory
![Page 54: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/54.jpg)
Detour:plannedprogrammingassignments
• HW:SemanUclanguageunderstanding(senUmentclassificaUononmoviereviewtext)
• HW:VisualobjectrecogniUon(labelingimagesabouthandwriOendigits)
• HW:AudiospeechrecogniUon(unsupervisedlearningbasedspeechrecogniUontask)
9/1/16 54
Dr.YanjunQi/UVACS6316-4501/f16
![Page 55: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/55.jpg)
TodayRecap
q CourseLogisUcsq Mybackgroundq Basicsandroughcontentplanq ApplicaUonandHistory
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
55
![Page 56: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/56.jpg)
Task
Next lesson: Machine Learning in a Nutshell
Representation
Score Function
Search/Optimization
Models, Parameters
9/1/16 56
MLgrewoutofworkinAIOp+mizeaperformancecriterionusingexampledataorpastexperience,Aimingtogeneralizetounseendata
Dr.YanjunQi/UVACS6316-4501/f16
Next lesson: Review of linear algebra and basic calculus
![Page 57: UVA CS 6316 / CS 4501 – Fall 2015 : Machine Learning ... · Machine Learning Lecture 1: IntroducBon Dr. Yanjun Qi University of Virginia Department of ... Machine Learning 9/1/16](https://reader030.fdocuments.in/reader030/viewer/2022040306/5ec900e46b6392450e6126d9/html5/thumbnails/57.jpg)
References
q Prof.AndrewMoore’stutorialsq Prof.RaymondJ.Mooney’sslidesq Prof.AlexanderGray’sslidesq Prof.EricXing’sslidesq hOp://scikit-learn.org/q HasUe,Trevor,etal.TheelementsofstaUsUcallearning.Vol.2.No.1.NewYork:Springer,2009.
q Prof.M.A.Papalaskar’sslides
9/1/16
Dr.YanjunQi/UVACS6316-4501/f16
57