Secular Technological Tailwinds
-
Upload
jquant -
Category
Technology
-
view
172 -
download
0
Transcript of Secular Technological Tailwinds
Trends vs. Predictions
1
“ A prediction or forecast is a statement about the way
things will happen in the future (…) ” Wikipedia
“Trend: To extend in a general direction : follow a general course ”
Webster Dictionary
4
• Concerned about learning theprocess, the interactions and theemerging laws.
Trends Predictions
• Concerned about being right.
5
Yes, we may use trends to make predictions.
Steve Ballmer, former chief executive of Microsoft, on the iPhone shortly
after Steve Jobs announced it. Ballmer went on to promote Microsoft's cheaper phones,
saying "right now we're selling millions and millions and
millions of phones a year. Apple is selling zero."
Trends vs. Predictions
7
“ A prediction or forecast is a statement about the way
things will happen in the future (…) ” Wikipedia
“Trend: To extend in a general direction : follow a general course ”
Webster Dictionary
Trends vs. Predictions
8
“ A prediction or forecast is a statement about the way
things will happen in the future (…) ” Wikipedia
“Trend: To extend in a general direction : follow a general course
” Webster Dictionary
Trends vs. Predictions
9
“ A prediction or forecast is a statement about the way
things will happen in the future (…) ” Wikipedia
“Trend: To extend in a general direction : follow a general course
” Webster Dictionary
This is what this presentation is about.
11
Dionisio [email protected]
https://plus.google.com/+DionisioChiuratto/
Facebook: Dionisio Chiuratto
Twitter: josaum
YO!: DIONISIO
17
Cloud for what?!
IT Overall Hardware Costs are Decreasing...
but change in IT is very expensive.
And it takes lots of time.
19
Cloud for what?!
So came outsourcing.
Outsourcing was the first attempt to have IT-as-a-service.Whenever there was change, the contractor adapted (or should have).
“If IT isn’t our core business, why have it in-house?”
So the servers, databases, developers and support team were gone.
20
Cloud for what?!
So came outsourcing.
Outsourcing was the first attempt to have IT-as-a-service.Whenever there was change, the contractor adapted (or should have).
“If IT isn’t our core business, why have it in-house?”
So the servers, databases, developers and support team were gone.
And so there was a bug...... and the company was unable to invoice customers for a week.
21
Cloud for what?!
There was (and there is) a need for reliability in IT, but with tight budgets and control over the process.
There was (and there is) a need for quickly scaling storage, processing, networking and licensing.
22
Cloud for what?!
I don’t want someone to take care of my ERP. I have the right guys.
I want someone to take the burden of hosting the infrastructure for my ERP.
I don’t want someone to take care of my DB. I have my DBA.
I want someone to take the burden of hosting the infrastructure for my DB.
And updating them.And taking care of their maintanance, power shutdowns, backups...
23
Cloud for what?!
The cloud concept can be understood as a evolution from the IT outsourcing desires.
Outsourcing
Infrastructure-as-a-Service (IaaS)
Platform-as-a-Service (PaaS)
Software-as-a-Service (SaaS)
Monitoring-as-a-Service (MaaS)
Communication-as-a-Service (CaaS)
AWS (EC2, RDS, ...)
AWS – Elastic Beamstalk
Office 365
Datadog
Skype
24
Got it. Now why so much buzz about it?
Conceptually, both for users and enterprises, the cloud allowed a detachment from the physical IT.
Before After
My computer with:my hardware, my apps, mymusic, my documents, mydata, my contacts, my e-mails and my games.
There is this place called“Cloud” which holdseverything, no matter thedevice I’m using to accessit.
25
Got it. Now why so much buzz about it?
We just need a minute (or twenty) to getthrough the evolution of the...
Ok! Hold on about cloud computing!
26
Internet1.0
Read-Only WEB
The Shopping Cart
Company Page
Your own (html-coded) Personal Home Page!
31
Internet 2.0Social Internet / Collaborative
Open-sourceLike
ShareRepositories
Crowd-sourcingCrowd-funding
32
Internet 2.0PROSUMER
It became easy for anyone not only consume, but also produce content over the internet.
33
Got it. Now why so much buzz about it?
Everybody started producing contents, usingPaaS and SaaS.
Social media emerged.
Ok! We’re back with the cloud!
34
Got it. Now why so much buzz about it?
http://articles.economictimes.indiatimes.com/2012-05-27/news/31860969_1_instagram-largest-online-retailer-users
35
Got it. Now why so much buzz about it?
The cloud enabled us all to be creators in the internet.
The cloud enabled the mobile smart-devices widespread.
The cloud enabled companies to process and store HUGE amounts ofdata (more of that soon in the big data section)
The cloud enabled us all to connect with each other and share.
Without buying more hardware or dealing with any IT-specificproblems.
RESPECT THE BUZZ!
38
It’s all about context!
The cloud got us covered with the infrastructure, platform andsoftwares. Now other questions arise:
“Who’s in the picture?”Twitter Trending Topics
#hashtagApple Siri
Microsoft CortanaGoogle Now
39
It’s all about context!
People have been producing and interactingwith contents over the internet. We’veevolved so much!
...
Meh.
40
The problem. Yes there is a problem.
The internet was made for people…Limited Time
Limited KnowledgeLimited Accuracy
Ambiguous Concepts…
50
Internet > 3.0URL = http://www.receitas.com.br/Palmirinha/Batata.html
Page with Recipes
Why only define addresses for pages?Pages are for people…
52
Internet > 3.0
URI http://www.productontology.org/id/Potato_salad
URI http://purl.org/goodrelations/v1#ProductOrService
URI http://purl.org/goodrelations/v1#closes
54
Internet > 3.0The Semantic Web has the purpose of conecting and relating data.
The Semantic Web is based on LINKED DATA.
The Semantic Web is MACHINE READABLE.
55
RDF triples?
Sorocaba , belongs to , SP stateSubject Predicate Object
SP state, has climate , Subtropical
RDF – Resource Description Framework
Subject Predicate Object
56
SPARQL
SELECT ?cityWHERE {
?city relationship:belongs ?state?state relationship:hasClimate “Subtropical”
}
List all cities in the state of SP that has the Subtropical climate.
How to make queries in a knowledge graph.
59
Ontologies
Explicit representation of knowledge –definitions and relationships
Formal 1st order logic
XML-based
Object-Oriented(?!)
OWL – Ontology Web Language
64
Ok, so far we’ve seen that:
• Cloud computing gave us a smart(cheap?) andscalable way of using platforms, softwares andinfrastructure;
• We are producing A LOT of content through;
• Semantic web is giving context, meaning andrelationship to data.
• Semantic web makes data machine-readable.
65
Now...The machines! (and other things)
As the cloud concept is interesting because wedetached from physical devices...
...the semantic web is interesting because it willdetach us from the graphical user interfaces. (andfrom browsing the internet)
66
Now...The machines! (and other things)
• Hardware costs are falling. (and are more powerfull)
• M2M (machine to machine) communication is growing fast
• Advanced Software
• Cloud Services
http://www.microsoft.com/en-us/server-cloud/internet-of-things.aspx
68
Dev
ices
, se
nso
rs, c
ame
ras,
....
KnowledgeBase
Processing Results
Dev
ices
, se
nso
rs, c
ame
ras,
....
Cloud Services (IaaS, PaaS, SaaS)
Internet...of things
69
A lot of things connected throughthe internet, talking to each otherand living within the world aroundthem, creates data...a lot of data...
71http://www.binarylaw.co.uk/2010/11/08/the-hype-cycle/
The hype cycle! (or the buzz evolution chart)
Take my money!!!
Meh. Doens’t work.
Interesting. Shouldanalyse the pros andcons!
72
What is Big Data?
• Companies have been producing and storing data since the ERP’s, CRM’s, BI’s, WMS’s, etc. adoption waves.
• Users have been producing content, giving away personal informations, and (lettingbig cloud tech companies) storing it since the PROSUMER feature of the web >= 2.0.
• With the widespread of IoT, billions of new devices are starting to produce and storedata.
75
What is Big Data?
That’s Big Data. Really.
Just tons and tons of data that are now being created and stored.
76
So...?
Big Data by itself is nearly useless.There were important developments on processing and storing peta/hexa/yota-scale
data.But as for business and individuals, there were very few outcomes.
Nice solutions were deployed: visualization of twitter hash-tags in real time and geo-located. Truck-tracking, health-care experiments on collecting patient’s data, and so on.
77
So...?
But the killer application for Big Data is yet to come.
And other buzz-words (Analytics? Deep learning?) might take the merit for themselves.
80
Insights do not emerge by themselves.They need algorithms for Optimization, AI, Data Mining, Graphs, ...
And these requires processing power.
84
Where is the 100GHz processor?
CPU’s have been advancing in speed since they were born.
There are a number of factors resulting in the overall speed of a CPU, but by far themost straightforward is the clock speed (the MHz, GHz !)
85
Where is the 100GHz processor?
Now it is easy to aknowledge that companies such as Intel or AMD are no longerincreasig the clock speed.
A barrier was hit around 3-4GHz. Why is that?
88
But...computing power continues to expand, right?
Yes. There are two major avenues of computing power growth.
New Materialsand Hardwares
Parallelism
89
New Materials andHardwares
Parallelism
Post-SI Computing
• Graphene• Silicene• Quantum Computing
materials (nanotechnology)
Coarse-Grain Parallelism
• Distributed Computing• Hadoop
Fine-Grain Parallelism
• Multi-Core Processing• Many-Core Processing
Coarse-Grain Parallelism
• Distributed Computing• Hadoop
Fine-Grain Parallelism
• Multi-Core Processing• Many-Core Processing
90
New Materials andHardwares
Post-SI Computing
• Graphene• Silicene• Quantum Computing
materials (nanotechnology)
Under Research! In Stock!
Parallelism
91
Parallelism
Coarse-Grain Parallelism
• Distributed Computing• Hadoop
Master
Slave Slave ... Slave
TaskTask
Task
MAP
Master
REDUCEPartial Ans.Partial Ans.
Partial Ans.
92
Parallelism
Coarse-Grain Parallelism
• Distributed Computing• Hadoop
How many letters in this sentence?
Slave Slave Slave
Counts: 7 Counts: 9 Counts: 12
Master
Sum Reduce: 7 + 9 + 12 = 28
93
Parallelism
Fine-Grain Parallelism
• Multi-Core Processing
http://www.techpowerup.com/reviews/Intel/Core_i7-5960X_5930K_5820K_Comparison/2.html
94
Parallelism
Fine-Grain Parallelism
• Many-Core Processing
http://www.techpowerup.com/reviews/Intel/Core_i7-5960X_5930K_5820K_Comparison/2.html
Massively Parallel Processing Paradigm
95
Parallelism
Fine-Grain Parallelism
• Many-Core Processing
http://www.techpowerup.com/reviews/Intel/Core_i7-5960X_5930K_5820K_Comparison/2.html
Massively Parallel Processing Paradigm
96
Parallelism
Fine-Grain Parallelism
• Many-Core Processing
http://www.techpowerup.com/reviews/Intel/Core_i7-5960X_5930K_5820K_Comparison/2.html
Massively Parallel Processing Paradigm
97
Paralellism is the way to go on speeding up applications and data processing.
All major tech companies are using parallelism (GPU’s, Xeon Phi’s, Hadoop) toanalyse, process and store data.
Parallelism is the way to go with Big Data.
98
Paralellism is the way to go on speeding up applications and data processing.
All major tech companies are using parallelism (GPU’s, Xeon Phi’s, Hadoop) toanalyse, process and store data.
Parallelism is the way to go with Big Data.
http://www.networkworld.com/article/2167576/tech-primers/hadoop---gpu--boost-performance-of-your-big-data-project-by-50x-200x-.html
99
But even with lots of data and the correct processingapproach through parallelism,
one question remains:
102
An algorithm is a set of instructions to be performed.
To be honest, all codes are algorithms by definition.
103
There are, however, non-trivial sets of instructions (i.e. algorithms) that aims tosolve specific problems.
105
What is the shortest path between two nodes in a directed graph?
Graphs 101:
Vertices (nodes) – Circles
Edges – Lines connecting them
If the lines have arrows – Directed Graph (Digraph)
If the lines doesn’t have arrows – Undirected Graph
The numbers over the edges indicates the weight of each one.
109
What is the shortest path between two nodes in a directed graph?
Ok! There’s an app for that! algorithm for that!
111
Dijkstra Algorithm (1956)
Shortest path is considered solved by the academia.
The computational time complexity is
This means that if you run the algorithm in your computer with say, 5 vertices, and it takes 10s to run.
When you run it with 10 vertices (2x), it is guaranteed that in the worst case it will take 40s (4x) to run.
113
What is the shortest path in a directed graph, leaving from node A, passingthrough all vertices and returning to A? (TSP = Travelling Salesman Problem)
A A
http://mathworld.wolfram.com/TravelingSalesmanProblem.html
114
This is called NP-Complete. No one knows the time complexity of the problem, and there is no known analytical solution to it.
A A
http://mathworld.wolfram.com/TravelingSalesmanProblem.html
119
Number of Ed. Institutions in the city Ipads sold (x100)
1 1
2 4
3 4
… …
10 20
Trying to learn the law relating Ipad sales with educational institutions
121
Here, learn is the process in which we seek the best red-dotted line.
Because once we find it, we’ll know a mathematical formula relating the two variables.
Best can mean anything you like. In most cases, we are looking for the best fit, i.e. the line that minimizes the error across the training set.
122
Looking at past data, applying some black-box “learning algorithm” and inferring a mathematical relationship between variables.
The trainning phase is the most time and resource-consuming part of the process.
124
Once we’ve learned the relationship, let’s say it is:
Ipads sold = 2*ed.Inst
If now we want to predict the number of ipads sold in a city with 20 educational institutions, we only need to do a few operations.
126
That’s how Siri (and Cortana, and Google Now) talks to you.
They’ve been trained for long hours on powerful hardwares, and they now can rely on the smartphone hardware to execute the algorithm and “understand” what you meant with “I am hungry”.
127
Trainning
• Heavyweight• Time and resource-
consuming• Seeks minimize
errors and maximize generality
• Take place in huge processing clouds
Processing (predicting, classifying, …)
• Lightweight (relative)• Real-time (or almost)• Only apply the pre-
determined operations• Might take place even
in mobile devices
128
Trainning
• Heavyweight• Time and resource-
consuming• Seeks minimize
errors and maximize generality
• Take place in huge processing clouds
Processing (predicting, classifying, …)
• Lightweight (relative)• Real-time (or almost)• Only apply the pre-
determined operations• Might take place even
in mobile devices
Both processes can be severely sped-up with parallel computing.
133http://www.telegraph.co.uk/motoring/road-safety/10570935/Autonomous-cars-is-this-the-end-of-driving.html
Self-driving cars
140http://www.forbes.com/sites/roberthof/2015/01/31/now-even-artificial-intelligence-gurus-fret-that-ai-will-steal-our-jobs/
“The U.S. took 200 years to get from 98% to 2% farming employment,”
“Over that span of 200 years we could retrain the descendants of farmers.”
“With this technology today, that transformation might happen much faster,”
Self-driving cars, he suggested could quickly put 5 million truck drivers out of work.
Andrew Ng
143
Algorithms will be the new standard.
For people’s jobs.
For companies' competition.
For countries’ sovereignty.
147
14 trillionDollars added to the global economy by 2030
http://www.accenture.com/SiteCollectionDocuments/PDF/Accenture-Industrial-Internet-of-Things-Positioning-Paper-Report-2015.PDF