Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things...
-
Upload
abzetdin-adamov -
Category
Data & Analytics
-
view
228 -
download
3
description
Transcript of Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things...
![Page 1: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/1.jpg)
Assoc. Prof. Abzetdin ADAMOV
Chair of Computer Engineering Department
http://ce.qu.edu.az/~aadamov
Shahdag, 29 November 2014
Latest Trends in Technology:
BigData Analytics, Virtualization, Cloud
Computing, Internet of Things (IoT)
![Page 2: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/2.jpg)
Content
• Why Data Mining in BigData?
• Internet Statistics
• BigData Infrastructure
• Web Crowlers for Web Analytics
• Natural Language Processing (NLP)
• Virtualization
• Introduction to Cloud Computing
• Introduction to Internet of Things (IoT)
![Page 3: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/3.jpg)
Digital Universe
volume of digital data
• 2008 – 480.000 petabytes (PB)
• 2009 – 800.000 PB
• 2010 – 1200 000 PB or 1.2 zettabyte (ZB)
• 2011 – 1.8 ZB
• 2012 – 2.7 ZB
• 2014 ~ 6.2 ZB
• Expected to reach 35 ZB by 2020
IDC's Digital Universe Study
![Page 4: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/4.jpg)
Big Measures for Big Data
• kilobyte (kB) 103 210
• megabyte (MB) 106 220
• gigabyte (GB) 109 230
• terabyte (TB) 1012 240
• petabyte (PB) 1015 250
• exabyte (EB) 1018 260
• zettabyte (ZB) 1021 270
• yottabyte (YB) 1024 280
![Page 5: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/5.jpg)
Why Data Grows so Fast?
Data sets gathered by ubiquity devices:
• Information-sensing mobile devices,
• Aerial sensory technologies (remote sensing),
• Software logs,
• Cameras,
• Microphones,
• Radio-frequency identification readers,
• wireless sensor networks
![Page 6: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/6.jpg)
Internet is Biggest Country
311
1155
1339
3016
0 1000 2000 3000 4000
million
USA
India
China
Internet
Population
9,62
9,97
17,07
514,45
0 200 400 600
million square km
USA
Canada
Russia
Internet
Area
![Page 7: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/7.jpg)
Internet Penetration
13 12,4
8,2
4,1
1,5 1,1 0,9 0,6 0,59 0,5 0,34 0,32 0,04 0,03 0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
EE LV RU LT BY UA KG AM KZ UZ GE MD AZ TM TJ
Inte
rne
t P
en
etr
ati
on
(%
)
Country Internet Codes
Note: Internet stats for December 2001 Avarage Internet usage ın the world 8% - 500 Million - 2001
![Page 8: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/8.jpg)
Foundations of the Web
34,7
18
15
10
5,2 53 2,7 2,3 2,2 1,8 1,8 1,3 0,9 0,8
02468
10121416182022242628303234363840
EE LV RU LT BY UA KG AZ AM KZ UZ GE MD TM TJ
Country Internet Codes
Inte
rnet
Pen
etr
ati
on
(%
)
Note: Internet stats for December 2004
![Page 9: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/9.jpg)
Foundations of the Web
65,6
59,4 59,2
29,127,1
18,216,2 14,7 13,8 12,3
8,8 7,8 6,6 5,8
1,4
0
5
10
15
20
25
30
35
40
45
50
55
60
65
70
EE LV LT BY RU AZ MD UA KG KZ UZ GE TJ AM TM
Country Internet Codes
Inte
rnet
Pen
etr
ati
on
(%
)
Note: Internet stats for September 2009 Avarage Internet usage in the world 21.9%
![Page 10: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/10.jpg)
Foundations of the Web
68,2
59,5
47,1 4644,1
40 39,3
34,1 33,930,9
28,3 26,8
9,2
1,6
0
5
10
15
20
25
30
35
40
45
50
55
60
65
70
EE LV LT AM BY AZ RU KG KZ UA MD GE UZ TJ TM
Country Internet Codes
Inte
rnet
Pen
etr
ati
on
(%
)
Note: Internet stats for March 2011 Avarage Internet usage ın the world 30.2%
![Page 11: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/11.jpg)
Foundations of the Web
78
71,1
65,160,6
5047,7 46 45 44,8
34,130,2
28,426
13
5
05
101520253035404550556065707580
EE LV LT AM AZ RU BY KZ MD UA UZ GE UZ TJ TM
Country Internet Code
Inte
rnet
Pen
etr
ati
on
(%
)
Note: Internet stats for June 2012 Avarage Internet usage ın the world 34.3%
http://www.internetworldstats.com
![Page 12: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/12.jpg)
Internet Penetration 79
74
68
54,2 53,3 53,3
46,9 45,5 43,4
39,2 36,5
33,7
21,7
14,5
7,2
0
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
EE LV LT AZ RU KZ BY GE MD AM UZ UA KG TJ TM
Inte
rne
t P
en
etr
ati
on
(%
)
Country Internet Codes
Note: Internet stats for March 2013 Avarage Internet usage ın the world 39% - 2,7 Billion - 2013
![Page 13: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/13.jpg)
Top 15 Most Popular Social Networking Sites | January 2014
1,310,000,000 - Estimated
Unique Monthly
Visitors | 2 - Compete
Rank
313,000,000 - Estimated
Unique Monthly Visitors | 24 -
Compete Rank
277,000,000 - Estimated
Unique Monthly Visitors | 44 -
Compete Rank
70,500,000 - Estimated
Unique Monthly Visitors | 51 -
Compete Rank
740,000,000 - Estimated
Unique Monthly Visitors
25,500,000 - Estimated
Unique Monthly
Visitors | 346 - Compete
Rank
20,500,000 - Estimated
Unique Monthly
Visitors | 605 - Compete
Rank
19,500,000 - Estimated
Unique Monthly
Visitors | 447 - Compete
Rank
17,500,000 - Estimated
Unique Monthly
Visitors | *NA* - Compete
Rank
12,500,000 - Estimated
Unique Monthly
Visitors | 127 - Compete
Rank
12,000,000 - Estimated
Unique Monthly
Visitors | 617 - Compete
Rank
7,500,000 - Estimated
Unique Monthly
Visitors | 838 - Compete
Rank
5,400,000 - Estimated
Unique Monthly
Visitors | 122 - Compete
Rank
3,000,000 - Estimated
Unique Monthly
Visitors | 451 - Compete
Rank
2,500,000 - Estimated
Unique Monthly
Visitors | 1,596 - Compete
Rank
Social Networking
![Page 14: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/14.jpg)
Why Internet became
so Popular?
?
![Page 15: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/15.jpg)
SUN SUN
1
2
3
4
5
6
7
8
Foundations of the WEB
DNS DNS
DNS
DNS
- Countries, Cities, User Groups, …
![Page 16: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/16.jpg)
Problem with Moore’s Law
• The number of transistors that can be
placed on an integrated circuit doubles
every 18 months to two years
• It’s predicted to reach its limit with existing
technology in 2020
• Cutting the size of a transistor to a single
atom may defeat that concept
• The Digital Universe is growing much
more faster than Processing Power
![Page 17: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/17.jpg)
BigData Infrastructure
![Page 18: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/18.jpg)
Google’s First Data Centers
Google’s first data center
![Page 19: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/19.jpg)
Google New Data Centers
Map of Google Data Centers Worldwide
450,000 servers range upwards of
20 megawatts, which cost on the
order of US$2 million per month in
electricity charges.
![Page 20: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/20.jpg)
Google Data Centers
Google data center in Belgium
![Page 21: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/21.jpg)
Google Data Centers
Google data center
in Finland
![Page 22: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/22.jpg)
Everything as a Service
• Utility computing = Infrastructure as a Service (IaaS) – Why buy machines when you can rent cycles?
– Examples: Amazon’s EC2 (Elastic Compute Cloud), Rackspace, Microsoft Azure
• Platform as a Service (PaaS) – Give me nice API and take care of the
maintenance, upgrades, …
– Example: Google App Engine
• Software as a Service (SaaS) – Just run it for me!
– Example: Gmail, Salesforce
![Page 23: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/23.jpg)
Web Crowlers for Web Analytics
• Indexing
• Searching
• Ranking
• Analysis
• Crowling is Essential Job for all Internet
Giants: Google, Yahoo, Facebook, etc.
Some of available open source crowlers: Apache Nutch, Crawler4j,
Bixo, Heritrix, etc.
![Page 24: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/24.jpg)
Web Crowlers for Web Analytics
• Thanks to Crowlers any website can appear in
search results without doing any extra work.
• Customized Crowling by METATags and
“ROBOTS.TXT”
![Page 25: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/25.jpg)
Natural Language Processing
(NLP)
• Natural Language Processing (NLP)
• Computational Linguistics (CL)
• Machine Translation (MT)
![Page 26: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/26.jpg)
Natural Language Processing
(NLP)
• Multilingual NLP
• Text Mining in Multimedia Networks
• Mining Text Streams
• Text Mining in Social Media
• Cross-Lingual Mining of Text Data
• Contextual analysis of text data
Some of availables NLP tools: NLTK, Apache OpenNLP, MontyLingua,
VisualText, etc.
![Page 27: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/27.jpg)
Data Mining and Knowledge
Discovery
Data-driven Decision Making Model
![Page 28: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/28.jpg)
Virtualization as an
Infrastructure
Hardware
Operating System
App App App
Traditional Stack
Hardware
OS
App App App
Hypervisor
OS OS
Virtualized Stack
![Page 29: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/29.jpg)
Cloud Computing
![Page 30: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/30.jpg)
Cloud Computing
![Page 31: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/31.jpg)
Cloud Computing
![Page 32: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/32.jpg)
IP Address –
Dotted decimal notation
• 32 bit binary
• Four 8-bit octets
Ex: 11100011010100101001101110110001
11100011 - 01010010 - 10011101 - 10110001
E3 - 52 - 9D - B1
• What’s a subnet ?
– device interfaces with same subnet part of IP address
– can physically reach each other without intervening router
Example: 122.97.211.200
We can view these values in their binary form.
122 97 211 200
01111010 01100001 11010011 11001000
![Page 33: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/33.jpg)
Internet of Things (IoT)
![Page 34: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/34.jpg)
Internet of Things (IoT)
![Page 35: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/35.jpg)
CeDAWI Center
Applied Research Center for Data Analytics
and Web Insights (CeDAWI)
![Page 36: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/36.jpg)
Data Mining and Knowledge
Discovery
• Text Mining from Web
• Natural Language Processing
• Web Crawling
• Large-Scale Data Management and Processing
• Internet Structure Research and Visualization
• Knowledge visualization
• Cluster and Distributed Computings
![Page 37: Latest Trends in Technology:BigData Analytics, Virtualization, Cloud Computing, Internet of Things (IoT)](https://reader034.fdocuments.in/reader034/viewer/2022052508/559d19971a28ab58558b470a/html5/thumbnails/37.jpg)
Thank you
Questions
http://ce.qu.edu.az/~aadamov