Next Generation Analytics & Big Data (A Reference Model for Big Data)
description
Transcript of Next Generation Analytics & Big Data (A Reference Model for Big Data)
Next Generation Analyt-ics & Big Data
(A Reference Model for Big Data)
Jangwon GimSungjoon LimHanmin Jung
ISO/IEC JTC1 SC32 Ad-hoc meetingMay 29, 2013, Gyeongju Korea
32N2386
Contents
Background Brief history of discussions Case study Procedure for developing standardizations for Big Data Reference model for Big Data Conclusions
2
Discussion of Big Data
Data analytics Data analysis Baba: Vocabulary, Use-case, and so on
Stabilize ArchitectureDefine InterfacesStandardization opportunities
Jim: The aspect of Big Data is “There is many different forms” Krishna: Refers to Wikipedia definition Keith Gorden: Volume, Complex, Velocity Keith W. Hare: Open Big Data Volume, Variety, Velocity, Value, Veracity
Any combination is OK.
3
Background
Emerging Technologies For Big DataIn 2012, The hype cycle of Gartner
Diverse definitions of technologies and services, having different views of data
4
Background
Big Data on hype cycle
A general and common reference model for Big Data is needed
5
Brief history of discussions
6
Issue Date Summary
16 November 2011. [SC32N2181] ISO/IEC JTC 1/SC 32 N2181, “Resolutions and topics from the recent JTC 1 meeting of particular interest to SC 32 participants”, SC32 Chair – Jim Melton
12 January 2012.
[SC32N2198] ISO/IEC JTC 1/SC 32 N 2198, “Analysis of 2012 Gartner Technology Trends”, JTC1 SWG-P - Mario Wendt – Convener SC 6 Telecommunications and information exchange between systems SC 32 Data management and interchange SC 39 Sustainability for and by Information Technology
19 March 2012. [SC32N2199]ISO/IEC JTC 1/SC 32 N 2199, “Discussion: SC 32 Response to 2011 JTC 1 Resolution 33”, SC32 Chair – Jim Melton
6 June 2012. [SC32N2241] Ad-hoc on “Next gen analytics” - Keith Hair - Chair
The view of Next-Generation Analytics of SC32
Referencing from [SC32N2241]
Need a reference model for Big Data to enhance interoperability
7
Next-Generation AnalyticsSocial Analytics
From Baba
Architectural
Mechanisms
Metadata
Raw Storage
Case Study (1)
Korea Institute of Science and Technology (KISTI)Dept. of Computer Intelligence Research
8
Case Study (2)
Architecture of InSciTe Adaptive Service
9
Case Study (3)
Semantic AnalysisText Data to Ontology
10
Case Study (4)
Semantic AnalysisOntology Schema
11
Case Study (5)
Semantic AnalysisExample of Semantic Analysis
12
Case Study (6)
InSciTe Service Functions – (Hybrid Vehicle)
13
Technology Navigation
TechnologyTrend
Core ElementTechnology
Convergence Technology
Agent Level Agent Partner Integrated Roadmap
Report
Case Study (7)
In 2013, About 10 Billion triples from diverse sites will be extracted
14
Sites The number of Count
Freebase 1,015,762,951
Yago 224,949,079
DBPedia 449,383,705
DBLP 81,986,947
baseKB 147,549,529
Etc (WhoisWho,NYTimes,LinkedObervedData,…) 2,296,838,760
Total 4,216,470,971
Case Study (8)
In 2013, System Architecture of InSciTe Adaptive Service
15
Procedure for developing a reference model for Big Data
4. Deriving use-cases for applying the Big Data
3. Defining a concept model / a reference model / a framework for Big Data
2. Establishing visions and strategies for achieving the goal of Big Data
1. Eliciting requirements and analyzing the environment of Big Data
16
We are here
A lifecycle of Big Data
17
1. 2.
3. 4.
• Collection/Identification• Repository/Registry• Semantic
Intellectualization• Integration
• Data Curation• Data Scientist• Data Engineer
Data Insight
Action Decision
• Workflow• Data Quality
Big Data
• Analytics / Prediction
• Visualization
Reference Model for Big Data
A Reference Model for Big Data
18
Data Layer
Platform LayerData Semantic Intellectualization
Data Integration
Data Quality Management
Big DataManagement
Data Curation
Service LayerAnalysis & Prediction
Security
Data Visualization
Service Support Layer
Workflow Management
Interface
Data Collection
Data Identification (Data Mining & Metadata Extraction)
Data Registry Data Repository
Interface
Interface
Reference Model for Big Data
A Reference Model for Big Data
19
Data Layer
Platform LayerData Semantic Intellectualization
Data Integration
Data Quality Management
Big DataManagement
Data Curation
Service LayerAnalysis & Prediction
Security
Data Visualization
Service Support Layer
Workflow Management
Interface
Data Collection
Data Identification (Data Mining & Metadata Extraction)
Data Registry Data Repository
Interface
Interface
9075
13249
11179
19763
???
Conclusions
SummaryAnalyzing the circumstance of Big DataBuilding a framework for Big DataDefine detail procedure to create the Big Data
DiscussionPossible suggestions
• New Working Group for the reference model of Big Data New Work Items could be derived from the model
• New Study Group
Future workDiscussion of the concept of NWI
• 2013. 11. Interim meetingsPropose extended the reference model of Big Data (NWI)
• 2014. 5 Plenary meeting
20