The CLASS Software Architecture
9
The CLASS project has received funding from the European Union's Horizon 2020 research and innovation programme under the grant agreement No 780622 COORDINATING EDGE AND CLOUD FOR BIG DATA ANALYTICS The CLASS Software Architecture Eduardo Quiñones {[email protected]} Barcelona Supercomputing Center (BSC-CNS)
Transcript of The CLASS Software Architecture
The CLASS project has received funding from the European Union's Horizon 2020 research and innovation programme under the grant agreement No 780622
COORDINATING EDGE AND CLOUD FOR BIG DATA ANALYTICS
The CLASS Software Architecture
Eduardo Quiñones{[email protected]}Barcelona Supercomputing Center (BSC-CNS)
General Information
Edge and Cloud Computation: A Highly Distributed Software Architecture for Big Data AnalyticS- Under the scope of H2020 ICT16-2017 (RIA) - Big data PPP: research addressing main technology
challenges of the data economy
- 42 months (starting January 2018)
- 3.900.803 € budget
1Big Data Value Association (BDVA) Webinar
Motivation: The Importance of CLASS
1. Geographically distributed data sources and data analytics requirements, e.g., smart cities
2. The fulfillment of real-time requirements inherited from the application domain
3. Constant increment of volume, variety and velocity of data-sets
2
A coordination of edge and cloud resources is needed!
Data sources
Large data-sets
Edge computing
- Data collection and transfer- Limited computation capabilities
Cloud computing
- Data storage- High computation capabilities Data
Centers
Data Analytics
Net
wo
rk
LatentData-at-rest
analytics
Reactivedata-in-motion
analytics
Co
mp
ute C
on
tinu
um
Big Data Value Association (BDVA) Webinar
The Vision of CLASS
1. Significantly increase the capabilities of the data analytics- Integrate both responsive data-in-motion and
latent data-at-rest analytics in a single complex workflow
2. Fulfill the real-time requirements
3. Use advance parallel and energy-efficiency embedded platforms at edge side
3
Productivity+ Programmability+ Portability/Scalability+ (Guaranteed) Performance
Cloud Computing
Data sources
Edge C
om
pu
ting
Large data-sets
Network
Complex data analytics workflows across the compute continuum
Co
mp
ute
Co
nti
nu
um
Big Data Value Association (BDVA) Webinar
Workflow orchestrator and distributed storage
Main Contribution:The CLASS Software Architecture
4
Integrate technologies from different computing domains into a single development framework
1. Powerful API for the development of advanced data-analytics methods
2. QoS Serverless and CaaS cloud technologies
3. Advanced orchestration methods for time-predictable workflow scheduling and deployment across the compute continuum
4. Used of advanced embedded parallel and heterogeneous processor architectures
Distributed Data-Analytics API
Serverless
Map/Reduced Task-based Model
CaaS
Rotterdam
Programming Models
Compute Continuum
Big Data Value Association (BDVA) Webinar
Smart City Use-Case
Deployed on the Modena Automotive Smart Area (MASA) in the city of Modena (Italy)
1. A living lab urban area with IoT connectivity and a compute continuum infrastructure
2. Three connected cars equipped with sensors (cameras and LiDAR) and V2I communication
Information exchange between the city and vehicles to enhance mobility
1. Computation of emission of pollution in real-time
2. Advanced Driving Assistant Systems
- Virtual Mirror
- Two Sources of Attention
5ACTIVITY, LOCATION
Smart Cit y Use-case
Deployed on the Modena Automotive Smart Area (MASA) in the city of Modena (Italy)
- 1 Km2 urban area with connectivity that enables IoT devices to exchange information
- Three connected Maserati cars equipped with sensors (cameras and LiDAR) and V2I communication
From the city perspective, an intelligent traffic management
- “Green routes” for emergency vehicles- Smart valet parking system
From the car perspective, advanced driving assistance systems
- Trajectory prediction and collision detection
9Big Data Value Association (BDVA) Webinar
Smart CarCam_1
Data-Analytics Methods
1. Sensor Fusion2. Object Detection3. Object Tracking4. Data deduplication5. Trajectory Prediction6. Air pollution computation7. Data model creation8. Collision Detection (CD)9. Generation of WA10. WA alert visualization
Cam_2
Rotterdam
Cloud Layer
Edge/FogLayer
IoTLayer
pedestrian
Co
mp
ute
Co
nti
nu
um
Virtual Mirror Use-Case
2
3
2
4
1
7
10
598
3
6
Knowledge Base
Workflow orchestrator
and distributed storage
Distributed Data-Analytics API
Serverless
Map/Reduced Task-based Model
CaaS
Rotterdam
Programming Models
Summary
1. CLASS aims to develop a novel software architecture with the following capabilities:
- Increase data analytics capabilities by efficiently combine data-in-motion and data-at-rest analytics into complex workflows
- Increase the development and deployment productivity of systems requiring data-analytics
- Guarantee the real-time properties inherited from the domain
2. CLASS aims to apply the software architecture to develop a distributed sensing/computing infrastructure within the MASA for advanced urban mobility applications
7Big Data Value Association (BDVA) Webinar
www.class-project.euTwitter: @EU_CLASS
LinkedIn: linkedin.com/company/classproject
