Tackling Big Data Complexity with Spring
-
Upload
spring-io -
Category
Technology
-
view
754 -
download
1
description
Transcript of Tackling Big Data Complexity with Spring
![Page 1: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/1.jpg)
© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Tackling Big Data Complexity With Springwith Mark Fisher, Mark Pollack, Adam Zwickey
![Page 2: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/2.jpg)
WEB
Controllers, REST,WebSocket
INTEGRATION
Channels, Adapters,Filters, Transforms
BATCH
Jobs, Steps,Readers, Writers
BIG DATA
Ingestion, Export,Orchestration, Hadoop
DATA
NON-RELATIONALRELATIONAL
CORE
GROOVYFRAMEWORK SECURITY REACTOR
GRAILS
Full-stack, Web
XD
Stream, Taps, Jobs
BOOT
Bootable, Minimal, Ops-Ready
![Page 3: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/3.jpg)
S P R I N G I O E X E C U T I O N :
Spring XD
![Page 4: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/4.jpg)
Agenda• Big Data Landscape• Spring XD Features and Architecture• Deep dive into modules• Taps• Analytics• Jobs• Big Juicy Demo – Retail Fraud Detection• Runtimes• Resources
![Page 5: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/5.jpg)
Spring XD – Features
• Unified Platform• Developer Productivity• Modular Extensibility• Distributed Architecture• Portable Runtime • Hadoop Distribution Agnostic• Proven Foundation
![Page 6: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/6.jpg)
Spring XD
TapsCompute
HDFS
Workflow Export
Spring XD Runtime
Inge
st
Jobs
Export
Files Sensors Mobile Social
RDBMS
NoSQL
R, SAS
Spring XD Shell
Streams
Redis
Gemfire
Predictive modeling
![Page 7: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/7.jpg)
Stream Processing Model
How can we make this easier?http | filter | file
![Page 8: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/8.jpg)
Single Node
![Page 9: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/9.jpg)
Distributed Mode
![Page 10: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/10.jpg)
D I S T R I B U T E D
Demo
![Page 11: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/11.jpg)
Taps
![Page 12: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/12.jpg)
TA P
Demo
![Page 13: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/13.jpg)
Analytics• https://github.com/spring-projects/spring-xd/wiki/Analytics
![Page 14: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/14.jpg)
A N A LY T I C S
Demo
![Page 15: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/15.jpg)
Jobs• https://github.com/spring-projects/spring-xd/wiki/Batch-Jobs
![Page 16: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/16.jpg)
J O B S
Demo
![Page 17: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/17.jpg)
F R A U D D E T E C T I O N
Demo
![Page 18: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/18.jpg)
XD Runtimes
http | filter | file
Rabbit, Redis, (Pluggable)
XD Admin
CLUSTERED NODE
FilterModule
CLUSTERED NODE
HTTPModule
CLUSTERED NODE
FileModule
In MemoryTransport
http | filter | file
SINGLENODE
AllModules
![Page 19: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/19.jpg)
Portable Runtimes
CLOUDFOUNDRY
AWS
GEMFIRE YARN
SIMPLECLUSTER
![Page 20: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/20.jpg)
Spring YARN• http://bit.ly/spring-yarn-blog
![Page 21: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/21.jpg)
Resources• Project: http://projects.spring.io/spring-xd/ • Issues: https://jira.springsource.org/browse/XD• GitHub: https://github.com/spring-projects/spring-xd/• Wiki: https://github.com/spring-projects/spring-xd/wiki • Blogs:
– https://spring.io/blog/2013/06/12/spring-xd-1-0-milestone-1-released/– https://spring.io/blog/2013/08/14/spring-xd-1-0-milestone-2-released/
• Release 1.0 M3 (just now)– http://bit.ly/spring-xd-m3-release
![Page 22: Tackling Big Data Complexity with Spring](https://reader034.fdocuments.in/reader034/viewer/2022042700/554fa00cb4c9057b298b4858/html5/thumbnails/22.jpg)
RELATED SESSIONS
Real Time Analytics with Spring