An Introduction of Big data; Big data for beginners; Overview of Big Data; Big data Tutorial
Big Data
-
Upload
marian-borca -
Category
Technology
-
view
337 -
download
0
description
Transcript of Big Data
![Page 1: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/1.jpg)
BIG DATAPLATFORM, TECHNOLOGY & TOOLS
![Page 2: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/2.jpg)
Summary
Intro – what is Big Data? Objectives Technology approach ETL, infrastructure, applications & tools Existing platforms and tools Evolution
![Page 3: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/3.jpg)
What is Big Data?
Big Data = 3V High Volume High Velocity High Variety
Includes: Capture, Curation, Storage, Search, Sharing, Transfer, Analysis, Visualization
![Page 4: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/4.jpg)
Objectives
Actionable analytics A/B testing Channel content automation and
optimization Accountable marketing
Measure marketing initiatives impact Using predictive technology
Creative discovery Using BI tools Explore what questions could be asked
![Page 5: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/5.jpg)
Brand Ecosystem
VOLUME / VELOCITY / VARIETY
Web & E-commerceSocial MediaMobile ApplicationsAd ServingData & CRMPlatforms & Services
![Page 6: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/6.jpg)
Connecting the dots – Big Data Platform
BIG DATA PLATFORM
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools
Analytics & Reporting
Automation & Optimization
![Page 7: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/7.jpg)
Big Data - High Level System Architecture
Brand Ecosystem
Automation&
Optimization
WebPlatforms
Social Media
Mobile Application
s
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing – Tracking, Logging, ETL
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 8: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/8.jpg)
Big Data - Data Flow & Tools
DATA SOURCES
LOG SERVICES
DATA WAREHOUSE REAL TIME DATA STORAGE
ANALYTICS REPORTING AUTOMATION, OPTIMIZATION
DB DataUnstructured Data
Log Files Exhaust Data Social Media Sensors, Devices
d3.js Real Time APIsA/B Testing
![Page 9: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/9.jpg)
Big Data Roles
Program manager Project scope definition and planning, delivery, documentation and circulation of an
end to end plan, driving a unified message to all stakeholders, provide actionable detail on future requirements, present program status and issues
Infrastructure IT Administrators – cluster configuration, management and maintenance
Software Software Engineers – programming and technical analysis for Big Data main
solution and related products
Software Architects – solution and application architecture for all related products (ETL, data warehouse, real time databases, platforms and tools)
Data Architects – distributed data storage architecture, related platform and tools database architecture
BI Developers – programming for distributed queries, predictive analysis tools, automation tools
Analysis Data Analysts – data analysis, reporting tools, cross platform data analysis
BI Analysts – predictive multichannel analysis, BI tools
Data Scientists – Big Data algorithms for BI and predictive models
![Page 10: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/10.jpg)
Big Data Components
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
Events and Data Capturing Distributed Infrastructure Platforms & Tools Reporting & Analytics Automation & Optimization
![Page 11: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/11.jpg)
Events and Data Capturing
Every user action or state change on each client platform will be logged using a common structure (Json format): USER uid, reg_uid
Unique identifier for each en user When a user is known (logged, across multiple platforms)
merge previous activity (events) on a single thread EVENT tstamp, client_id, app_id, obj_id, event_id
When the event occurred What event is logged (platform, object, event)
CONTEXT ip, uagent, referrer, qstring, geo_coords User context
(application used) IP address
and geo-location
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 12: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/12.jpg)
Events and Data Capturing
Additional data to be captured to complement user related events and states, such as: Sales information Context information – weather, events, etc. Other relevant dataData stored using a common structure (Json) – somewhat similar to user events but related to the context or the business client, not the user
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 13: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/13.jpg)
Events and Data Capturing
Shared libraries and protocols to be used
across all platformsLIBRARIES
Web & E-commerce
Social Media
Mobile Applications
Ad Serving
Data & CRM
Platforms
& Services
Browser client library
✔ ✔ ✔ ✔
Mobile client libraries
✔
Log files import ✔ ✔ ✔ ✔
Data import ✔ ✔ ✔ ✔
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 14: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/14.jpg)
Distributed Infrastructure
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 15: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/15.jpg)
Platforms & Tools
CRM Marketing View Media Publishing Platform Other Platforms & Tools – related to
social media, loyalty platforms, ecommerce, CRM, etc.
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 16: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/16.jpg)
Platforms & Tools – CRM Marketing View
Segment CRM users based on a group/segment definition schema
Generic admin interface for managing segments and quality control
Generic solution for any CRM platform Simplify CRM operations Simplify custom CRM dashboards and
reports Integrates smoothly
with other Big Data components
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 17: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/17.jpg)
Platforms & Tools – Publishing Platform
Generic scalable platform
Easily add any type of input
Manage real-time aggregation rules
Automatically publish live banners, ads, etc.
A/B testing for output media
Integration with CRM and live feeds
Integration with other Big Data components
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 18: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/18.jpg)
Platforms & Tools – Top Voice
Social media brand influence platform Real time data synchronization Scalable infrastructure & services
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 19: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/19.jpg)
Analytics & Reporting
Big Data Ultimate Dashboards Trends & Semantic Analysis BI Applications & Tools
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 20: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/20.jpg)
Analytics & Reporting – Dashboards
Tableau Software platform Leader on data visualization Connects with relational databases Connects with data stores such as
Hadoop, Google Big Query, HP Vertica
Rich and interactive dashboards and reports
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 21: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/21.jpg)
Analytics & Reporting – Sentiment Analysis
Nexalogy Process unstructured text data Easily connects with social, CRM or any
other brand proprietary data Finds relevant streams of conversations
and data
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 22: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/22.jpg)
Analytics & Reporting – BI Applications
BI Tools Sophisticated reports and correlations Predictive technology Software solutions such as Mahout, HP
Vertica, R, Platfora, Datameer, SAS, SPSS, PSPP, Pivotal
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 23: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/23.jpg)
Automation & Optimization
Automation services and processes Dynamic and personalized offers and
content on websites, social media, mobile, ad banners, etc.
Feed Big Data analytics into live input channel applications
A/B testing
Brand Ecosystem
Automation&
Optimization
WebE-
commerce
Social Media
Mobile Applicatio
ns
Ad Serving
Data CRM
PlatformsServices
Events and Data Capturing
Distributed Infrastructure
Platforms & Tools Analytics & Reporting
![Page 24: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/24.jpg)
Big Data – Client Facing Tools
Platforms and tools Media Publishing Platform for real-time content
automation CRM Marketing View for cross platform state
marketing Other tools integrated with Big Data
Analytics & Reporting Big Data Ultimate Dashboards using Tableau
Software Predictive models for content and campaign
optimization Possibility to expose query tools directly to end-users
![Page 25: Big Data](https://reader036.fdocuments.in/reader036/viewer/2022081414/54c6e9094a79590e788b45f7/html5/thumbnails/25.jpg)
Big Data
BEFORE AFTER