Develop once deploy everywhere - KNIME€¦ · » Planning & forecasting » ... analysis » Credit...
Transcript of Develop once deploy everywhere - KNIME€¦ · » Planning & forecasting » ... analysis » Credit...
1 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Develop once – deploy everywhere
Advanced Text Analytics with KNIME Server
Stefan Weingärtner, DYMATRIX CONSULTING GROUP GmbH
KNIME User Day UK, 25th June 2013
2 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Agenda
1 Company Introduction
2 The growing importance of text analytics
3 DYMATRIX Text Mining Prozess
5 Application Integration I: DynaSocial
6 Application Integration II: Advanced Email Classification
4 Benefits
7 New Developments
8 Q & A
3 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Company Introduction
4 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX – The analytical CRM Company
» Solution provider for Customer Intelligence, Marketing Automation and
Advanced Predictive Analytics
» Consulting, development and implementation know how, based upon
more than 900 projects with mid- and large cap companies across
industries
» Goal- and client- oriented project execution based upon award winning,
established solutions
» Owner managed and independent
5 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Our Consulting Competence Centers
Business Intelligence
Advanced Analytics
Campaign Management
» Conception of (big) data warehouse and business intelligence architectures
» Corporate group reporting systems
» Dashboards
» Sales controlling
» Planning & forecasting
» Balanced scorecard
E-commerce insight
» Customer segmentation
» Customer value analysis
» Propensity Modeling (Cross-/Upsell/Churn)
» Shopping basket analysis
» Credit rating analysis & credit scoring
» Text Mining
» Data mining automation
» Big Data Analytics
» Design and optimization of campaign processes and workflows
» Implementation of campaign management systems
» Integration of Data Mining Models in Campaign Processes
» Campaign Optimization
» Consulting & Implementation of Next Best Activity Processes
» Web tracking
» Web controlling
» Web mining
» Real time recommendation
» eCRM
» Social media tracking & analysis
» Web performance measurement
» Customer Journey Analytics
Analysis of client oriented processes Initial situation – Analysis – Conception of processes for customer retention and its optimization -
customer reactivation and new customer activation – benchmarking against industry leaders
6 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Solution Portfolio – The Customer Insight Suite
DynaCampaign
» Intelligent multi-touchpoint campaign management platform
» Planning, target group selection, execution and response measurement of campaigns
» Event-triggered realtime campaigning
DynaMine
» End2end automation of data mining processes
» Intelligent model management for automation of preprocessing, training & scoring of models
DynaCision
» Realtime decision management platform
» Design & exection of complex embedded decision processess
DynaSocial
» Social CRM platform to listen, track, identify and quantify customer needs and sentiments
7 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Our KNIME Solution Nodes & KNIME Consulting Services
PMML2SQL / PMML2SAS Converter
» Convert PMML to executable SQL Code for In-Database-Scoring
» Convert PMML to executable SAS Code for Model Scoring within SAS
Big Data Integration
» Access any Hadoop large-scale distributed batch processing infrastructure from KNIME
» Efficiently distribute large amounts of data & preprocessing across a set of machines
Uplift Modeling
» Train a predictive model that predicts the incremental response to marketing actions
» For up-sell, cross-sell, churn and retention activities
Interactive Scorecard Builder
» Powerful interactive Scorecard Building Nodes for Design of Credit or Marketing Scorecards
+ Business Consulting + Analytical Consulting + Technical Consulting + Trainings
8 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Referenzen References
Telecommunication Travel, Transportation Retail, Service Provider
9 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
References
Schwäbisch Hall
Media Banks, Insurances Utilities, Industries, Public
10 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
The growing importance of Text Analytics
11 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
80% of the world’s data is unstructured.
Unstructured data is growing at 15 times the rate of structured data.
Source: Google Trends April 6, 2012
Big Data is not just about structured data…
15 times
80%
12 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Needs
Possibilities
Decisions
Approach
Purchase Delivery
Usage
Service & Support
Remember
Challenge: Big Data Collection & Integration
Source: Phil Winters, 2011
13 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
» …to classify all customer related text messages by
Source / Origin
Sentiment
Product or Service
Business Transaction
Context
etc.
» …to identify unknown trends
» …to identify cause and effect relations
» …to react on that information, e.g.
Technical Problems
Needs
Usability
Competition
etc.
Imagine…
The KNIME platform supports these efforts with comprehensive Text Analytics & Network Analytics capabilities!
14 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process
15 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process
Text Datasources
Datasources: • Facebook • Twitter • Emails • Data Provider
like GNIP, Datasift etc.
For Machine Learning • Provide Training
Data for Classification (e.g. Sentiment)
Text Enrichment
Language Detection • English • German • Many more… Language individual NLP POS Tagging • Penn Treebank
Tagger • STTS Tagger Text Cleansing • Stop Words • Stemming • Punctuations Sentiment Amplifier • Matching of
Sentiment- & Emoticon-Dictionaries
Subject Matching
Text Tagging with any Subjects • Products • Brands • Business
Transactions • Service • Complaints • Requests • etc.
Fuzzy Matching with Dictionary Tagger • Matching of
Subject-Dictionaries
Sentiment Classification
Text Vectorization • Creation of text
predictors to predict sentiments
Machine Learning • Classification with
Predictive Analytics (e.g. Decision Tree)
Retraining Interface • Adjustment of
misclassified messages for permanent optimization of classification
Information Delivery
Text Data Mart • Make information
available in central Text Data Mart for visualization, alerting etc.
Fields of Application • Email-Routing • Event triggered
Campaign Management
• etc.
16 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process: Datasources
Text Datasources Information
Delivery Sentiment
Classification Subject
Matching Text
Enrichment
Access any Text Datasource to start the Text Mining Process
» Emails
» Crawler
» Data Provider like GNIP, Datasift etc.
Exemplified contribution on Facebook Fanpage
Vodafone UK
17 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process: Text Enrichment
Why not sort your signal issues out instead of bringing new phones out!!!! Wk 3 of crap [----] signal but yet paying FULL monthly contract! Vodafone sort it.
Sentiment Amplifier
sort[VBG] signal[VBP] issues [VBZ] instead[RB] bringing[VBG] phones[NNS] Wk[NNP] 3[CD] crap[NN] paying[VBG] monthly[RB] contract[NN] Vodafone[NNP]
Removal of Stop Words & Punctuations
Penn Treebank POS Tagger (English Messages)
Why[WRB] not[RB] sort[VBG] your[PRP] signal[VBP] issues [VBZ] out[IN] instead[RB] of[IN] bringing[VBG] new[JJ] phones[NNS]!!!![SYM] Wk[NNP] 3[CD] of[IN] crap[NN] but[CC] yet[RB] paying[VBG] FULL[NNP] monthly[RB] contract[NN] ![SYM] Vodafone[NNP] sort[VBG] it[PRP] .[SYM]
Text Datasources Information
Delivery Sentiment
Classification Subject
Matching Text
Enrichment
Original Facebook Message
Why not sort your signal issues out instead of bringing new phones out!!!! Wk 3 of crap signal but yet paying FULL monthly contract! Vodafone sort it.
18 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process: Subject Matching
Subject Matching (Fuzzy Matching)
Why not sort your signal issues out instead of bringing new phones out!!!! Wk 3 of crap signal [NETWORK] but yet paying FULL monthly contract! Vodafone sort it [COMPLAINT].
Text Datasources Information
Delivery Sentiment
Classification Subject
Matching Text
Enrichment
Why not sort your signal issues out instead of bringing new phones out!!!! Wk 3 of crap signal but yet paying FULL monthly contract! Vodafone sort it.
BUSINESS TRANSACTION: Complaint
NETWORK: No Signal
PRODUCT: Nokia Lumia 925
Original Facebook Message
19 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process: Sentiment Classification
Output from Text Enrichment
Predictors relevant for Text Classification , e.g. - Emoticons positive/negative - Length of message - Fragments positive/negative - Likes - Words positive/negative - Comments - Author-related Inputs - Other linguistic Inputs
Text Vectorization (Transformation)
Text Datasources Information
Delivery Sentiment
Classification Subject
Matching Text
Enrichment
Why not sort your signal issues out instead of bringing new phones out!!!! Wk 3 of crap signal but yet paying FULL monthly contract! Vodafone sort it.
Original Facebook Message
Text Classification with Decision Tree
Resulting Classification
20 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process: Information Delivery
Make information available in central Text Data Mart Visualization in DynaSocial
Original Facebook Message
Other Fields of Application
» Subject-oriented Email-Classification & Email-Routing
Text Datasources Information
Delivery Sentiment
Classification Subject
Matching Text
Enrichment
Why not sort your signal issues out instead of bringing new phones out!!!! Wk 3 of crap signal but yet paying FULL monthly contract! Vodafone sort it.
Sentiment Business Transaction
Product Relevance
+
+ + +
Network
21 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DYMATRIX Text Mining Process: KNIME Workflow
22 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Benefits
23 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
» The KNIME-Server based Text Enrichment & Classification Workflow can be used for classification of any electronic text messages (e.g. Social Content, Blogs, Emails).
» The Text Enrichment & Classification Workflow is deployed as a webservice and can be called easily from any other application.
Develop once, deploy everywhere!
Benefits
» Uniformed Sentiment- and Classification-Handling for all customer-related messages.
» Batch- or Realtime-Execution from any application.
24 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Application Integration I: DynaSocial
Social Media Monitoring & Analytics
25 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Generic Big Data Model
Social Media Analytics Data Management
Social Media Analytics Dashboard
DynaSocial – Social Media Excellence Architecture
Text Enrichment & Classification Network Insights
Advanced Social Media Analytics Text Mining & Network Mining
Social Media Analytics Content Extractor
Client individual Sources
Social Media Data Provider
Social Service Platforms
Emails Integrated Social Inbox including all Social Touchpoints
Social Engagement
Data Sources Sentiments & Classifications Reports & Dashboard
DynaSocial Configuration Center
26 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DynaSocial Management Dashboard
Activities
Sentiment Ratio
Key Influencer
Platform Distribution
Trends compared to competition (Share of Voice)
Geographic Distribution
Overall Sentiments
Top Keywords
Flexible Selection of Time Windows
…
27 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
DynaSocial Management Dashboard (Project Example)
28 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Application Integration II: Advanced Email-Classification
Multidimensional realtime Email-Classification
29 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Email Classification: MS Exchange Connector
KNIME Server
Microsoft Exchange Webservice
.NET Batch
Microsoft Outlook
2 Call .NET Procedure and transfer email contents to KNIME Server via Webservice Call.
Incoming Email
Call KNIME Text Enrichment & Classification Workflows und return classification results.
Classification results are returned to Exchange Server and are saved persistantly with object categories.
Any clients having access to Exchange Server get the same classification.
1
4
3
5
Microsoft Outlook Webaccess
Other Email-Clients
30 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Livedemo
Realtime Email-Classification
31 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
New Developments
32 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
New Developments
Multilingual NLP
» Alongside German, Englisch, French and Italian also Spanish and Chinese is supported.
Influencer Detection
» Influencer Identification (DYMATRIX-Algorithm)
Social Earthquake Warning System / Social Hype Detection
» Early warning of hypes by monitoring of news extension
» Comparison to spreading patterns of past hypes/hot topics
Authenticity Score
» Authenticity-Score for automatic identification of Fake-Assessments (e.g. Assessments of Products or Hotels etc.)
33 © DYMATRIX CONSULTING GROUP DynaSocial Social Media Analytics
Thank you for your attention. We are happy to answer any of your questions!
DYMATRIX CONSULTING GROUP GmbH Zeppelin Carré Lautenschlagerstrasse 2 D-70173 Stuttgart Your Contact: Stefan Weingaertner
Phone
Fax E-Mail
Web
+49.711.22.007.88 - 12 +49.711.22.007.88 - 88 [email protected] www.dymatrix.de
Contact