Twitter Trend Analyzer
-
Upload
matthew-chang -
Category
Software
-
view
92 -
download
0
Transcript of Twitter Trend Analyzer
Team ProjectTwitter Trend Analyzer
Group 20Ikwhan Chang / Quan Trung Nguyen
Prof. Hungwen Li
11-May, 2017
1
Final Presentation
INDEX• Overview
• Twitter Developer API
• Search API
• Streaming API
• Web Socket
• Chart & Map
• Heroku for the PaaS
• Architecture
• Data Flows
• User Interface & Demo
2
Overview• Project Title
• The trend analysis and predict engine for a specific keyword of Twitter by using cloud platform
• Description
• Twitter is one of popular social networking tools to share our think or news, and it will rapidly spread across the worlds.
• We will make a web application to collect the Twitter's feeds related to the specific word, and then provide the user a daily trend by using a realtime linear chart and geographical chart
• Team Members
3
Name Ikwhan Chang Role Frontend / API / Cloud / UX
Name Quan Nguyen Trung Role Backend / Data Handling
Twitter Developer API• Register the application
• To use the Twitter’s API, we need to register our application at dev.twitter.com
• OAuth
• Twitter data are collected by using Twitter APIs, which require OAuth to identify Twitter applications and users.
• Allow users to access to a protected database. (e.g. twits, mentions..)
• The user authentication will use a signed request to identify an application‘s identity along with the user’s access token. There are four secret keys
4
dev.twitter.com
Consumer Key/Secret
Access Token/Secret
Twitter Search API• It generates queries to collect data of recent and popular Tweets with certain criteria
• We can also use a set of parameters to have a better control of the search results.
• Result Type: indicates that the search results will be recent or/and popular Tweets
• Geolocalization: restricts the location of Tweets
• Language: restricts the languages of Tweets
• Iterating in a result set: controls size of search results
• When the user enters the keyword, the application will create an URL-encoded search query and will be sent to the server via the HTTP request.
• After receiving the HTTP request, the server will pass keyword value to parameter q {q: keyword} to perform the search.
• When the server receives the search result from Twitter, it will send the result back to client via HTTP response in form of JSON files.
5
Search API - backend
Search - frontend
Twitter Streaming API
• We used Streaming APIs to obtain high volume, real-time data from Twitter for our charts
• Twitter provides several basic streaming types:
• Public streams: the streaming data comes from the public data flowing through Twitter
• User streams: the single-user streaming data comes from a single user’s view of Twitter
• Site streams: this is the multi-user version of user streams
• The streaming connection process will get the collected data, perform the tasks parsing, filtering and store the result into a data store.
• Then the HTTP server will get the processed result to send back to the user by using Web Socket
6
Streaming API overview
Web Socket
• WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection.
• WebSockets are based on http, but are completely different protocols than the http protocol.
• Since WebSockets do not send header information except for the initial connection, they are much more cost effective in terms of network cost and can be implemented quickly and without waiting because of full-duplex communication.
• Our application uses WebSockets to send the streamed Twits into web browser by using socket.io
7
Web Socket Flows
socket.io
Chart and Map• HighCharts (www.highchart.com)
• For the real-time chart, we used the javascript library of HighCharts’s linear chart.
• In order to draw the graph like above, you need to use jquery called Highcharts
• Google Map API (https://developers.google.com/maps
• We used the Google Map API in order to the geographical chart and the geocoding.
• The geocoding means we can convert from actual user’s address to coordinates like latitude and longitude.
• Since twitter API provide us just user’s location character, we need to convert from it to the actual geo coordinates in order to show into our geographic chart.
8
Heroku for PaaS
• Heroku is a new way of deploying web applications, where developers are providing services based on the idea that they can concentrate on the code of the web application without losing time for tasks such as server configuration, deployment environment, and server expansion.
• Developers are only deployed and restarted by writing source code, committing the code to the git repository for deployment, and pushing it to the remote Heroku repository.
• If the success of your web application requires more servers, increasing the number of processors called Dyno will have the same effect as adding multiple servers.
9
Type of Cloud Computing
Heroku Deployment Architecture
Architecture10
Back-end Front-end
Integrated Framework
UI/CSS Framework
Dependencies Management
Javascript Framework
Testing
Task Management
Test Runner
Pre-processor
Compressor
UglifyJS, UglifyCSS
Code Qualifier
Documentation
JSDoc3
Lazy Loader
Integrated Framework
Cloud
Streaming
Twitter Stream API
Data Flows11
Frontend Backend OAuth
Search Search
/search?q=iphone&start_date=20160227&end_Date=20170327
Trend Chart
Streaming
Geographic Chart
{twits:[{ ‘no’:’1’, ‘twit’:’#iphone is gorgeous!!’ ‘date’:’2017-01-18’, ‘location’:’San Jose, CA’},….. ]}
{[{‘date’:’2017-03-26’,’expected’:45’},{‘date’:’2017-03-27’,’expected’:45’},
….]
Date Range
http://localhost:8080
http://localhost:8080
Stream API{keyword:’iphone’}
Key Exchange (Secret key, token)
User Interface & DemoSearch Box
Realtime Linear Chart
Geographical Chart
Actual Twit Data
# of streamed Tweets
http://twitter-trend-analyzer2.herokuapp.com
Thank You13
https://github.com/IkwhanChang/twitter-trend-analyzer