A PROJECT PROGRESS REPORT

ON

SENTIMENT ANALYSIS &

INFORMATION EXTRACTION

IN

PARTIAL FULFILLMENT OF THE REQUIREMENT FOR THE AWARD

OF THE DEGREE

OF

BACHELOR OF TECHNOLOGY

SESSION 2010-2014

GUIDED BY SUBMITTED BY

Ms. PARUL YADAV DIKSHA MAHAJAN (25011503110)

CERTIFICATE

This is to certify that the project entitled “SENTIMENT ANALYSIS &

INFORMATION EXTRACTION” is the original work carried out by Diksha Mahajan

(25011503110) student of B.Tech (IT), BVCOE, affiliated to GGSIPU, during the year 2014, in

partial fulfillment of the requirements for the award of the Degree in Bachelor of Technology,

Information Technology and that the project has not formed the basis for the award previously of

any degree, diploma, associateship, fellowship or any other similar title.

Signature of the Guide

Ms. PARUL YADAV

IT Dept, BVCOE

1. Objective

1.1. Abstract: The project aims at providing a sentiment analysis system through a web interface that enables web users, analysts and product managers to get insights into public sentiment on particular products and services. The project makes extensive use of product and services review sites and forums like IMDB, as well as micro blogging sites like Twitter. The system aims to apply efficient information retrieval algorithms, as well as do the complex task of feature extraction for a more drilled down analysis, in the most efficient way.

2. Introduction

2.1. What is Part of Speech Tagging and how we implemented it?In the collection of linguistics Part of Speech tagging is also called grammatical tagging or word category disambiguation, in which we discern the words according to their category eg in English dividing words in categories of noun, verbs, prepositions etc. Part of Speech tagging is now been performed in the context of computer linguistics using algorithms built on Hidden Markov Model, Decision table, Dynamic Programming Models, Unsupervised Taggers etc.It comes in Natural Language Processing and a lot of successful contribution has been made under this topic

A Part-Of-Speech Tagger (POS Tagger) is a piece of software that reads text in some language and assigns parts of speech to each word (and other token), such as noun, verb, adjective, etc., although generally computational applications use more fine-grained POS tags like 'noun-plural'. We used Stanford POS tagger, this software is a Java implementation of the log-linear part-of-speech taggers developed by stanford engineers and researchers.

2.2. Sentiment analysis-introduction and how we are going to implement it2.2.1. Sentiment analysis

Sentiment Classification, a sub topic of Sentiment Analysis, is the study of computationally determining whether a given piece of text is positive or negative. We usually apply machine learning techniques to sentiment classification, in which a classifier is required to be trained on a labeled training set. This is called supervised learning. However, owing to its

nature and the number of tweets that can be collected, it is a challenging task to manually label a training set of such magnitude.

2.2.2. Algorithm Used :2.2.2.1. Naive-Bayes Classifier

2.2.3. Tools to use:2.2.3.1. Wekaparallel

2.3. Algorithm followed:2.3.1. Generate the imdb movie review url for the movie.2.3.2. Download all the reviews web pages from IMDB.2.3.3. Apply POS tagging on the downloaded movie reviews to get all the proper

nouns like "leonardo", "acting", "direction", "oscars" etc.2.3.4. Identify all the actors, actresses, directors and movie names present in the

above generated list (in 3rd point).2.3.5. Extract all the sentences which have the above generated keywords (as

generated in 4th point).2.3.6. Apply sentiment analysis on the sentences extracted from above step.

2.4. IMDBCrawler: We made an IMDB review extracter as IMDB does not provide any API for extracting reviews. We used an API provided which gives the imdb id for that movie, after that we download that web page and store the results. We used Jsoup java library for downloading web content and applying complex pattern matching on that text.

3. Handouts:

4. Progress:

S.NO TASKS ATTEMPTED STATUS

1 Feature Extraction

1.1 Actors Yes Completed

1.2 Actresses Yes Completed

1.3 Directors Yes Completed

1.4 Movies Yes Completed

2 Crawler

2.1 IMDB Yes Completed

2.2 Rotten Tomatoes No -

2.3 GSM Arena No -

3 Algorithm

3.1 POS Integration Yes Completed

3.2 Sentiment Analysis No -

3.3 Entity Recognition No -

4 User Interface

4.1 Main Module Yes In Progress

4.2 Contribution Module No -

4.3 Project Wiki No -

5. References:

[1] Toutanova, K., Klein, D., Manning, C.D., Singer, Y.: “Feature-rich part-of-speech tagging with a cyclic dependency network.” In: NAACL 3. (2003) 252–259[2]Christopher D. Manning. 2011.:” Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics? Computational Linguistics and Intelligent Text Processing” , 12th International Conference, CICLing 2011[3] Shen, L., Satta, G., Joshi, A.: “Guided learning for bidirectional sequence classification.” In: ACL 2007. (2007)[4]Spoustov´a, D.j., Hajiˇc, J., Raab, J., Spousta, M.: “Semi-supervised training for the averaged perceptron POS tagger.” In: Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009). (2009) 763–771[5]Søgaard, A.: “Simple semi-supervised training of part-of-speech taggers.” in proceedings of the ACL 2010 Conference Short Papers. (2010)[6] B Pang, L Lee .: “Opinion mining and sentiment analysis”, In:Foundations and trends in information retrieval, 2008 - dl.acm.org[7] Changhua Yang, Kevin Hsin-Yih Lin, Hsin-Hsi Chen, .: “Building emotion lexicon from weblog corpora” in proceedings of ACL '07 ACL on Interactive Poster and Demonstration Sessions [8] Alec Go, Lei Huang, and Richa Bhayani. 2009 .:Twitter sentiment analysis. Final Projects from CS224N for Spring 2008/2009 at The Stanford Natural Language Processing Group.