Redefining Sentiment Analysis

Post on 26-Jan-2015

119 views 0 download

description

Why current sentiment analysis theory and practice is fundamentally flawed and what to do about it. Talk at the research session of the Sentiment Symposium in San Francisco on Nov 8, 2011. Based on joint work with Dr. Ken Bloom.

Transcript of Redefining Sentiment Analysis

Redefining(Sentiment(Analysis1

Shlomo Argamon Subtext3, Inc.

Illinois Institute of Technology

Strong(foundations…18 November, 2011 Sentiment Analysis Symposium, San Francisco

…are(critical!18 November, 2011 Sentiment Analysis Symposium, San Francisco

The(Archetype1Review classification – the earliest, and still

stereotypical sentiment analysis application

8 November, 2011 Sentiment Analysis Symposium, San Francisco

5.0$out$of$5$stars$Great!1This(Nikon(is(great.(It(takes(perfect(pictures(and(I(tried(the(shaking(effect;(it(was(the(bom.(I(shaked(the(camera(while(taking(a(picture(and(the(picture(came(out(picture(perfect.((Great(product!(1

1.0$out$of$5$stars$Buy*cheap,*buy*twice2Im(absolutely(livid(with(myself(for(buying(this(camera.(IFve(gone(through(a(divorce(and(been(left(with(nothing((including(the(loss(of(my(SLR(cameras)(and(this(was(the(one(I(could(afford(that(promised(to(be(the(most(similar(to(the(SLRs.(I(have(a(2(year(old(son,(and(armed(with(only(this(camera,(I(have(missed(documenting(the(last(6(months(of(his(time(with(me.(Every(picture(is(blurredNN(granted(heFs(2(and(moves(a(lot,(but(even(the(slightest(movement(by(the(photographer(blurs(the(image.(Blurred.(EVERYTHING.(I(wish(I(had(saved(up(and(bought(something(decent.(I(will(take(my(loss(on(this(one(for(an(Amazon(gift(card,(but(DONFT(WASTE(YOUR(MONEY.(*sigh*(1

5/5$stars$$.Opera*with*subtitles?**Who*knew?*I(had(the(great(fortune(to(aZend(a(dress(rehersal(for(The(Damnation(of(Faust.((This(updated(version(was(amazing.((1A(M(A(Z(I(N(G.(((The(theatre(is(gorgeous.((The(acoustics(are(incredible.(Go.((Go(now.1

Assumptions1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Text(has(a(single(main(sentiment(about(a(single(main(topic1

Sentiment(is(graded(on(a(linear(scale1Usually$positive/negative.

Words(carry(sentiment(information1

Analysis(=(Classification1

Very(liZle(semantics((if(any)1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Let’s*look*a*li=le*more*closely…2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1 Easy2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1 Hard2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

…but(it’s(generally(lousy?1

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

The(rest(of(it,(however,(…1

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(…(1

1

Positive?2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Negative?2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Positive?2Negative?2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Generally*positive2

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Negative*on*sound2

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Positive*on*price2

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Do(they(fit(the(archetype?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1

This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(fingernails.(1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(first(time.(1

Loud(but(nice(room(at(a(great(price(with(bad(view.1

When(the(tent(was(finally(completed,(the(result(was(much(nicer(than(that(Kiva(tent(

but(not(as(nice(as(the(Arctic(oven…1

Negative*on*view2

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

Does(it(maZer?1Pa=ern2 Results*(Google)2 Percentage2

X is good 4,820,000,000 X is as good as Y 2,530,000,000 52% X is better than Y 2,120,000,000 44% X is bad 1,730,000,000 X is worse than Y 437,000,000 25% X is a good Y 14,160,000,000 X is a better Y than a Z 781,000,000 6% X at a good price 888,000,000 X at a good price but ... bad Y 29,400,000 3%

8 November, 2011 Sentiment Analysis Symposium, San Francisco

AZitude

1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

We(believe(in(the(sincerity(of(the(United(States(in(promising(not(to(mix(up(its(counterNterrorism(drive(with(the(Taiwan(Strait(issue1

the(United(States(1 the(sincerity(of(the(United(States(in(promising(not(to(mix(up(its(counterNterrorism(drive(with(

the(Taiwan(Strait(issue1

AZitude

1

Topic1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

…what(is(important(is(that(your(Internet(connection(will(never(even(reach(the(speed(capabilities(of(this(router...1

important1 your(Internet(connection(will(never(even(reach(the(speed(capabilities(of(this(router1

AZitude

1

Topic1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

It’ll(make(life(a(lot(easier,(and(preclude(you(from(having(to(give(this(product(a(negative(review.((1

positive1 product1

AZitude

1

Topic1

Annotation(oddities1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

However,(this(product(performed(well(below(my(expectation.((1

negative1 router1

So(what’s(really(going(on?1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance(and(demeanor(are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1 Negative2

Positive2

Classification+

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance(and(demeanor(are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1 Negative2

Positive2

A-itude+

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance*and*demeanor*are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1

Target+

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her(appearance*and*demeanor*are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1

Superordinate+

The(Structure(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Her*appearance*and*demeanor*are(excellently(suited(to(her(role.1

SheFs(the(most(heartless(coqueZe(in(France!1

Aspect+

The(Sentiment(Complex1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

…the(result(was(much(nicer(than(that(Kiva(tent…(1

Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1

aZitude1target11 target21

target1 aZitude11 aZitude21

superordinate21superordinate11

The(Semantics(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

This unique dog or chicken house was made with green practices in mind.

Appreciation2Valuation1

Oh, and on principle I hate extremists / zealots of any ilk.

Affect2Affection1

Oh and Spooks, … with this really tough and sexy old lady in it, kicking ass.

Judgement2Tenacity1

The(Semantics(of(Sentiment1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Other(Key(Pieces1•  Modality

o  Maybe my throat will feel better o  The burgers are probably good but I went with the fish tacos.

•  Evidentiality o  Why are people always so angry? You're trying to prove that your mean

and it isn't cute nor do you need to be on a social website. o  -__- and she thinks that it's funny. BUT IT'S NOT!!! 

•  Sarcasm o  Nice to see Oakland Police attacking protesters again. Of course, it was

"agitators" at fault, not out of control cops.

•  Inferred sentiment o  College Without Weed Is Like A Notebook & No Pen..

8 November, 2011 Sentiment Analysis Symposium, San Francisco

The(Sentiment(Analysis(Stack1Collections/Streams:(Analysis,(comparison,(trending,(…1

Texts:(Aggregating(and(aZributing(sentiments(from(expressions1

Expressions:(Contextualization(and(interpretation1

Expressions:(Extraction(and(linking1

8 November, 2011 Sentiment Analysis Symposium, San Francisco

Shlomo Argamon argamon@subtext3.com @ShlomoArgamon

Subtext3, Inc.

http://www.subtext3.com @Subtext3

Illinois Institute of Technology Linguistic Cognition Lab

http://lingcog.iit.edu

8 November, 2011 Sentiment Analysis Symposium, San Francisco