Detecting the level of trust in governmentusing social media analysis

– case study of Korean and US government2015-2 TSMM research presentation

SuLyn HongJuYoung An

CONTENTS 1. Introduction

2. Related works and Implication of this study

3. Research methodology

4. Changes and Incorporated feedback

5. Result and Discussion

00

INTRODUCTION

01

1. Research background

2. Research problem

3. Definition of terms

Research background

• Trust in government is the core power of government.

• It is harder to secure legitimacy and work efficiently than to plan new policy.

• The loss of trust in government increases co-operation cost and acts as obstacle,

so could be a cause of policy failure.

• It would be helpful if analysis of social media to identify public opinion is

conducted on trust in government.

• We will attempt to compare the difference between the opinion and attitude of

Korean and American toward their governments on social media.

01

4

Research problem

1. How are the trust and attitude of Korean to the Korean government

expressed in various Social Media? What are public discourses related to

trust in the government?

2. Can social media data be an indicator for measuring the level of trust in

government?

3. How different the Korean’s perspective to the government and the

American’s perspective to the government which has a relatively long

history of democracy?

01

5

Definition of terms

• In this study, trust in government is defined to ‘the degree of trust which people

have on governmental performance, in other words, positive expect’. (K M Yang,

2007)

• In the case of Korean Social media, it is hard to differentiate the ‘government’

from ‘present government’ by keyword ‘government’, so in this study, we defined

the target of trust in government as a present government.

• We used a common word ‘government/정부’ as the initial keyword for data

collection, not specific names of administrative agencies.

• In case of American Social media, the target of trust in government is the federal

government which is governed by president.

01

6

RELATED WORK and IMPLICATION of this study

02

1. Related works

2. Implication of this study

Related works

• H J Son(2005). The studies on trust in government can be categorized to three

kinds; Theoretical discussion about importance of trust in government,

Composition and Measurement of concept on trust in government, Factors which

affects trust in government.

• O'Connor, et al. (2010). Analyzes public opinion measured by vote with emotion

extracted from text data. As a result, it is revealed that there is co-relation

between the frequency of emotional words generated in tweeter and trend of

public opinion. (about 80%)

02

8

Implication of this study

• Existing studies on trust in government use questionnaire survey, which costs

high and is limited to small sample. This study has an implication that identifying

trust in government empirically by using big social data.

• It is possible to catch various discussions from social media text, which is

impossible from closed questionnaire.

• In abroad, there are more studies to identify public opinion on government of

political issue than in Korea, but the data source is usually singular, not various,

for example, blog, Facebook or Twitter. (Griffiths, 2004)

• This study suggests a new possibility to an area of study on trust in government

by analyzing opinions of people in various social media channel to identify the

level of trust in government in various aspects.

02

9

RESEARCH METHODOLOGY

03

1. Overall phase

2. Data description

Overall phase03

• It is hard to find study on extracting emotion about ‘trust’ in text data.

• The important thing is how the emotion is extracted in Social media text.

• We decided to find supplement points while conducting the whole process with

the partial data in advance, and proceed the second experiment again.

• Mainly applying Topic Modeling to identify topics related to government.

• The emotion ‘trust’ should be found by emotion analysis rather than sentiment

analysis, so Topic modeling way will be helpful.

11

Overall phase03

12

Overall phase03

13

Data description03Meaning of data Data channel Description

1Opinion of media which produces

issueNews article

• It is impossible to get perfect objectivity even media deals with news• Literary style, so it has objectivity than other data• Korea: KINDS / US: EBSCO American news• Search query: ‘정부’ (Korea) / ‘government’ (US)

2Opinion without

establishingAgenda

Tweet• Use words related with government as keyword of data collection• Search query: ‘정부’, ‘정권’ (Korea) / ‘government’, ‘gov(gov’t)’ (US)• Filtering of english tweet during collection process

3Opinion with establishing

Agenda

Ko

News article + comment

• Considering the cultural difference of opinion presentation, select similar data source

• Korea: comment high rank news of ‘Naver news’ of Korea• US: ‘US Message Board’, ‘Debate Politics’ forums’ US > Politics topicU

SForum topic

+ reply

14

Data description03

Meaning of data Data channel Date range The # of data

The # of filtered

data

1Opinion of media which produces

issueNews article

Ko1995-01 ~ 2015-06

(10 years)668,820 565,409

US1995-01 ~ 2015-06

(10 years)532,986 207,704

2Opinion without

establishingAgenda

Tweet

Ko2009-07 ~ 2015-06

(6 years)8,393,551 57,668,422

US2009-07 ~ 2015-06

(6 years)57,683,814 17,360,556

3Opinion with establishing

Agenda

Ko

News article + comment

2006-07-01 ~ 2015-07-03(9 years)

6,727 / 6,826,141

6,490 /2,763,721

US

Forum topic + reply

2006-07 ~ 2015-06(9 years)

44,570 /1,776,857

32,896 /813,333

15

CHANGES and INCORPORATED FEEDBACK

04

1. English tweet filtering

2. Expansion of trust related words

English tweet filtering

Supplement point 1:

• In the case of English tweet, because English is used in many worlds, method to

differentiate the tweet related to the US government is needed.

04

17

English tweet filtering

Complementary measures:

In English tweet:1. Deleting data contains ‘USA’, ’US’, ’Obama’2. Ordering the word frequency to 100th, select the words represent other

countries3. Selection 30 noise words considering meaning of word

In the database:1. Lowercase tweet text body2. Select text does not contain ‘Obama’, ‘bush’, ‘us’, ‘usa’, ‘united state’3. Delete tweets contains noise words

04

18

English tweet filtering

Excluded words:

uk , british , scottish , bbc , britain, india, china, chinese, greece, delhi, nigeria,canada, pakistan, hong kong, italian, italy, japan, Japanese, EU, french, iraq, iraqi,syrian, iran, world news, world issue, global news, global issue, egypt, korea

deleted 4,703,883 (the rest: 58,759,436)

04

19

Supplement point 2:

• A lot of data are disappeared when the data are filtered by words related trust:

Need to expand scope of words

• Clear criteria when select topics related to trust is needed

• How we classify the words have neutral emotion such as just ‘trust’?

04

20

Expansion of trust related words

Complementary measures:

1. Quantitative expansion: Using word2vec, train data of phase 1, after than selectrelated word from word2vec result extracted by existing trust-related words asseed words

2. Secure trustworthy• Select the number of words have even distribution by applying reliable

governmental trust effect factor model.• Think about the way separate positive/negative words

04

21

Expansion of trust related words

Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of managementreview, 20(3), 709-734.

04

22

Expansion of trust related words

Ability

(능력)

Benevolence

(호의)

Integrity

(공정성)

• We did not simply select the common wordssuch as 'believe', 'trust', but selected thewords according to the 3 factors affect totrust of government.

• Examples:

o Ability: disappoint, fool, incompetent,reliable, solve

o Benevolence: help, vested right, mercy, welfare, protect

o Integrity: transparent, conspiracy, manipulate, lie, fraud

04

23

Expansion of trust related words

‘Trust’ related term list

Korean English

Ability능력

멍청, 어리석, 바보, 순진, 악마, 철없, 무모, 야만인, 어리석음, 경멸, 흰밥, 멋있, 부끄럽, 불쌍, 쓸데없, 괴짜, 괴상, 똑똑,

악동, 대단, …

dissapoin, outwit, entertain, ridicule, annoyance, fool, forget, stupid, dreadful, unenforceable, exempt, inconceivable, really, doubt, incompetent, competent, inept, insensitive, ineffective, corrupt,

…

Benevo-lence호의

특권, 권력, 밥그릇, 지역주의, 사익, 정치권력, 구체제, 당파, 수구, 이기주의, 스스로, 이념, 정당, 소수, 지배층, 정치, 헤게

모니, 계파, 대의, 대변자, …

help, assist, aid, encourage, assistance, desperately, collaborate, incentive, relief, wean, insure, enable, boost, induce, persuade, protect, distress, n

eedy, grant, rescue, …

Integrity진실성

불신, 불신감, 불안감, 실망감, 반감, 증오심, 반목, 분노, 피로감, 적대감, 신뢰, 적개심, 갈등, 혐오감, 공포감, 혼란, 도덕적

해이, 불협화음, 불화, 증오감, …

plot, scheme, fraud, bribery, charge, felony, murder, forgery, indictment, allege, racketeering, masterminding, case, misdemeanors, collusion, mzo

udi, obstruction, fraud, fraudulent, cheat, …

Result and Discussion

05

1. Difference of expression (in terms of medium/nation)

2. Difference of amount of data included in three

dimension

(ability / benevolence / integrity)

3. Difference of time series analysis

4. Discussion

Result 01: Difference of expression

1. News article• Similarity: Most words are ‘noun’s which indicate a certain object. There are just

a few topics related to trust.2. Tweet• Similarity: There are a lot of words indicate administration.• Difference: In Korea, people talk about certain person of the government. 27%

of tweet has words indicate the president of the government.In America, the objective words indicate government such as ‘government’ are

used. Only 5% of tweet has words indicate the president of the government.3. Forum comment• Difference: It could be difference from difference of data source, but in general,

the words from English data are more objective than the words form Koreandata. The most emotional word in English data is ‘stupid’.

05

25

Result 02: Difference of amount of data

1. The difference of mention directly related to trust according to mediumo In American news, there are relatively few mention directly related to trust,

so the amount of filtered data is smaller than in Korean news.o In Forum comment and tweet, because of the diversity of expression, the

amount of filtered data is smaller than the amount of filtered data fromother mediums.

2. In both America and Korea, there are more contents related to the ‘benevolence’of the government than contents related to the other factors of trust; It seemsthat both nation’s public has a big expectation on welfare or tax.

3. In Korea, there are more contents related to ability than contents related tointegrity, in America, it is opposite. Korean public refer on corruption and fraud,but American public are more interested in ‘ability’ of the government.

05

26

Result 02: Difference of amount of data05

27

0

0.2

0.4

0.6

0.8

1

en ko en ko en ko en ko

forum_comment forum_topic news tweet

Sum of filtered rate

Sum of Rate of Ability

Sum of Rate of Benevolence

Sum of Rate of Integrity

Result 03: Difference of time series analysis

1. In Korea, specific events have a big impact on topic fluctuations. Big tragedies arestrongly related to government’s ‘ability’.

05

28

MERS

Sewol ferry

Result 03: Difference of time series analysis05

29

2. In the US, there’s a little fluctuation among topics. The most fluctuated topicamong trust related topics is related with ‘Obamacare’.

3. The sharpest rising topic among the whole US forum data is about ‘governmentshutdown’(the level of topical distribution is 3.7). The expressions of the US forumcomments are relatively objective.

05

30

Discussion

• Overall, in Korea, public responds directly (and emotionally) to

social/political issues, but in America, public tends to collectively

express their own opinion about the issues (not emotional response)

and focuses on the political opinion.

• In Korean forum data, there is a certain period when topics are not

“hot issue.” In other words, public does not discuss those topics

frequently.

05 Discussion

There is a certain period when topics are relatively not “hot issue”

31

05 Things to be Done

32

• Detailed Analysis of Topic Modeling Results

REFERENCE

• 김대욱, & 이승종. (2008). 정부신뢰의 참여에 대한 효과. 한국사회와 행정연구, 18(4), 43-62.• 김용철. (2010). 네트워크 사회와 정부-시민 관계. OUGHTOPIA, 25(2), 103-131.• 김용학, & 손재석. (1998). 미시적 신뢰와 거시적 위험. 사상, 115-132.• 문원기, 이수범. (2015). 소셜미디어가 정치참여에 미치는 영향. 한국언론학보, 59(4), 133-162.• 박노일, 한정호. (2008). 블로그 쓰기와 사회정치참여에 관한 연구. 한국언론학보, 52(2), 282-302.• 박순애. (2006). 정부에 대한 이해와 정부 신뢰의 관계. 한국행정학보, 40(2), 73-97.• 박종민. (1991). 연례학술대회 발표논문: 정책산출이 정부신뢰에 주는 영향. 한국행정학보, 25(1), 291-305.• 박희봉, 이희창, & 조연상. (2003). 우리나라 정부신뢰 특성 및 영향 요인 분석. 한국행정학보, 37(3), 45-66.• 배정환, 손지은, & 송민. (2013). 텍스트 마이닝을 이용한 2012 년 한국대선 관련 트위터 분석. 지능정보연구, 19(3), 141-156.• 손호중, & 채원호. (2005). 정부신뢰의 영향요인에 관한 연구. 한국행정학보, 39(3), 87-113.• 심재승. (2011). 정부· 행정에 대한 시민신뢰를 둘러싼 논점 및 향후 연구의 방향성 탐색. 한국정책과학학회보, 15(4), 155-176.• 양건모. (2007). 정부신뢰 개념의 이론적 논의 및 타당성 검증. 한국행정학회 하계학술발표논문집, 2007(단일호), 1-22.• 오경민, & 박흥식. (2002). 일반논문: 정부신뢰 수준의 측정과 비교에 관한 연구. 한국정책학회보, 11(3), 113-137.• 이곤수. (2011). ‘삶의 질’관점에서 본 평택시의 정부신뢰. 서울: 동아시아연구원.• 이숙종. (2006). 정부신뢰와 거버넌스. 국정관리연구, 9(3), 143-172.• 이창호. (2012). 소셜미디어가 대학생들의 정치효능감, 정치관심 및 투표참여에 미치는 영향 분석-19 대 총선을 중심으로. 정치커뮤니케이션 연구, 27, 231-259.• 하승태. (2012). 소셜 네트워크 서비스 (SNS) 이용이 수용자의 정치사회적 태도에 미치는 영향. 언론과학연구, 12(4), 575-606.• Ausserhofer, J., & Maireder, A. (2013). National politics on Twitter: Structures and topics of a networked public sphere. Information, Communication & Society,

16(3), 291-314.• Butler, J. K. (1991). Toward understanding and measuring conditions of trust: Evolution of a conditions of trust inventory. Journal of management, 17(3), 643-

663.• Citrin, J. (1974). Comment: the political relevance of trust in government. American Political Science Review, 68(03), 973-988.• Griffiths, O. (2004). E-citizens: Blogging as democratic practice. Electronic Journal of E-government, 2(3), 1-10.• Kavanaugh, A., Kim, B. J., Perez-Quinones, M. A., Schmitz, J., & Isenhour, P. (2008). Net gains in political participation: Secondary effects of Internet on

community. Information, Communication & Society, 11(7), 933-963.• Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of management review, 20(3), 709-734.• Muller, E. N., & Jukam, T. O. (1977). On the meaning of political support. American Political Science Review, 71(04), 1561-1595.• O'Connor, B., Balasubramanyan, R., Routledge, B. R., & Smith, N. A. (2010). From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series.

ICWSM, 11(122-129), 1-2.• Welch, E. W., Hinnant, C. C., & Moon, M. J. (2005). Linking citizen satisfaction with e-government and trust in government. Journal of public administration

research and theory, 15(3), 371-391.• Younus, A., Qureshi, M. A., Asar, F. F., Azam, M., Saeed, M., & Touheed, N. (2011, July). What do the average twitterers say: A twitter model for public opinion

analysis in the face of major political events. In Advances in Social Networks Analysis and Mining (ASONAM), 2011 International Conference on (pp. 618-623).IEEE.

THANK YOU

TEAM.

홍수린(문헌정보 석사 2, 2014321156)안주영(문헌정보 석사 2, 2015311099)

Contact :

lynnn.hong@gmail.comjuyoung228@gmail.com