FACILITATING STATISTICAL ANALYSIS OF TEXTUAL...
Transcript of FACILITATING STATISTICAL ANALYSIS OF TEXTUAL...
Purdue Tourism & Hospitality Research Center
FACILITATING STATISTICAL ANALYSIS OF TEXTUAL DATA:
A TWO-STEP APPROACH
Svetlana StepchenkovaAndrei P. Kirilenko
Alastair M. Morrison
Purdue Tourism & Hospitality Research Center
Presentation Outline
• Introduction• Proposed methodology• Example 1: Russia’s affective images• Example 2: Organic images of China &
Russia• Discussion
Purdue Tourism & Hospitality Research Center
Introduction• Analysis of destination images provide insights into
consumer travel behavior (Ahmed, 1991)• Strong preference for structured image measurement
(Gallarza et al, 2002; Pike, 2002)• Comparative advantages of structured and
unstructured methodologies• Holistic images: theoretically researched but not
adequately tested• Large amounts of available data
Purdue Tourism & Hospitality Research Center
Introduction (cont.)
• Computer-assisted procedure• Aid in identification of image variables• Efficiently deal with a large number of
text blocks• Assist in “cleaning up” the data
Purdue Tourism & Hospitality Research Center
Two-Step Approach
Analysis of textual data with specialized software
Statistical analysis
1. Identification of destination image variables in textual files using CATPAC (Woelfel, 1998)
2. Counting the occurrences of identified variables in every textual file with WORDER (Kirilenko, 2004): 1000 files, 1000 variables
Statistical analysis of frequencies matrix with general-purpose statistical software (SPSS, SAS)
Purdue Tourism & Hospitality Research Center
Worder The problems of content analysis, such as different spellings, multi-word concepts, synonyms, singular/plural, negatives, are addressed and solved by WORDER. depressed
depression
despair
hopeless
dreary
drab
dark
darker
gloomy
glum
bleak
gray
grey
depressing
Purdue Tourism & Hospitality Research Center
Application Example 1
• Russia’s Destination Image online survey (Stepchenkova, 2005)
• Followed conceptual framework by Echtner & Ritchie (1993)
• “What images or characteristics come to mind when you think of Russia as a travel destination?” (psychological holistic, or affective, images)
• 337 survey responses, 317 textual responses
Purdue Tourism & Hospitality Research Center
Step 1: CATPAC
• Found all evaluative descriptors (around 240) in textual responses
• Combined them into 42 synonymic groups. One word for each group was selected as affective image variable
Example of a synonymic group:Austere, stark, Spartan, primitive, stoic, lack, minimal
Purdue Tourism & Hospitality Research Center
Step 2: WORDER• In the textual data words belonging to the same synonymic
group were replaced by the representative image variable
• Frequencies of all 42 affective image variables were counted by WORDER and entered into SPSS
WORDER Input Table (.csv format)
alcoholism alcohol vodka drugs drink too muchaustere stark spartan primitive stoic lack
minimalawesome awe beautiful great glorious grandeur
incredible wonderful wondrous
boring dull bland indifferencelack of enthusiasm
Purdue Tourism & Hospitality Research Center
Affective Image VariablesVariable Freq Variable Freq Variable Freqfriendly 85 free 11 alcoholism 6somber 47 open 11 hardworking 6depressing 45 interesting 11 festive 5unfriendly 28 austere 11 contrasts 5cold 18 hostility 10 happy 5poor 18 unhappy 10 uncomfortable 5reserved 17 pleasant 10 serene 4exciting 15 difficult 9 safe 4tense 15 sad 8 hopeful 4unsafe 15 cosmopolitan 8 ruthless 4good 15 cordial 8 seedy 4upbeat 14 cautious 7 historical 4awesome 14 boring 7 unpleasant 3undeveloped 13 fascinating 7 relaxing 2
Purdue Tourism & Hospitality Research Center
Favorability: OperationalizationAffective Variable Favorability Scores: Averaged evaluations
of 35 experts. Scale: -2, -1, 0, +1, +2 Variable Score Variable Score Variable Score fascinating 1.97 serene 1.53 tense -1.11 friendly 1.92 relaxing 1.47 boring -1.19 happy 1.83 cosmopolitan 1.44 difficult -1.19 exciting 1.81 upbeat 1.43 seedy -1.28 festive 1.78 free 1.36 uncomfortable -1.36 good 1.72 open 1.36 sad -1.42 awesome 1.72 contrasts 1.06 hostility -1.44 hardworking 1.69 reserved 0.08 ruthless -1.53 historical 1.67 cold -0.31 unhappy -1.56 safe 1.64 cautious -0.33 unfriendly -1.64 interesting 1.61 somber -0.39 depressing -1.67 pleasant 1.58 austere -0.41 unpleasant -1.68 cordial 1.56 undeveloped -0.58 alcoholism -1.75 hopeful 1.53 poor -1.00 unsafe -1.78
Purdue Tourism & Hospitality Research Center
Favorability Variable OperationalizedResponse example:“Fascinating country. Overall, people are caring but emotionally controlled. Boring nightlife, dull food, though”:“Fascinating country. Overall, people are friendly but reserved. Boring nightlife, boring food, though”:
Fascinating = 1.97; Friendly = 1.92; Reserved = 0.08; Boring = -1.19
1.97+1.92+0.08-1.19-1.19 = 1.59
Descriptive Statistics
337 -6.0832 8.7222 .326728 2.2402324337
FavorabilityValid N (listwise)
N Minimum Maximum Mean Std. Deviation
WORDER
EXPERTS
Response overall favorability value
Purdue Tourism & Hospitality Research Center
Hypothesis testing: ResultsHypothesis: People who have visited Russia previously have more favorable images of Russia than those who have not.
Levene's Test for Equality of
Variances t-test for Equality of
Means Visitation N Mean F Sig. t df p-value
visitors 54 0.808 3.572 0.060 1.726 335 0.085 non-visitors 283 0.235
n1=54: Normal distribution confirmed
Purdue Tourism & Hospitality Research Center
Application Example 2• Comparison of organic images of China and Russia in the
U.S. media (Stepchenkova, Chen, & Morrison, 2005)• 2002-2004 general news articles from U.S. regional
sources: Midwest, Northeast, Southeast, and Western.• LexisNexis database: Words “China” or “Russia” in
headlines.• Systematic random sampling: 540 + 540, 15 per month.
Purdue Tourism & Hospitality Research Center
Two-Step Approach
• Step 1. CATPAC: Organic image variables, frequencies.• Step 2. WORDER: Counted the identified image variables
in every file of “Chinese” and “Russian” samples.
In SPSS: Factor Analysis to identify organic image themesQualitative part:Assignment of a favorability rating to organic image themesFavorability comparisons of organic images
Purdue Tourism & Hospitality Research Center
Organic Image Variables
companies 658 city 316 industry 216 steel 190government 586 Taiwan 302 u.s. 213 growth 190military 499 foreign 275 power 212 leaders 184country 474 state 268 Bush 209 job 179market 462 president 265 technology 207 Shanghai 178official 433 economic 246 economy 201 American 176trade 382 price 238 high 198 Asia 174officials 355 Hong Kong 228 school 196 cost 173
Moscow 729 people 428 Iraq 312 weapons 274Putin 657 government 421 international 310 US 269year 530 states 418 country 308 former 263united 472 time 409 against 302 security 258world 461 nuclear 389 percent 296 military 258president 459 war 366 company 291 foreign 254Soviet 454 American 324 officials 286 Bush 252oil 448 state 317 million 278 Kremlin 246
China
Russia
(first 32)
(first 32)
Purdue Tourism & Hospitality Research Center
Factor Analysis: China and Russia• Number of subjects: 540 and 540• Number of variables: 83 and 70
• KMO statistic of sampling adequacy: 0.766 and 0.752• Bartlett’s test: p < 0.0001• Principal Components Analysis• Direct Oblimin Rotation, allows factors to co-vary (Kline,
1994)• Number of factors specified: 15• Variance explained: 57.2% and 58.1%
Purdue Tourism & Hospitality Research Center
China: Main Organic Image Themes
0.7792.152students, teamEducational Exchange15
0.6632.484security, council, againstSecurity Concerns14
0.4232.909power, nuclear, American, KoreaAsian Politics13
0.5832.940Wang, local, communist party, Communist China12
0.9403.121human rights, world, yearHuman Rights11
0.6993.140health, disease, SARS, news, province, city, Hong KongSARS10
0.6363.161public, school, Shanghai, U.S., old (year-old), helpCultural Communication9
0.6723.176job, high, work, construction, centerLabor Market8
0.7843.537Jiang, Hu, Wen, leaders, presidentGovernment7
0.7453.943administration, technology, Bush, sales, export, officials, LiTechnology Transfer6
0.7263.967military, official, government, Taiwan, Asia, WashingtonTaiwan5
0.6234.048state, major, global, market, university, ChineseGlobal Market4
0.7124.464WTO, trade, foreign, import, countryWorld Trade Organization (WTO)3
0.7264.717demand, price, cost, steel, industry, company, workersIndustry2
0.5875.911economy, bank, money, growth, central, investment, economic, ChinaEconomic Growth1
Cronbach’s Alpha
Varianceexplained
ItemsFactorNo.
0.7792.152students, teamEducational Exchange15
0.6632.484security, council, againstSecurity Concerns14
0.4232.909power, nuclear, American, KoreaAsian Politics13
0.5832.940Wang, local, communist party, Communist China12
0.9403.121human rights, world, yearHuman Rights11
0.6993.140health, disease, SARS, news, province, city, Hong KongSARS10
0.6363.161public, school, Shanghai, U.S., old (year-old), helpCultural Communication9
0.6723.176job, high, work, construction, centerLabor Market8
0.7843.537Jiang, Hu, Wen, leaders, presidentGovernment7
0.7453.943administration, technology, Bush, sales, export, officials, LiTechnology Transfer6
0.7263.967military, official, government, Taiwan, Asia, WashingtonTaiwan5
0.6234.048state, major, global, market, university, ChineseGlobal Market4
0.7124.464WTO, trade, foreign, import, countryWorld Trade Organization (WTO)3
0.7264.717demand, price, cost, steel, industry, company, workersIndustry2
0.5875.911economy, bank, money, growth, central, investment, economic, ChinaEconomic Growth1
Cronbach’s Alpha
Varianceexplained
ItemsFactorNo.
Purdue Tourism & Hospitality Research Center
Russia: Main Organic Image Themes
0.6781.871Beslan, schoolTerrorism15
0.7252.318space, station, programU.S.-Russia Space Cooperation14
0.4892.444team, national, world, timeSports13
0.6352.778power, place (take place), investmentPower Sector Reform 12
0.5932.823Chinese, China, RussiansRussia-China Relations11
0.5122.841NATO, defense, Bush, countriesNATO10
0.6733.198family, home, children, year-old, wantRussian Children in the U.S.9
0.7303.291nuclear, weapons, Iran, securityIran8
0.7203.519Soviet, Union, former, yearsSoviet Past7
0.6673.554Chechen, Chechnya, people, war, MoscowChechnya6
0.6903.691oil, percent, gas, Russia’s Russia, million, governmentNatural Monopolies5
0.5953.849law, foreign, Russian, international, against, countryLaw4
0.7474.110election, Vladimir, Putin, political, party, Kremlin, presidentPresidential Elections3
0.7764.732United, States, Iraq, nations, AmericanIraq2
0.8104.936Yukos, company, Khodorkovsky, state, billion, business, companiesYukos1
Cronbach’s Alpha
Varianceexplained
ItemsFactorNo.
0.6781.871Beslan, schoolTerrorism15
0.7252.318space, station, programU.S.-Russia Space Cooperation14
0.4892.444team, national, world, timeSports13
0.6352.778power, place (take place), investmentPower Sector Reform 12
0.5932.823Chinese, China, RussiansRussia-China Relations11
0.5122.841NATO, defense, Bush, countriesNATO10
0.6733.198family, home, children, year-old, wantRussian Children in the U.S.9
0.7303.291nuclear, weapons, Iran, securityIran8
0.7203.519Soviet, Union, former, yearsSoviet Past7
0.6673.554Chechen, Chechnya, people, war, MoscowChechnya6
0.6903.691oil, percent, gas, Russia’s Russia, million, governmentNatural Monopolies5
0.5953.849law, foreign, Russian, international, against, countryLaw4
0.7474.110election, Vladimir, Putin, political, party, Kremlin, presidentPresidential Elections3
0.7764.732United, States, Iraq, nations, AmericanIraq2
0.8104.936Yukos, company, Khodorkovsky, state, billion, business, companiesYukos1
Cronbach’s Alpha
Varianceexplained
ItemsFactorNo.
Purdue Tourism & Hospitality Research Center
Favorability Rating: China• Favorable (+1):
– Economic Growth– Industry– WTO– Global Market– Technology Transfer– Cultural
Communications– Educational Exchange
• Unfavorable (-1):– Taiwan– SARS– Human Rights– Communist China
• Neutral (0):– China’s Government– Labor Market– Asian Politics– Security Concerns
Purdue Tourism & Hospitality Research Center
Favorability Rating: Russia• Favorable (+1):
– Russian Children in the U.S.– Power Sector Reform– Sports– U.S.-Russia Space
Cooperation
• Unfavorable (-1):– Yukos– Iraq– Chechnya– Iran– Law– Terrorism• Neutral (0):
- Presidential Elections - Natural Monopolies - Russia-China Relationships- Soviet Past- NATO
Purdue Tourism & Hospitality Research Center
Comparison of Organic Images
-6
-4
-2
0
2
4
6
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Factors
Vari
an
ce e
xp
lain
ed
RussiaChina
Economic Growth
Sports
Yukos
SARS
Terrorism
Technology Transfer Favorable
Unfavorable
Purdue Tourism & Hospitality Research Center
Aggregated Organic Image Themes
Chechnya; TerrorismSARSSafety
Presidential ElectionsGovernment; TaiwanInternal affairs
Iraq; Iran; NATO; Russia-China Relations; Soviet Past
Asian Politics; Security ConcernsForeign policy
Russian Children in the U.S.; Sports; U.S.-Russia Space Cooperation
Global Market; Cultural Communications; Educational Exchange
Exchange(education, technology, sports, human relations)
Yukos; Natural Monopolies; Power Sector Reform
Economic Growth; Industry; WTO; Global Market; Technology Transfer; Labor Market
Economy
Yukos; Law; ChechnyaHuman Rights; Communist ChinaHuman rightsRussiaChinaCategory
Chechnya; TerrorismSARSSafety
Presidential ElectionsGovernment; TaiwanInternal affairs
Iraq; Iran; NATO; Russia-China Relations; Soviet Past
Asian Politics; Security ConcernsForeign policy
Russian Children in the U.S.; Sports; U.S.-Russia Space Cooperation
Global Market; Cultural Communications; Educational Exchange
Exchange(education, technology, sports, human relations)
Yukos; Natural Monopolies; Power Sector Reform
Economic Growth; Industry; WTO; Global Market; Technology Transfer; Labor Market
Economy
Yukos; Law; ChechnyaHuman Rights; Communist ChinaHuman rightsRussiaChinaCategory
Purdue Tourism & Hospitality Research Center
Comparison of Organic Images
0123456
Human rights
Economy
Exchange
Foreign policy
Internal affairs
Safety
RussiaChina
Purdue Tourism & Hospitality Research Center
Discussion• Large number of similar textual files (survey
responses, newspaper articles, etc.)• Different types of textual data• Enhancement of numerical data matrices by contextual
variables – data source, type, length, time, etc. (visitors/non-visitors in Ex. 1)
• “Embellishment” by secondary variables, e.g., “favorable-unfavorable”
• Flexibility of the two-step approach: can be combined with quantitative (Ex. 1) and qualitative (Ex. 2) methods
• “Black box” issue
Purdue Tourism & Hospitality Research Center
Discussion (cont.)
• Interpretational dimension (latent vs. manifest variables)
• Structural dimension: thematic, semantic, or neural (Roberts, 2000)
• Generalization issue• Appropriateness of the approach
Purdue Tourism & Hospitality Research Center
Selected References• Alexa, M. and Zuell, C. (2000). Text analysis software: Commonalities, differences and limitations: The
results of a review. Quality and Quantity, 34, 299-321.• Echtner, C. M. and Ritchie, J.R. B. (1993). The measurement of destination image: An empirical assessment.
Journal of Travel Research, 31(4), 3-13. • Gray, J. H. and Densten, I. L. (1998). Integrating quantitative and qualitative analysis using latent and
manifest variables. Quality and Quantity, 32, 419-431.• Jenkins, O. H. (1999). Understanding and measuring tourist destination images. The International Journal of
Tourism Research, 1(1), 1-15. • Insch, G. S. and Moore, J. E., (1997). Content analysis in leadership research: Examples, procedures, and
suggestions for future use. Leadership Quarterly, 8(1), 1-25.• Kirilenko, A. P. (2004). WORDER (Version 2.0) [Computer software]. Http://kirilenko.org/worder.
[Accessed the 18th of November 2005, 10:20]. • Krippendorf, K. (1980). Content analysis: An introduction to its methodology. Newbury Park, CA: Sage.• Pike, S. (2002). Destination image analysis – a review of 142 papers from 1973 to 2000. Tourism
Management, 23, 541-549.• Roberts, C. W. (2000). A conceptual framework for quantitative text analysis. Quality and Quantity, 34, 259-
274.• Stepchenkova, S. (2005). Russia’s destination image among American pleasure travelers. Master’s thesis.
Purdue University, West Lafayette, IN.• Stepchenkova, S., Chen, Y. and Morrison, A. M. (2005). China and Russia: A comparative analysis of
organic destination images. The 11th APTA Conference Proceedings, Vol. 1. New Tourism for Asia-Pacific (pp. 273-283).
• Weber, R. P. (1985). Basic content analysis. Beverly Hills, CA: Sage.• Woelfel, J. (1998). CATPAC: Users guide. New York, NY.: RAH Press: The Galileo Company.
Purdue Tourism & Hospitality Research Center
Thank you!
Questions?