A self-assessment tool.pdf

38 IEEE POTENTIALS0278-6648/12/$31.002012IEEE

Digital Object Identifier 10.1109/MPOT.2012.2187104

Now is the age of anxiety. W.H. Auden

hen American poet and dramatist W.H. Auden said the above line during the Great Depression, he must have been feeling great fear and worry about his self-existence. Today,

we live in an age when there is no escape from fear, stress, frustration, and anger. Undoubtedly, every human being suffers a great deal in one way or another. In our everyday life, we need a variety of information from various sources, which in our age include social networking Web sites, news, blogs, and wikis. In this present era, knowingly or unknowingly, we consume a lot of information from these sources. We try to create, read, and write using these popular tools. Most of the time, it is not easy for us to distinguish the good from the bad and the bad from the ugly. On one hand, we feel happy and alive when we are connected in this age of the Internet. On the other hand, we may be unaware of our own actions or behavior. S. Aydin suggests that the Internet in itself does not cause anxiety; rather, it is the users participation situation that results in anxiety.

Date of publication: 31 October 2012

dIgITAL vISION

Santosh Kalwar

A self-assessment tool for measuring Internet anxiety

W

NOvEMBER/dECEMBER 2012 39

Fig. 1 A user on the Internet experiencing different factors.

Our behavior on the Internet is likely to cause stress, possibly because of the numerous sources and contents we use. This may sometimes lead to psychologi-cal problems, mental illness, and anxi-ety. It is apparent that we are the victims of the different types of information that we use.

Preece et al., in their book Human-Computer Interaction, suggest that such interaction has several key factors, includ-ing health and safety, that play a major role in our daily habit of using the con-tents on the Internet. Based on the infor-mal observation, we can say that the routine tasks of sharing, Tweeting, and posting on the Internet can be fun. How-ever, many times the social networking experience may turn out to be a social-not-working experience. Therefore, to measure users anxiety in using the contents on the Internet, we devised set of questionnaires and presented them to users.

In this article, we first provide some background of our study. Second, we describe our study goalto measure users anxiety on the Internet by provid-ing them with a score sheet. Third, we raise a research question, which we answer by means of qualitative research methods. Fourth, we describe our meth-ods and procedures via a self-assessment measurement tool. Finally, we conclude with a brief discussion of future works.

Measured anxietyCharles Darwin and Sigmund Freud

first formulated the ideas of emotional state and anxiety in human beings. For centuries since then, researchers and practitioners in the field of clinical psy-chology and psychoanalysis have been working on finding effective measures of human anxiety.

Many researchers agree that anxiety is a very complex phenomenon to explore. The lack of appropriate measuring instru-ments and the various moral or ethical problems associated with measuring human anxiety have made Internet anxi-ety an even more complicated phenom-enon. For simplicity in our work, we have categorized human behavior on the Internet based on behavioral (e.g., addic-tion, avoidance), physical (e.g., head-ache, nausea), cultural (e.g., Asian, Western), cognitive (e.g., lack of focus, attention, and concentration), psycho-logical (e.g., attitude, beliefs), emotional (e.g., fear, frustration, anxiety), and anxi-ety (e.g., general anxiety, Internet anxi-ety) traits (see Fig. 1). A wide variety of techniques are used to measure anxiety, including questionnaires, rating scales, and laboratory tests. The most well known include the Beck Anxiety Inven-tory (BAI), the Spielberger State-Trait Anxiety Inventory (STAI), the General-ized Anxiety Disorder Severity Scale (GADSS), the Penn State Worry Ques-

tionnaire (PSWQ), and the Hamilton Anxiety Scale (HAM-A). A brief summary of these popular general anxiety mea-surement techniques is shown in Table 1.

From our perspective, these existing measuring techniques suffer from two shortcomings. First, they rarely consider Internet anxiety measurement, and none of them focus on the Internet. Second, they are general anxiety measuring tools developed a long time ago for general use. Therefore, these shortcomings con-tributed to the paucity of this research.

The question is...Based on the rationale of our study,

the following research question and hypothesis were constructed:

RQ1: How can we measure Internet anxiety (as distinct from other types of anxiety)?

H1: Since users have some level of anxiety on the Internet, a self-assessment tool may be used to measure Internet anxiety.

Our goal was to create questions and measure the level of anxiety of users. For this, we created a simple process and came up with 35 questions with the help of the users. First, we divided the users into three different sets to get more feed-back. The first and second sets consisted of two users each. The final set was com-posed of three users. The procedure developed for generating questions is self-explanatory, as shown in Fig. 2. We recruited users and asked them general questions on Internet anxiety. We then validated those questions through two iterations, first with seven users and then with 35 users. Finally, we provided the users with a score sheet.

Our first task in this study was to hire a select group of participants with techni-cal expertise. We conducted a prelimi-nary interview with these participants to confirm that they were tech savvy. We then asked questions about Internet anxiety and recorded their data by a note-taking mechanism. The appropriate questions were developed through an iterative mechanism. The final question was selected only after it was found suit-able enough. Finally, we came up with questions that could be used to measure Internet anxiety. We conducted a prelimi-nary study using the above process and procedure and then selected 35 ques-tions from the discussion with the users. In particular, 35 questions were asked, five for each participant. During a thor-ough discussion with the users, the majority of these questions were raised.

Behavioral(e.g., Addiction, Avoidance)

Physical(e.g., Headache, Nausea)

Cultural(e.g., Asian, Western)

Cognitive(e.g., Lack of Focus or,

Concentration)

Psychological(e.g., Attitude, Beliefs)

Emotion(e.g., Fear, Frustration, Aversion)

Anxiety(e.g., General Anxiety,

Internal Anxiety)

A User

40 IEEE POTENTIALS

Thus, the Internet anxiety self-assessment tool was developed. We asked all seven participants to answer the questions in two to four weeks time. We formulated a score sheet based on the 35 questions. There were two attributes pertaining to each question, i.e., true and false. We presumed that the total number of true responses provide the level of Internet anxiety. Our assumed score sheet looked exactly as follows, where x is the score obtained by the participant.

A score of 0,5x,5 5 very low anxiety.A score of 5,5x,10 5 low anxietyA score of 10,5x,15 5 medium anxietyA score of 15,5x,20 5 high anxiety.A score of 20,5x,25 5 very high anxiety.

On to the setupThe study was conducted in two itera-

tions involving 80 participants (popula-tion size 5 80). The sample size was seven in the first iteration and 35 in the second iteration. The participants were researchers, students, and our colleagues from the Department of Information Technology at Lappeenranta University of Technology. The selected participants were familiar with the Internet both tech-nically and in general. They reported on the severity of Internet anxiety.

In the first iteration, the research was conducted using a semistructured

Internet Anxiety CreateQuestion

35 Questions

Validate Questions

I II

Level ofInternet Anxiety

1+ +...(n 1)...+ n

(Validate with n Number of Participants)

III

Fig. 2 The process and procedure that was used to create questions.

Fig. 3 An example question.

Table 1. Comparison of anxiety measuring techniques.

Measuring Techniques Description Usage Advantage Disadvantage Reference

Beck Anxiety Inventory (BAI)

Developed during 196121 multiple choice questionsseverity of individual anxiety

Variety of different groups (adult, elderly)Used to measure panic, depressionIt has been used in over 2,000 empirical studies

Most widely used by researchers and clinical practionersEasy to take test

Usually unsuitable in social settingsOnly measures very general anxiety

Beck, T. Aaron et al., P. Richter, et al.

Spielberger State-Trait Anxiety Inventory (STAI)

Developed during 1960Two sections i.e., state and trait, each having 20 questions

Used to measure anxiety in adultsOnly anxiety not depression

It has been adapted into 48 languagesCan be completed in less than ten min.Reliable and brief self report

Unsuitable to measure both depression and anxiety

C.D. Spielberger et al.

Generalized Anxiety Disorder Severity Scale (GADSS)

Focuses on general anxiety 12 items that are rated on a nine-point Likert scale (08)

Used to measure anxiety especially physical: worry, tension mood, satisfaction with life

Widely usedCost efficient

Not used to measure other categories such as behavioral or cognitive disorders

Carmen Andreescu et al., T.D. Borkovec et al.

Penn State Worry Questionnaires (PSWQ)

Developed during 1990 16-item inventory

Used for both clinical and nonclinical populations

Consistent and reliable inventory

Minimal researchSpecific and focuses mainly on worry

D.M. Fresco et al.

Hamilton Anxiety Scale (HAM-A)

First introduced during 1960Originally 17 questions, 5-point scale

Used to measure low mood, agitation, anxiety, and weight loss

Focuses on those who suffer great deal with anxiety, and depressionSimple to use

Mainly used for depression

M. Hamilton


interview among seven participants dis-tributed in a group. These participants included five males and two females, as shown in Table 2. The total population (N 5 7), standard deviation (2.058), pop-ulation variance (3.63), and population standard deviation (1.906) were rela-tively very low, which indicates that the ages of the participants were very close to the mean age.

In the second iteration, 40 partici-pants answered questionnaires distrib-uted over the Internet; 35 questions were considered valid. Two answers were missing data on USER_ID. Another three were considered outliers (box plot). A simple Web-based question-naire was created (see Fig. 3) for the participants to select their appropriate answers. This Web-based tool featured a progress bar that indicated the per-centage of questions answered, a simple drop-down button for answer selection, and another button for navigating through the questions.

Expert validationWe recruited experts from the Jour-

nal of Cyberpsychology, Behavior, and

Social Networking, contacting them via e-mail. Fifteen experts were contacted, but only a few responded to our e-mail query. We gave them the 35 question-naires that were posited and asked for their opinions. According to one expert, the questionnaires leaned more toward addiction than anxiety. She suggested giving these questionnaires to a large sample in order to allow validation using factorial analysis.

Another expert reported that it would be better to put anxiety in a much broader perspective. He put forth a prerequisite question on validity by asking: Are we talking about pure autonomic arousal, a foreboding sense that something is not right, and misper-ceptions, reactions to the visual and auditory stimuli being presented? Meanwhile, another expert suggested we use her validation questionnaires, which we determined were unfit for our study.

Perhaps the best analysis came from another expert who suggested a factor analysis to identify subscales for our questionnaires and a test of reliability (e.g., Cronbachs alpha) for each sub-scale. Finally, one expert stated that the Kuder-Richardson formula is analogous to Chronbachs alpha; hence, the use of

Table 2. The standard deviation of the age of participants.

Sample Standard Deviation

First Iteration Sample Size

Population Variance

Mean Age

Population Standard Deviation

Sample Variance

2.0587 7 3.6328 26.2857 1.906 4.2382

Measure Your Internet Anxiety

How much anxiety do you have on the Internet? Click True or False according to your response to each statement.

1. True False While browsing the contents on the Internet, I find myself lost in hyperspace.2. True False If I were to take an anxiety test, I would worry a great deal before taking it.3. True False While browsing contents, I spend a great deal of time on it.4. True False During browsing, I find myself thinking of things unrelated to the actual browsing materials.5. True False I get to feeling very panicky when I do not find what I am looking for on the Internet.6. True False During contents browsing, I find myself thinking of YouTubing, Twittering, and Facebooking.7. True False I sometimes feel my heart beating very fast while I am on the Internet.8. True False I usually get depressed after I do not find what I am looking for on the Internet.9. True False I have an uneasy, upset feeling before browsing contents on the Internet.10. True False When browsing contents on the Internet, my emotional feelings interfere with my performance.11. True False During browsing contents on the Internet, I frequently get so nervous that I forget facts I really know.12. True False I seem to defeat myself while browsing contents on the Internet.

Fig. 4 Block 1, questions 112.

13. True False The harder I work on the Internet or study on the Internet, the more confused I get.14. True False As soon as Internet browsing is over, i try to stop worrying about it, but I just cant.15. True False During browsing contents, I sometimes wonder if III ever get back to browsing again.16. True False I would rather be online than offline.17. True False I wish the Internet did not bother me so much.18. True False I think I could do much better work on the Internet if I could be alone.19. True False Thinking about the social networking (Facebook, Twitter, Myspace, YouTube) I may get better performance.20. True False If Internet is unavailable all the time, I think I would actually learn less.21. True False On Internet I take the attitude, If I don't know it now, theres point in worrying about it.22. True False I feel upset while browsing the Internet.23. True False Thoughts of my experience on the Internet, I often share with my friends.24. True False I study much harder on the Internet.25. True False Even when Im well prepared on what I am looking for on the Internet, I feel very anxious about it. I dont enjoy.


42 IEEE POTENTIALS

either would not make much difference in the results.

Lets review the resultsThere were two sets of results obtained

from this work. In the first set of results, we developed 35 questions that we believe have significant contribution to human-computer interaction (HCI). For the pur-pose of this discussion, the questions are shown in three separate blocks [Block 1, Block 2, and Block 3 (see Figures 46)].

We asked the first set of users ques-tions such as, Do you feel lost in hyperspace? One user said, Some-

times, I feel lost while browsing... Thus, we formulated Q1. Similarly, we asked them, Would you worry if you had to take an anxiety test? One user said, Yes, I may worry to some extent. Another user reported that she usually hangs out in social networking appli-cations, which yielded Q6. With the second set of users, we asked: Do you feel worried after you finish browsing? One user said she never worries. We then formulated Q15. Similarly, one user said, I really dont see why some people get so upset about browsing the Internet. Thus, we formulated Q22.

With the third set of users, we asked questions such as, Do your hands or legs tremble while browsing the Inter-net? A user said: Sometimes, it happens We then formulated Q26. During the discussion, one user said he uses the contents on the Internet every day, which was changed to: I extensively use abusive contents on the Internet. Similarly, one user reported that she worries a great deal when there is no Internet connection; thus, we formulated Q30. The other questions were formulated following the same procedure.

Our second set of results consists of the score sheet of the seven partici-pants. The results in the score sheet showed that the majority of the partici-pants had a high level of anxiety; two participants had low anxiety and the other participants had high anxiety, as shown in Table 3. The table also shows that user ID 1 had high anxiety, whereas user ID 4 had low anxiety. The user with low anxiety said, I know exactly what I am doing on the Internet, and I dont see it as a fearful or depressing thing; I have total con-trol over my behavior on the Internet. On the other hand, user ID 1 with high anxiety said, I feel fearful, depressed, and frustrated when the Internet connection is not working, or due to unnecessary pop-ups and advertisements.

MeasuresThere are popular and standard sta-

tistical software packages used for data mining and statistical analysis, including SPSS from IBM. An analog open-source software such as PSPP can also be used. Since our sample size was relatively smaller, Del Siegles reliability calculator was preferred. The following measures were calculated (see Table 4):

Cronbachs alpha for calculating the coefficient of consistency

split-half (odd-even) correlation for splitting the test in half (odd) and comparing test with remaining half (even) to test reliability

Spearman-Brown prophecy to re-flect the entire test length

Kuder-Richardson formulas 21 (KR21) and 20 [KR20; see (1) and (2)] for calculating the reliability of the self-assessment tool. It is easy to compute and measure the reliability of items of a true/false nature. The results illus-trate a conservative estimate of the tests reliability.

26. True False Before browsing contents on the Internet, I find my hand or arms trembling.27. True False I started feeling very uneasy just after I finished browsing contents on the Internet.28. True False I have experienced anger when using contents on the Internet.29. True False I extensively use contents on the Internet.30. True False I worry a great deal if there is no Internet connection.31. True False I have usage problems on the Internet.32. True False During browsing contents, I sometimes wonder about particular terminologies on the Internet.33. True False I am often distracted by eye-catching applications on the Internet.34. True False I have experienced fear of using contents on the Internet.35. True False I am an Internet addict.


Table 3. Statistical score sheet from the seven users.

Questions Subjects

35 7

USER_ID TRUE FALSE SCORE

1 20 15 High

2 20 15 High

3 16 19 High

4 9 26 Low

5 16 19 High

6 16 19 High

7 6 29 Low

Cronbachs Alpha 0,708263069

Split-Half (odd-even) Correlation 0,555392325

Spearman-Brown Prophecy 0,714150785

Mean for Test 14,71428571

Standard Deviation for Test 4,919764388

KR21 0,666699732

KR20 0,708263069


The formulas are:

KRnn

nS

nX X S21

11 2 2 2#- =

-- -` j= G) 3

(1)

and

/KRnn s pq sX21

0 x x2 2#- =-

-^ h8 6B @ (2)wheren is the number of test itemsX is the average scoreS is the standard deviationsx2 is the variance of scoresp is percentage answering item rightq is percentage answering item wrongpq is the sum of pq products for all n items.

More samplesSince the first iteration consisted of

only seven users, its validity was ques-tionable. Thus, in the second iteration, 35 Internet users were asked to answer the questionnaires using our simple web-based tool (see Fig. 3). The score sheet obtained from the 35 users is shown in Fig. 7.

Conclusion and future worksIn this article, we described how

we formulated a self-assessment tool for measuring human anxiety on the Internet. The internal consistency, measured by Cronbachs alpha coeffi-cient (a 5 0.71), and the reliability (KR-21 5 0.67) were shown to be sat-isfactory. The higher reliability score indicates a higher internal consistency among the items tested. In other words, it indicates a stronger relation-ship between items, which assesses anxiety.

In the first iteration, the results showed that the users in the sample possessed a high level of Internet anx-iety. Similarly, in the second iteration, 31% of the sample showed a medium level of Internet anxiety, whereas only 11% had a very low level of Internet anxiety. These findings suggest that Internet anxiety may be a significant condition leading to stress, mental dis-order, trauma, addiction, and behav-ioral disorder. Furthermore, the results fully support the hypothesis that a self-assessment tool can be used in order to measure Internet anxiety. Therefore, we believe that this self-assessment tool is a significant contribution to the measure-ment of users Internet anxiety level in general.

The following points illustrate the sig-nificance of a self-assessment tool for measuring Internet anxiety.

It is expected that a self-assessment tool will encourage users involvement in measuring Internet anxiety.

It can be assumed that a self- assessment tool will give users increased responsibility, autonomy, and a deeper understanding of their own selves.

It can be concluded that a self- assessment tool for measuring Internet anx-iety gives a thorough and critical reflection.

It is presumed that a better under-standing of users subjectivity, decisions, and judgment can be investigated.

In the future, highly advanced, automated, and effective technical tools for the design and development of HCI can be created using a self-assessment tool.

A self-assessment tool provides more relevant feedback to users and generates active discussion among user peer groups.

A self-assessment tool can pave the way toward finding techniques to reduce Internet anxiety.

To our utmost surprise, the majority of the participants showed a medium level of anxiety. The reason for our sur-prise was simply that we were not expecting our participants, who had technical expertise, to possess Internet anxiety at all. We believe that our self-assessment tool is valid for measuring anxiety among Internet users, but this needs to be validated further with a larger sample.

This work has a few limitations. The first is the selection of participants, who were all students; thus, a self-report of Internet anxiety may not be the best validation for the scale. Although a few experts analyses were used in validation, these might still be insufficient. An expert third-party observer may provide a more objective

score. Moreover, a larger and more diverse sample should be used in the future. Thus, in the near future, we plan to engage more participants, e.g., by utilizing a self-assessment tool in a social networking application (e.g., Facebook, Google+) or by conducting a survey among a larger and more diverse population.

AcknowledgmentThe author would like to thank the

anonymous reviewers, colleagues, and Prof. Jari Porras and Dr. Kari Heikkinen for supervising the writing process of this article.

Read more about it A. T. Beck, N. Epstein, G. Brown, and R. A. Steer, An inventory for mea-suring clinical anxiety: Psychometric properties, J. Consult. Clin. Psychol., vol. 56, no. 6, pp. 893897, Dec. 1988. P. Richter, J. Werner, A. Heerlein, A. Kraus, and H. Sauer, On the valid-ity of the Beck Depression Inventory. A review, Psychopathology, vol. 31, no. 3, pp. 160168, 1998. C. D. Spielberger, R. L. Gorusch, and R. E. Lushene. Manual for the State-Trait Anxiety Inventory. Palo Alto: Con-sulting Psychologist Press, 1970. C. Andreescu, B. Herbeck Belnap, B. L. Rollman, P. Houck, C. Ciliberti, S. Mazumdar, M. K. Shear, and E. J. Lenze, Generalized anxiety disorder severity scale validation in older adults, Amer. J. Geriatric Psychiatry, vol. 16, no. 10, pp. 813818, 2008. T. D. Borkovec, R. Shadick, and M. Hopkins, The nature of normal an-pathological worry, in Chronic Anxi-ety, Generalized Anxiety Disorder, and Mixed Anxiety Depression, R. M. Rapee and D. H. Barlow, Eds. New York: Guil-ford Press, 1991, pp. 2951.

Table 4. Statistical score sheet from the 35 users.

Questions Subjects

35 35

Cronbachs Alpha 0,718490752

Split-Half (odd-even) Correlation 0,574619559

Spearman-Brown Prophecy 0,729851926

Mean for Test 13,77142857

Standard Deviation for Test 4,951684933

KR21 0,678728663

KR20 0,718490752

44 IEEE POTENTIALS

D. M. Fresco, D. S. Mennin, R. G. Heimberg, and C. L. Turk, Using the Penn State Worry Questionnaire to identify indi-viduals with generalized anxiety disorder: A receiver operating characteristic analy-sis, J. Behavior Ther. Exp. Psychiatry, vol. 34, no. 34, pp. 283291, 2003. M. Hamilton, The assessment of anxiety states by rating, Br. J. Med. Psy-chol., vol. 32, no. 1, pp. 5055, 1959. A. Osman, J. Hoffman, F. X. Bar-rios, B. A. Kopper, J. L. Breitenstein, and S. K. Hahn, Factor structure, reli-ability, and validity of the Beck Anxiety Inventory in adolescent psychiatric inpa-tients, J. Clin. Psychol., vol. 58, no. 4, pp. 443456, 2002. R. I. Kabacoff, D. L. Segal, M. Hersen, and V. B. Van Hasselt,

Psychometric properties and diag-nostic utility of the Beck Anxiety In-ventory and the State-Trait Anxiety Inventory with older adult psychiatric outpatients, J. Anxiety Disord., vol. 11, no. 1, pp. 3347, 1997. C. D. Spielberger and S. J. Sydeman, State-trait anxiety inventory and state-trait anger expression inven-tory, in The Use of Psychological Test-ing for Treatment Planning and Out-come Assessment, M. E. Maruish, Ed. Hillsdale, NJ: L. Eribaum Associates, 1994, pp. 292321. D. Siegdel. (2011, June 21). Reli-ability calculator [Online]. Available: http://www.gifted.uconn.edu/siegle/research/Instrument%20Reliability%20and%20Validity/Reliability.htm

S. Aydin, Internet anxiety among foreign language learners, TechTrends, vol. 55, no. 2, pp. 4654, Mar./Apr. 2011.

About the authorSantosh Kalwar ([email protected])

is a doctoral student at the Lappeenranta University of Technology, Lappeenranta, Finland. His research area is in human-computer interaction. He is a Member of IEEE, ACM, IADIS, and the Internet Society Finland Chapter and also acts vol-untarily as a reviewer for several interna-tional journal and conferences. He holds an M.S. in information technology from Lappeenranta University of Technology and a B.E. in computer science and engi-neering from Visvesvaraya Technological University, Bangalore, India.

1 23

4

5

6

7

8

9

10

11

TrueFalse

12

13

14

15

1617

18192021

22

23

24

25

26

27

User_ID

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

Average

True

14

19

25

20

22

20

19

10

10

13

11

19

13

22

7

16

4

9

9

15

11

11

7

3

19

14

12

14

10

5

5

21

13

19

21

False

21

16

10

15

13

15

16

25

25

22

24

16

22

13

28

19

31

26

26

20

24

24

28

32

16

21

23

21

25

30

30

14

22

16

14

21.22857143

Score

Medium

High

Very High

High

Very High

High

High

Low

Low

Medium

Medium

High

Medium

Very High

Low

High

Very Low

Low

Low

Medium

Medium

Medium

Low

Very Low

High

Medium

Medium

Medium

Low

Very Low

Very Low

Very High

Medium

High

Very High

Medium

28

29

30

31

32

3334

35 35

30

25

20

15

10

5

0

13.77142857

Metric Used

A Score of 0

A self-assessment tool.pdf

Documents

Transcript of A self-assessment tool.pdf