The effects of individual differences and visual instructional aids on disorientation, learning performance and attitudes in a Hypermedia Learning System

Abstract

Research suggests that certain visual instructional aids can reduce levels of disorientation and increase

learning performance in, and positive attitudes towards, HLS for learners with specific individual

differences. However, existing studies have looked at only one or two individual differences at a time,

and/or considered only a small number of visual instructional aids. No study has considered the impact

of the three most commonly studied individual differences – cognitive style, domain knowledge and

computer experience – on learning performance, disorientation and attitudes in a HLS incorporating a

full range of visual instructional aids. The study reported here addresses this shortcoming, examining

the effects of, and between, these three individual differences in relation to learning performance,

disorientation and attitudes in two HLS versions: one that incorporated a full set of visual instructional

aids and one that did not. Significant effects were found between the three individual differences with

respect to disorientation, learning performance and attitudes in the HLS that provided no instructional

aids, whereas no such effects were found for the other HLS version. Analysis of the results led to a set

of HLS design guidelines, presented in the paper, and the development of an agenda for future

research. Limitations of the study and their implications for the generalizability of the findings are also

presented.

Keywords: Hypermedia Learning; Individual Differences; Visual Support; Disorientation, Learning

Performance; Attitudes

1. Introduction

Hypermedia Learning Systems (HLS) are being increasingly used in Higher Education [55] to

support students’ access to learning material in a flexible way. A defining feature of HLS is that the

learning material is presented using a non-linear structure [29], allowing students to determine their

own path through the material [4]. Allowing learners to decide the sequence in which they encounter

1

the learning content has been suggested to offer improvements in learning and cognitive flexibility

[53]. However, some users have difficulty in navigating through HLS to find the information that they

need and, as a result, experience disorientation [6]. A consequence of disorientation is that learners can

miss at least some of the relevant content in the system, which may hinder their learning performance

[36]. Performing less well may lead to these learners having a negative attitude towards HLS and also

to have less interest in learning using these types of learning system [22].

Research findings suggest that not all learners are comfortable using, satisfied with or learn

effectively from HLS, implying that the value of HLS varies depending on the individual and may be

influenced by various characteristics of the learner [26]. This means that the individual differences that

these characteristics represent become important when designing and developing HLS. A range of

studies have looked at the relationship between individual differences and student learning in HLS,

with the results tending to show that individual differences, particularly cognitive style (most often

examined in terms of Field Dependence/Field Independence), domain knowledge and computer

experience, influence learners’ levels of disorientation, learning performance and attitudes in HLS.

Though widely studied individually or in pairs, the influence of, and relationships between, all of these

individual differences in relation to disorientation, learning performance and attitude have not been

looked in a single study.

It is argued that supporting individual differences in HLS to reduce learners’ disorientation would

be helpful, as it can improve learning performance and increase learning satisfaction [51]. A common

way to reduce disorientation is to provide instructional guidance, in the form of visual navigational aids

(e.g., maps) and a set of visual cues (e.g., breadcrumbs, highlighting of context, pagination and so on),

within the HLS. The relationship between individual differences and such instructional guidance has

been explored in many studies, yet there are no studies that have integrated a map and these visual cues

in a single HLS.

This paper reports a study that seeks to address these gaps in the research literature through by

examining the effects of three individual differences (cognitive style, domain knowledge and computer

experience) on disorientation, learning performance and attitudes in HLS with and without instructional

guidance in the form of visual navigational aids and a set of visual cues. The paper begins by

reviewing relevant literature related to individual difference and HLS use, in order to identify and to

frame three research questions that will address these gaps. The paper then sets out the methodological

2

approach to exploring the research questions, introducing the empirical study, its research design,

sample, the range of materials and instruments used, and the detailed experimental procedure. The

paper then presents the results of the study and analyses the resulting data in order to provide answers

to the three research questions. The paper ends by framing a set of design guidelines from the analysis

of the data and by setting out directions for future work.

2. Background

As the World Wide Web (WWW) becomes ever more widely used as an educational platform [55],

HLS are gaining increased attention from researchers [5, 31]. One of the major reasons for moving

from traditional classroom-based learning to offering instruction through the use of HLS is that the

latter can present learning material in a non-linear structure [10]. Such non-linearity affords learners

greater flexibility in navigating the learning content and allows them to choose their own paths through

it to meet their learning goals [4]. Additionally, non-linearity allows learners to access and sequence

information in accordance with their individual needs [25]. Furthermore, allowing learners to have

control over their learning may also make them motivated to learn, improving their learning

performance and cognitive flexibility [53].

However, the flexibility offered by HLS may cause problems for some users. It has been argued

that not all users can ‘develop’ their own learning paths effectively to achieve their learning goals when

using HLS [27], and many studies have shown that users may experience disorientation – reflected as

questions of ‘where am I?’, ‘where have I been?’ and ‘where can I go next?’ – when seeking to

navigate through HLS [6].

Some studies have suggested that learners who encounter higher levels of disorientation may, in

turn, perform less well in learning tasks [36]. Separately and in combination, disorientation and poorer

performance in learning tasks may have an impact on learners’ attitudes towards HLS. Dringus [22],

for example, argues that when learners experience disorientation in HLS, and are hindered in their

learning performance, there have an increased chance of showing negative attitudes towards the non-

linear learning environment. As a result, such learners may feel less motivation to learn using HLS.

The range of reactions to being given freedom in terms of navigation may be explained by the

different characteristics that learners possess, meaning that the individual differences that these

characteristics represent are critical for effective HLS’ design. It has been suggested that individual

3

differences such as cognitive style [15], domain knowledge [8], and computer experience [55] are the

most commonly studied in research related to student learning and HLS use. Each of these individual

differences will be briefly introduced in the context of research into HLS.

The ways in which an individual thinks, memorizes, perceives, organizes, processes and presents

information is often referred to as cognitive style [47]. Among the different dimensions of cognitive

style that have been studied to date, Field Dependence (FD) and Field Independence (FI) are often

argued to be of interest, especially with respect to research that is related to HLS [12]. FI learners tend

to rely on internal references, adopt an active approach to learning and process information using an

analytical approach. Conversely, FD learners tend to rely on external references, adopt a passive

approach to learning and accept information in exactly the way it is presented to them [17, 58].

Witkins et al. [58] Group Embedded Figure Test (GEFT) and Riding’s [46] Cognitive Style Analysis

(CSA) are two common instruments used to identify a learner’s cognitive style.

The research related to cognitive style and HLS suggests that FI learners: prefer being given high

levels of freedom of navigation; experience less disorientation in HLS; perform well in learning tasks;

and have a positive attitude towards HLS compared to FD learners [16, 32, 54]. One explanation of FI

learners’ performance in HLS is that they are able to follow a restructuring approach more easily

because they are internally directed and tend to adopt an active approach to learning; and they can

extract relevant items from within the complex context offered within complex content, such as that

offered in HLS, because they are more analytical. In contrast, FD learners prefer to follow the

structure of the learning material because they are externally-directed and tend to adopt a passive

approach to learning; they also have difficulties extracting the relevant items within the complex

context because they are less analytical [17, 59].

Research into domain knowledge, mostly suggests that domain knowledge influences the degree of

disorientation and learning performance in HLS, with low domain knowledge (novice) learners

experiencing higher levels of disorientation and performing less well in HLS than high domain

knowledge (expert) learners [7, 23, 35, 57]. A general conclusion drawn in studies in this area to

explain this finding is that, novices are unfamiliar with the subject content, which makes it difficult for

them to impose a meaningful conceptual structure on the content compared to experts [13].

In terms of the research related to computer experience, studies suggest that compared with those

with lower levels of computer experience, learners with high levels of computer experience: prefer the

4

non-linear pathways that are normally offered in HLS; navigate effectively; take fewer steps to reach

the information they need in the tutorial; browse more pages and are able to reach more detailed levels

of the subject content; enhanced their time efficacy; and overall, perform well in learning tasks in the

HLS [30, 37, 52, 55]. It is argued that this is because a well-developed understanding of different

computer applications enables these learners to better navigate through the HLS to achieve their

learning goals. In contrast, the lack of skills related to the use of computers and their applications

makes it difficult for those with low levels of computer experience to successfully navigate through

HLS to find the information that they need [40, 52].

It is suggested that reducing the levels of disorientation experienced by learners with different

characteristics may improve their learning performance and, in turn, may show more interest in

learning using HLS [12, 13]. To reduce learners’ disorientation, the use of instructional aids, such as

maps which support visual navigational, has been suggested [5, 8, 13, 14], and a number of studies

have examined the effects of both maps and individual differences (such as those considered in this

paper) on learners’ levels of disorientation, learning performance and attitudes in HLS [3, 8, 33, 34,

45]. The results from these studies suggest that when maps are provided in HLS, learners’ levels of

disorientation are decreased and learning performance and positive attitude are increased. However,

these studies have not considered the effects of all of the three individual differences – cognitive style,

domain knowledge and computer experience – in combination with maps in HLS. Rather, they have

tended to examine the effects of only one or two of the individual differences presented in this study.

Studies which examine the effect of maps in relation to a wider set of individual differences may help

to determine whether maps reduce disorientation and increase learning performance and learning

satisfaction for all users with different combinations of the individual differences, or whether maps

create problems for specific groups.

In addition to maps, previous studies suggest that visual cues – such as breadcrumbs [1, 44],

graphic visualizations [39, 42], history-based mechanisms [24], context highlighting [20], page labels

[18], different colors link [42] and link annotations [9] – can also reduce disorientation and enhance

efficiency of learning performance and satisfaction in HLS [19, 49].

Despite the fact that these studies have explored the same set of individual differences considered in

this paper and have considered a range of these visual cues, an important gap remains. There are no

studies where cognitive style, domain knowledge and computer experience have been considered

5

together in relation to all of the visual cues identified earlier within a single HLS. This implies that the

combined effects of these three individual differences and all of these visual cues on learners’

disorientation, learning performance and attitudes in HLS have not been fully examined. Though

methodologically complex in terms of study design and subsequent analysis, addressing this gap may

provide results which can help designers and developers to gain a better understanding of the

relationships between HLS instructional aids (in the form of visual cues), individual differences and

learner disorientation, performance and attitude. This leads to the framing of the three research

questions addressed in the study reported in the remainder of this paper:

Research Question 1:

What are the effects of and between cognitive style (FD/FI), domain knowledge, and

computer experience on learners’ levels of disorientation when using a HLS that includes a

map and the defined set of visual cues and when using a HLS without any instructional aids?

Research Question 2:

What are the effects of and between cognitive style (FD/FI), domain knowledge, and

computer experience on learners’ learning performance when using a HLS that includes a map

and the defined set of visual cues and when using a HLS without any instructional aids?

Research Question 3:

What are the effects or and between cognitive style (FD/FI), domain knowledge, and

computer experience on learners’ attitudes when using a HLS that includes a map and the

defined set of visual cues and when using a HLS without any instructional aids?

3. Methodology

This section describes the research methodology that was used to address the three research

questions. An explanation of the research design is given, followed by a description of the sample, the

materials and instruments used in the study, the experimental procedure and the approach to data

analysis.

6

3.1 Research Design

To answer the three research questions that were proposed, this study adopted an experimental

research approach [48], in which a set of independent variables and dependent variables were identified

and used. The independent variables were the HLS and the users’ individual differences (cognitive

style, domain knowledge and computer experience). With respect to the HLS, two versions, one

without instructional aids and one with visual instructional aids (in the form of a map and a set of

visual cues) were required. The dependent variables in this experimental study were learning

performance, disorientation and attitudes towards the HLS. A between-subjects design was used in this

study, with one half of the sample using the HLS that provided no instructional aids and the other half

of the sample using the HLS containing the set of visual instructional aids defined above. It is

acknowledged that this research design is complex and has a wider range of variables than have been

used in other studies in the field. This brings significant issues in terms of analysis of the data and the

implications that can be drawn from them. These issues are discussed, and reflected on, in later

sections of the paper.

This study also aimed to gather detailed user information on individual differences, learners’

attitudes, feelings and preferences, their experience of disorientation, and their interaction behavior

with respect to the HLS that they used. To support the experimental study, a descriptive study was also

employed (using a qualitative approach), in which learners were observed, surveyed and interviewed

[41, 48]. However, for reasons of space, this paper will consider only the experimental study and its

results.

3.2 Sample

The sample was drawn from university students across London, UK. University students were

considered to be suitable participants because, as mentioned in sections 1 and 2, this study was

concerned with research related to students and HLS in Higher Education. Recruitment of the sample

was also supported by the existence of channels through which we were able to contact a large number

of students at universities in the London region. Though the choice of university students as

participants can be seen as part of a purposive sampling strategy, the restriction of the recruitment of

participants to those from the London region introduce a ‘convenience’ characteristic to the sample

7

which is important to acknowledge, and which will be returned to in section 7 when the implications of

the findings with respect to the study’s limitations, are discussed.

192 participants took part in the study: split equally between the HLS that provided no instructional

aids and the HLS that contained the visual instructional aids. The participants were at undergraduate,

postgraduate taught and postgraduate research levels and were registered on a range of courses at

universities in the London region. They were informed about the experimental study through a

discussion forum, notice boards in the Students’ Unions of universities across London and by word of

mouth. Participants were also told that taking part in this study would help them to learn about, and to

build their own web site, either for personal or business use, and that small incentives in the form of

soft drinks, sweets and snacks would also be offered at the end of the practical study.

Participants were tested for cognitive style (in stage 2, see section 3.4) and their levels of domain

knowledge and computer experience were assessed (in stage 3, see section 3.4). This allowed the

identification of participants with appropriate cognitive style types and experience profiles. Having

secured the overall sample, participants were then assigned to one of the two HLS conditions (with and

without instructional aids). The assignment of each individual to the HLS versions was random, but it

was ensured that the sample was balanced in terms of the different experience profiles represented

under each HLS treatment. There was an equal chance of an individual being assigned to each

experimental condition (represented by the two versions of HLS), thus increasing the internal validity

of this study.

Table 1 presents a summary of the characteristics of the participants in the two versions of the HLS.

In terms of gender, in the HLS that provided no instructional aids (version 1), 42 were male and 54

were female; in the HLS that provided visual instructional aids (version 2), 50 were male and 46 were

female. With regards to age, the majority of the participants were between 18 and 23: 60 in version 1;

and 71 in version 2. There were 19 participants aged between 24 and 29 (19 used version 1 and 14

used version 2); 20 were aged between 30 and 35 (12 used version 1; and eight used version 2); and

eight were aged 36 or over (five used version 1 and three used version 2).

<<Insert Table 1 here>>

8

Data on level of education were also gathered. For version 1, 52 participants were undergraduates,

27 were taught postgraduates (Masters) and 17 were research postgraduates (PhD). For version 2, 60

participants were undergraduates, 24 were taught postgraduates and 12 were research postgraduates.

The participants were studying different subjects: business (18 in version 1; 11 in version 2);

computing and mathematics (45 in version 1; 43 in version 2); engineering and design (7 in version 1;

14 in version 2); health sciences (10 in version 1; 10 in version 2); law (five in version 1;10 in version

2); and social sciences (11 in version 1; eight in version 2). A number of the participants in the study

were registered as having dyslexia (10 who used version 1 and 14 who used version 2).

Participants’ levels of domain knowledge, computer experience and cognitive style were also

determined. In terms of domain knowledge, 96 participants were novice (defined as having low levels

of domain knowledge) and 96 were expert (high levels of domain knowledge). With regards to

computer experience, 96 had low levels of experience and 96 had high levels. Finally, in terms of

cognitive style, 96 participants were FD and 96 were FI. The different experience profiles were

assigned to the two HLS conditions to give a balanced sample in terms of these three individual

difference types.

3.3 Materials and Data Collection Instruments

To conduct the experimental study a range of materials and data collection instruments were used.

The remainder of this section describes each of the materials and instruments.

3.3.1 The subject content of the HLS

The two versions of the HLS were developed by the researchers for the sole purposes of this study

and contained the same subject-based information as the learning content of the tutorial. The learning

content covered core aspects of the Extensible Hypertext Mark-up Language (XHTML) and was

structured into seven sections: (i) introducing XHTML; (ii) how to create, save and view XHTML

documents; (iii) basic XHTML formatting: (iv) creating lists; (v) how to use images; (vi) how to insert

links within a page; and (vii) how to create tables.

9

3.3.2 HLS with visual instructional aids

A HLS incorporating visual instructional aids was designed and developed. The visual

instructional aids incorporated into the HLS took the form of: a structural map; different colors link;

breadcrumbs; pagination; page labels; annotation of links; graphic visualization; highlighting context;

and history based mechanism. The ‘map’ (see Figure 1) helped the user to navigate through the tutorial

to find the information that they needed in order to achieve their learning goals in three ways: it

presented a global overview of the Extensible Hypertext Mark-up language (XHTML) learning

content; it offered a representation of the relationships between the different information topics

presented in the HLS; and it offered a document structure of the information presented in the HLS.

The ‘different colors’ link (as shown in Figure 1) informed users about the pages that they had

visited. This approach prevented them from repeatedly opening the same page they had visited earlier,

hence, a save of time.

<<Insert Figure 1 here>>

The ‘link annotation’ cue (as shown in Figure 2) provided users with information about each link in

relation to the XHTML tutorial. The ‘breadcrumb’ visual orientation cue (see Figure 2) helped users to

identify their current location in the HLS and the path that had led them there. The ‘pagination’ visual

cue (as shown in Figure 2) allowed users to know how many pages there were in the section, the page

that they were currently on; the number of pages that they had viewed in the section; and the number of

pages left to view. The ‘page labels’ visual cue, in the form of headings and sub-headings (see Figure

2), helped users to locate particular information in the tutorial.

<<Insert Figure 2 here>>

The ‘graphic visualization’ cue (as shown in Figure 3) showed a global overview of the information

that was presented in the HLS, and offered a conceptual structure of the information presented in the

HLS. The ‘highlighting context’ visual cue that was provided in the graphic visualization (as shown in

Figure 3) showed disabled links as differently colored nodes, with the accompanying text provided in

different font sizes, styles and colors. Using the ‘highlighting context’ approach allowed users to be

10

alerted to: (i) their current position in the HLS; and (ii) their position in relation to the overall system

structure.

<<Insert Figure 3 here>>

The ‘history-based mechanism’ (as shown in Figure 4) allowed users to view and access the last

two visited pages. With the use of this visual cue, the chance of unintentionally opening previously

visited nodes was reduced.

<<Insert Figure 4 here>>

3.3.3 HLS without instructional aids

As part of this experimental study, a HLS without instructional aids was also designed and

developed, offering a non-linear structure that could be flexibly navigated, allowing users to set their

own learning paths in order to meet their learning goals (see Figure 5). In this version of the system

only one navigation support mechanism was provided – an index (as shown in Figure 6) which showed

an alphabetic list of the system’s links in relation to the XHTML content. The users were able to

access the links in any order. The same learning content – a tutorial on Extensible Hypertext Mark-up

Language (XHTML) – was used for both versions of the HLS. The tutorial comprised seven lessons,

each with a minimum of three and a maximum of seven sections.

<<Insert Figure 5 here>>

<<Insert Figure 6 here>>

3.3.4 Cognitive Style Analysis (CSA)

To identify learners’ cognitive styles, Riding’s [42] Cognitive Styles Analysis (CSA) was

employed. The CSA includes two sub-tests. In the first sub-test, learners were presented with items

containing pairs of complex geometrical figures from which they had judge whether the figures were

similar or different. In the second sub-test, learners had to indicate whether a simple figure was

contained within a complex one. The first sub-test required field-dependent capacity whereas the

second sub-test required field independence capacity. Following Riding’s [42] classification, FM

(Field Mixed), FD and FI users were identified with the following scores: those who scored less than

11

1.02 were classed as FD and those who scored 1.36 and above were classed as FI. Those falling

between 1.03 and 1.35 were classed as Field Mixed (FM); only FI and FD users were included in the

analysis.

3.3.5 Pre-test and Post-test Questions, and Practical Task Sheet

To determine the effects of the HLS’ visual instructional aids on participants’ learning performance

(in the form of declarative and procedural knowledge), a set of instruments were used. In line with the

approach taken in other studies in the field (for example, [11]; [23]; [36]; [37]; [56]), pre-test and post-

test instruments were respectively applied to measure the participants’ declarative knowledge of the

learning material (XHTML) prior to and after the use of the prescribed HLS. Given the specific

learning material addressed in the HLS, the pre- and post-test instruments had to be developed

specifically for this study. Each test contained 20 multiple-choice questions, with each question having

four possible answers and a “I do not know” option. Participants were asked to circle the answer that

they thought was correct. The answers to all of the questions were available in the tutorial. Ensuring

that the pre-test and post-test questions were comparable was achieved by either rewriting the question

and providing the possible answers in a different order or, where appropriate, by substituting different

numbers or variables into the questions.

To assess procedural knowledge, a practical task sheet was designed and developed, again

specifically for this study to address the XHTML learning content. The task sheet comprised five

questions related to the construction of web pages. The key areas addressed by the questions related to

the insertion of images and links, building tables, formatting text, creating lists, and creating, saving

and viewing a document in the browser.

3.3.6 Questionnaires

Finally, three closed-question questionnaires were used in this study. The first collected

information on: gender; age; course area and level; and disability. Participants’ levels of computer

experience, domain knowledge in relation the tutorial’s subject content (XHTML) and familiarity with

other programming languages (such as Hypertext Markup Language (HTML), Visual Basic, C++ and

Java) was also gathered, using a three-point Likert scale (1 representing “novice”, and 3 representing

“expert”).

12

The second questionnaire presented closed choice questions to measure participants’ levels of the

four types of disorientation: (i) I know my current location in the HLS; (ii) being on the current page, I

know where I was previously in the HLS; (iii) being on the current page, I know where to go next in

the HLS; and (iv) I know how to reach my desired location in the HLS.

The third questionnaire presented closed choice questions to rate attitude towards different aspects

of the HLS: structure; navigation; overall level of disorientation; dependency on and distraction by the

visual instructional aids; and overall satisfaction. A Likert scale was used for all questions in the

second and third questionnaire, with the scale items comprising 1 (strongly disagree) to 7 (strongly

agree). Participants were required to circle the response that most closely reflected their answer to each

question.

This use of closed choice questionnaires is evident in a number of other studies in the field (for

example, [2]; [21], in relation to disorientation; and [12]; [11]; [36]; [37]; [38]; [56], in relation to

attitudes and perception). Though there are several questionnaires which gather data on disorientation,

and attitudes and perception, this study had to develop its own instantiations in order to gather data

about participants’ use and perceptions of the specific HLS used in this study.

3.4 Procedure

The experimental study was conducted over a number of sessions in a computer usability laboratory

at Brunel University. Each session contained a small group of participants, each working individually

on a PC. The experimental study was divided into nine stages:

1. Participants were given a brief explanation of the study;

2. Participants took the CSA test to determine their level of field dependence (classified

into FD, FM and FI) according to their CSA score. Participants identified as being

FM did not undertake the subsequent stages;

3. Participants completed the questionnaire to determine their level of computer

expertise, experience of XHTML and other programming languages and to gather

demographic information;

4. Participants were given a maximum of 15 minutes to complete the pre-test (marked

out of 20) to determine their prior knowledge of XHTML;

13

5. Participants were then randomly assigned to use one of the two HLS (with visual

instructional aids; and without instructional aids) and used the assigned HLS to

complete the online instructional content of the XHTML tutorial;

6. Participants undertook the practical task to measure their learning performance (in the

form of procedural knowledge). They were allowed to use the HLS to find the

answers and the task was marked out of 50;

7. Participants were given a maximum of 15 minutes to complete the post-test (again,

marked out of 20);

8. The disorientation questionnaire was administered to measure participants’ levels of

the four types of disorientation;

9. The attitude questionnaire was administered to record participants’ attitudes towards

the HLS.

3.5 Data Analysis

As already noted, each participant’s cognitive style was determined using the CSA instrument,

categorizing participants as Field Dependent (FD) or Field Independent (FI) (see stage 2, above). The

‘domain knowledge’ variable was categorized as taking one of two values: novice – signifying low

domain knowledge; or expert – signifying high domain knowledge. Similarly, the ‘computer

experience’ variable was categorized as taking one of two values: novice – low computer experience;

or expert – high computer experience (see stage 3, above). Thus each participant had a profile of

individual differences – with one of two cognitive style types, one of two levels of domain knowledge,

and one of two levels of computer experience – from an overall set of eight possible profiles.

These profiles were used within the grouping strategy for the three-way ANOVA [28, 43] which

was used to determine whether there was a significant effect between the three individual differences

(cognitive style, domain knowledge and computer experience) on learners’ disorientation, learning

performance and attitudes when using the two versions of the HLS: one without instructional aids and

with visual instructional aids.

In addition to the ANOVA test, the Newman Keuls post-hoc test was used to identify whether or

not there were significant differences between the sub-groups or not. The data gathered from the

disorientation questionnaires (in stage 8), attitude questionnaires (in stage 9) and achievement

14

tests/practical tasks (stages 4, 6 and 7) were analyzed using the Statistical Package for Social Science

[50]. The level of significance was set at p < 0.05.

4. Findings

This section presents and considers the findings that were gathered from the experimental study, the

aim of which was to address the three research questions. To this end, data gathered from the pre- and

post-tests tests, practical tasks and closed questionnaires were analyzed.

4.1 Description of participants

192 university students participated in the study: 96 of them used the HLS that provided no

instructional aids; the remainder used the HLS that incorporated visual instructional aids. The

distribution of the participants according to their cognitive style (FD or FI), levels of domain

knowledge (DK) and computer experience (CE) to each HLS was presented in Table 1.

Since this study used a three-way ANOVA, providing a focus on three individual differences –

cognitive style (FD or FI), domain knowledge (high or low DK) and computer experience (high or low

CE) (which wasas discussed in section 3.5), a minimum of 12 participants were needed in each group

to ensure a sample that had the potential for revealing significant results (see Table 2), giving a total

sample comprising of 192 participants.

Table 2: Distribution of participants needed according to their cognitive style, DK and CE

‍

HLS that provided no instructional aids HLS that incorporated visual instructional

aids

FI FD Total FI FD Total

Low DK and low CE 12 12 24 Low DK and low CE 12 12 24

Low DK and high CE 12 12 24 Low DK and high CE 12 12 24

High DK and low CE 12 12 24 High DK and low CE 12 12 24

High DK and high CE 12 12 24 High DK and high CE 12 12 24

Total 48 48 96 Total 48 48 96

Total number of participants = 96 + 96 = 192

15

4.2 Results related to disorientation

With regards to the HLS that provided no instructional aids, the three-way ANOVA revealed a

significant effect between cognitive style, domain knowledge and computer experience in relation to

participants’ levels of the four types of disorientation in the HLS. The analysis of the Newman Keuls

post-hoc tests also revealed some significant differences between the groups. The summary results of

the ANOVA and analysis of the post-hoc tests analysis are presented in Table 2. For the group of

participants that had low DK and low CE, low DK and high CE, or high DK and low CE, cognitive

style had a significant impact. FD users with one of these experience profiles experienced higher levels

of all four types of disorientation than did FI users with the same experience profile. This is argued to

be a major finding of the study, though there are important issues relating to interpretation of this

finding which will be discussed in section 7.

<<Insert Table 2 here>>

However, for the group of participants that had high DK and high CE, cognitive style did not have a

significant effect: neither FD nor FI users in this group experienced higher levels of any of the four

types of disorientation in the HLS.

In the version of the HLS that incorporated visual instructional aids, the three-way ANOVA

identified no significant effect in relation to cognitive style, domain knowledge and computer

experience on the levels of the four types of disorientation. With regards to all four experience profiles

((i) low DK and low CE; (ii) low DK and high CE; (iii) high DK and low CE; and (iv) high DK and

high CE), cognitive style did not seem to have an impact: FD and FI users with the same experience

profile tended to agree equally with the statement that they did not experience higher levels of

disorientation in the HLS in relation to the four types.

Having considered disorientation, the next section reports the learning performance results in

relation to both versions of the HLS.

4.3 Results related to learning performance

16