Sheffield Conference 4 March 2006 - Reinder Vrielink International MSc E-learning, Multimedia and...

Sheffield Conference 4 March 2006 - Reinder Vrielink

International MSc E-learning, Multimedia and Consultancy Sheffield Conference 2006

Predicting the use of Blackboard with the Technology Acceptance Model

A cross-validation research among Dutch pupils aged 12-18

Dissertation

Author : Reinder Vrielink, headmaster department VMBO Stedelijk Daltoncollege, Zutphen, the Netherlands

Supervisor : Dr. Eric Siebenheller

Programme Leader : Professor Brian Hudson


Introduction of myself Reinder (Rein) Vrielink

22 August 1951 I’m headmaster of the

department VMBO Stedelijk Daltoncollege

Zutphen I live in Deventer with my wife

Christa, two cats, We have three children

Amke (29) Almere Erik (27) New York Marijke (24) Amsterdam


VMBO Basic secondary education ( 12-14) refers to the first two years. Pre-vocational secondary education (15-17) (VMBO-basis en

kaderberoepsgerichte leerweg) In the upper forms (third and fourth) pupils can choose from four

sectors: - Business and economics- Engineering and technology- Care and welfare- Agriculture.

General Secondary Education (VMBO-theoretische leerweg; MAVO): this course prepares pupils for Secondary Vocational Education (Colleges - MBO).

Apart from this regular education we offer special education for pupils with specific learning needs or abilities.


Why the Zernike College?

Zernike College was selected because of my personal acquaintance with this College and a mutual participation at the start of ICT-north in 1999 (comprising 13 schools, including the Zernike College) and the ICT-study trip to Finland in 2001. Zernike College introduced Blackboard in 1998. In the same year and also in 2001, Zernike College received the Dutch award ‘computer school of the year’. Zernike College introduced Blackboard in 2002 collectively with 35 other schools.


How do you measure the Efficiency of a by Brainbox supported lesson.



Figure 2. The modified Technology Acceptance Model by Yi (Yi, et. al., 2003).



Technology Acceptance Model (TAM) Davis (1993)

TAM has established a powerful model for explaining and predicting usage intentions and acceptance behaviour Perceived usefulness - improve my performance Perceived ease of use - find it easy to use Behavioural intention - intent to check announcements Enjoyment - is pleasant

Learning goal orientation - looking for opportunities to develop

Application specific self-efficacy - ability to send e-mails



Research questions and hypotheses.

Research questions. 1. Is the Technology Acceptance Model (TAM) a valid model for

pupils aged 12-18 for predicting the use of Blackboard? 2. What factors determine the use of Blackboard for pupils aged

12-18? 3. Is there a difference between pupils in lower classes (aged 12-

15) and higher classes (aged 16-18)? 4. Is there a difference between females and males? 5. Is there a difference, between American students in higher

education and Dutch pupils at a secondary school, in the way they respond to Blackboard?



The following hypotheses were tested:

Hypothesis 1: The same internal and external factors as the research Yi (Yi, et. al., 2002), determine the behavioural intention to use Blackboard.

Hypothesis 1a: Perceived usefulness and perceived ease of use are the key factors in predicting the acceptance of Blackboard among pupils at secondary schools in the Netherlands.

Hypothesis 1b: Enjoyment is a stronger predictor of usefulness than ease of use is.

Hypothesis 2: These findings are independent of age and gender. Hypothesis 3: The cross-validation of the research among American

students in higher education aged 18-20 and Dutch pupils aged 12 - 18 shows the same results.



Research Methodology

Procedure a questionnaire in Dutch for pupils according to Yi’s research’ (Yi, et

al., 2003). A tryout among peers in 2004 showed that TAM is a good predictor for the use of Blackboard. The questionnaires were distributed among pupils during the fifth module: Project Studies, September – November 2004.

Of 88 pupils the actual use was measured by counting the number of times (frequency) a pupil logged into Blackboard courses in the period of December 13th 2004 until April 10th 2005.

One hundred and eighty pupils filled in the questionnaires



Results

Background of the pupils (age, gender, class) The actual use of Blackboard by the pupils The score on the questions Reliability of the questionnaire The correlation between the questions Measurement of ß-coefficients to test the Technology Acceptance

Model and comparison with the results of American students Comparison male and female Comparison pupils aged 12-15 with pupils aged 16-19 Results of the sample taken among pupils Zernike College compared

with the sample among American students



Results

Background of the pupils (age, gender, class) The respondents’ age ranges from 12 to 19. Most of the pupils are

14 years old (39%). Average age: 16,5 years; s.d.= 0.9 years. 47% pupils are in class three Havo 39% of the respondents is female.

– Number % Female % Male

Lower classes 112 38 62 Higher classes 48 42 58



Results

The actual use of Blackboard by the pupils

– N Min Max Mean s.d.

Frequency 88 0 292 22 41.2



Results

The score on the questions Four out of twenty-three questions scored < 6.00. The question that

scored best was number 12 (6.66): “I find the Blackboard system would be useful in this course.” The question that scored lowest was number 15 (2.70): “I intend to visit other websites using the Blackboard system frequently.”

The results shows that the Dutch pupils show the highest score on ease of use (mean=6.22; s.d. =2.01) and the American students also show the highest score on ease of use too (mean=8.21; s.d. =1.71). The pupils scored significantly lower than the American students (t>6.31; p<0.10) except for enjoyment.



Results

Reliability results (Cronbach’s alpha) and The correlation between the questions As recommended, the internal consistency reliabilities are higher

than 0.7 except from the Application-specific self-efficacy. For research purposes, an ICR of 0.67 is acceptable

Pearson’s product-moment correlation coefficients. All variables are strongly interconnected.

The reliability results and the correlation results provide a good condition to test the Technical Acceptance Model with the regression analyses.



Regression analyses. Measurement of ß-coefficients to test the Technology Acceptance Model The structural model and hypotheses are assessed by examining

the significance of the ß-coefficients and the variance accounted for by the antecedent constructs. The most common approach is to standardize the independent variables through a quick operation which involves dividing the difference between each independent variable value and its mean by the standard deviation of that independent variable. This results in the standardized beta coefficient (Beta = standardized regression coefficient= ß-coefficient).


Usefulness

Behavioural Intention

Ease of UseEnjoyment

LearningGoal

Orientation

Application Specific Self-

Efficacy

Use

0.58/0.46

0.49/0.50

0.34/ns

0.34/0.27

0.18/0,19

0.18/0.490.22/0.24

0.35/ns



Results

Comparison male and female no differences were found between the responses of males and

females

Comparison pupils aged 12-15 with pupils aged 16-19 Pupils aged 16-18 scored significantly higher on application-specific self-

efficacy (p<0.05), while pupils aged 12-15 scored significantly higher on enjoyment (p<0.05) and on ease of use (p<0.01).

Results of the sample taken among pupils Zernike College compared with the sample among American students The pupils scored significantly lower than the American students (t>6.31;

p<0.10) except for enjoyment.


Predicting the use of Blackboard with the Technology Acceptance ModelDiscussion

When we look more closely at the underlying reasons why Dutch pupils do not respond so well to Blackboard, three sub-questions can be distinguished: A. Do Dutch pupils not respond very well to Blackboard or; B. Do Dutch pupils not respond very well to the way Zernike

handles Blackboard or; C. Do Dutch pupils not respond very well to e-learning in general?


Predicting the use of Blackboard with the Technology Acceptance Model Discussion

Some pupils supplied interesting comments:

Maarten (15, A5); ‘I know how to handle computers well and meanwhile I have become familiar with most of the functions of Bb. There is hardly any profit to gain for me, in the field of ‘skills’. The only problem is that Blackboard wants to do too much at the same time and that in most cases separate one-goal programs would function better. Like Gmail for e-mail and IM for contact. I don’t know how this can be put right.’

Joske (18, A6); Another comment given to improve Bb: Increase the organization of Bb by using clear titles and omitting unnecessary functions.’

Karin (18, A6); ‘About the question whether it is pleasant or not, I’d like to say that I find it pleasant because it might come in handy. However, not in the sense that you become cheerful or that you can compare it with something pleasant like dancing or laughing with friends. Besides, my internet speed is low, which is very frustrating’.



Advantages of Blackboard seen by pupils are: Teaching materials Practical information like the study results Digital drop assignment at any time Support from the teacher

Disadvantages of Blackboard seen by pupils are: Boring Slow Where are the study planners? English. Most courses at the Zernike have buttons in Dutch! RSI



Significant factor effects, which determine the behavioural intention to use Blackboard by Dutch pupils, aged 12-18.

Usefulness

Ease of use

EnjoymentLearning

GoalOrientation

ApplicationSpecificSelf-Efficacy

useBehaviouralIntention



usefulness is the key factor for behavioural intention to use Blackboard. Usefulness is influenced in two ways.

First, learning goal orientation influences usefulness through enjoyment. This is an intrinsic factor. If a pupil or a teacher is able to sufficiently clarify the goal, and render it obtainable, then there is enjoyment. It has a direct and positive effect on usefulness and on ease of use.

Second, a goal within one’s reach, affects application specific self-efficacy. If a pupil thinks he/she is competent enough to work with Blackboard, this supplies an experience of ease of use, which effects usefulness. Performance enhances when you are enjoying yourself. In this manner, you are able to develop competencies. Learning is an actual proficiency and to be practised now and not later! Maybe this also gives an explanation for the fact that older pupils score significantly better on application specific self-efficacy. They already are of the opinion that they are able to cope with the given task.



enjoyment learning

performance

Figure 5. The working triangle



The Seven Principles of good practice in undergraduate education. (Chickering, et. al., 1986).

1. Encourage contact between students 2. Develop reciprocity and cooperation among students 3. Encourage active learning 4. Give immediate feedback (Deinum 2002;, Vrielink 2004) 5. Emphasize time on task 6. Communicate high expectations 7. Respect different talents and ways of learning


Predicting the use of Blackboard with the Technology Acceptance ModelConclusions

‘Technology Acceptance Model’ appears to be a very reliable predictor regarding the use of Blackboard.

Usefulness is the best predictor regarding the use of Blackboard, while ease of use tends to only predict usefulness but not the actual use.

No distinctive differences have been demonstrated between either males or females.

Younger pupils have proved to score significantly better as regards ease of use and enjoyment than senior pupils, while this last group has proved to score significantly better as regards application specific self-efficacy.


Predicting the use of Blackboard with the Technology Acceptance ModelConclusions (continue)

The result of this study shows a significantly lower score by the Dutch pupils when compared with the sample taken from among American students. In the Dutch sample, 4 out of 23 questions have a score of > 6.00 (on a 11-point Likert-scale), whereas in the American sample all the 23 questions have a score of > 6.00.

There is a considerable degree of similarity as to the pattern and the results of both samples taken in relation to the effect of ease of use on usefullness. The predicting value therefore shows a strong similarity.


Predicting the use of Blackboard with the Technology Acceptance ModelConclusions (continue)

There is a need for a Blackboard teaching method for teachers. The Seven Principles of good practice in undergraduate education

(Chickering, et. al., 1986) can serve as a basis when working with an OFLE.

When a school starts to implement an OFLE there should be a strategic action plan, which includes:

a shared view on the use of an OFLE goals management participation exchange of good practice - celebrate successes! agreement on the arrangement the OFLE should be on the agenda of job evaluation


Predicting the use of Blackboard with the Technology Acceptance Model Conclusions (continue)

In order to render online courses more attractive for pupils, enjoyment should increase by: improving its usefulness

a consistent and simple structure attractive announcements figures animations applets


Predicting the use of Blackboard with the Technology Acceptance Model Recommendations

A first recommendation is to repeat this research at Zernike after six months covering ten courses, designed in the same way, with active announcements on Blackboard every week, including study planners and active feedback among peers through using the discussion board.

A second recommendation is to repeat this research in another secondary school with another OFLE in order to avoid the influence of Blackboard aforementioned results.

A third recommendation is to investigate a possible inclusion of other motivational variables such as computer anxiety and computer playfulness by future research.

Finally, further investigation has to point out whether Dutch pupils do not respond so well to Blackboard or handle the way an OFLE is used.



Thank’s for you’re attention and have a good conference!

Sheffield Conference 4 March 2006 - Reinder Vrielink International MSc E-learning, Multimedia and...

Documents

Transcript of Sheffield Conference 4 March 2006 - Reinder Vrielink International MSc E-learning, Multimedia and...