[IEEE 2011 International Conference on Multimedia Computing and Systems (ICMCS) - Ouarzazate,...

6
Genetic Algorithms for Online Remedial Education based on Competency Approach Khalid Jebari, Abdelaziz EL moujahid, Abdelaziz Bouroumi, Aziz Ettouhami LCS Laboratory, Faculty of Sciences, Mohammed VAgdal University, UM5A, Rabat, Morocco [email protected] Abdelaziz Bouroumi Ben M‟sik Faculty of Sciences, Hassan II Mohammedia University, UH2M, Casablanca, Morocco AbstractPersonalized learning is one of the main problems associated with the online remedial education (ORE) for e_learning systems. The current composition approaches fail to take into consideration the difference in individual learning- competency and the background knowledge of the individual learners and thus don’t provide the adequate teaching sequence that exactly meets the demands of the individual learners. In order to provide solution for this problem, we propose to use Genetic Algorithms (GAs) to configure personalized ORE for individual learners and maximize their success degree. To validate the practicability of the proposed approach, we investigate the achievements of students in actual learning environments. The investigated results show that the learning achievements of the students who are provided personalized ORE with our approach are better than the students who are provided a conventional uniform remedial education. Keywords- Genetic Algorithms; Course sequencing; Evolutionary computation; Web-based learning; personalized curriculum sequencing E-learning; Remedial Education. I. Introduction Remedial education has become an indispensable component of education in certain countries. According to the National Center for Education Statistics in USA, 89.9% of American degree-granting institutions offered remedial services through 2008-2009 [1, 2]. In Morocco, the government attention has been given to the issue of remedial education in recent years. Owing to the considerate decrease in the quality education, Moroccan colleges have to accept adding to teacher's timetable a number of hours for remedial teaching aid. They also devote whole weeks, before exams, for remedial courses and exercises for improvement. We have found that many teachers do tutoring in web sites or in specialized schools. But the major problem is that the class is heterogeneous. The level of students is very variable. Even the idea of classing in class A, B, C or D, as is customary in the objective-based approach is not good remedy [3]. Each student is a separate entity with his own knowledge; his own perceptions of things and of course his own skills. We must therefore associate for each student his own class which grants him a teaching sequence for his specific competencies. In a traditional class, this task is impossible [4] even if we want to benefit the advantages of E- learning. How can we lead a teaching sequence that respects the abilities and the potential of each student? How to choose the sequence of exercises for the student to pass gradually from one competency to another? How to maximize his success rate? In a test choose an exercise that is the first, another is the second and so on; this sequence plays a very important role in the learning process. We remember our teachers who suggested us to start with easy exercises and let the more complicated till the end. This advice is it fair? Various approaches to sequencing have been explored in numerous intelligent tutoring system (ITS) projects [5]. Huang et al used GA and case based reasoning for constructing an optimal learning path [6]. Van Den Berg et al constructed learning path by Particle swarm [7]. Others sequencing systems were present in literature like [8-12]. In this paper, a sequencing technique that automates the teacher‟s role is proposed. Tutoring units‟ sequences are defined in terms of competencies in such a way that generates a dynamic sequencing curriculum which based on GAs. GAs approaches [13], which have shown a good performance for solving a wide variety of problems, are used to find suitable dynamic personalized curriculum sequence of exercises within the solution space, respecting the constraints and maximizing student success. Section II describes the related literatures review. We present the methodology in Section III. This is followed by a description of the proposed method in Section IV. Section V presents the results obtained when GA is tested in simulated scenarios as well as in a real world situation (dynamic personalized curriculum sequencing in remedial education for mathematical in secondary school). Finally, in Section VI conclusions are summarized and future research lines are presented. 978-1-61284-732-0/11/$26.00 ©2010 IEEE

Transcript of [IEEE 2011 International Conference on Multimedia Computing and Systems (ICMCS) - Ouarzazate,...

Genetic Algorithms for Online Remedial Education

based on Competency Approach

Khalid Jebari, Abdelaziz EL moujahid, Abdelaziz

Bouroumi, Aziz Ettouhami

LCS Laboratory, Faculty of Sciences, Mohammed V–Agdal

University, UM5A,

Rabat, Morocco

[email protected]

Abdelaziz Bouroumi

Ben M‟sik Faculty of Sciences, Hassan II Mohammedia

University, UH2M,

Casablanca, Morocco

Abstract— Personalized learning is one of the main problems

associated with the online remedial education (ORE) for

e_learning systems. The current composition approaches fail to

take into consideration the difference in individual learning-

competency and the background knowledge of the individual

learners and thus don’t provide the adequate teaching sequence

that exactly meets the demands of the individual learners. In

order to provide solution for this problem, we propose to use

Genetic Algorithms (GAs) to configure personalized ORE for

individual learners and maximize their success degree. To

validate the practicability of the proposed approach, we

investigate the achievements of students in actual learning

environments. The investigated results show that the learning

achievements of the students who are provided personalized

ORE with our approach are better than the students who are

provided a conventional uniform remedial education.

Keywords- Genetic Algorithms; Course sequencing; Evolutionary

computation; Web-based learning; personalized curriculum

sequencing E-learning; Remedial Education.

I. Introduction

Remedial education has become an indispensable component

of education in certain countries. According to the National

Center for Education Statistics in USA, 89.9% of American

degree-granting institutions offered remedial services through

2008-2009 [1, 2]. In Morocco, the government attention has

been given to the issue of remedial education in recent years.

Owing to the considerate decrease in the quality education,

Moroccan colleges have to accept adding to teacher's

timetable a number of hours for remedial teaching aid. They

also devote whole weeks, before exams, for remedial courses

and exercises for improvement. We have found that many

teachers do tutoring in web sites or in specialized schools. But

the major problem is that the class is heterogeneous. The level

of students is very variable. Even the idea of classing in class

A, B, C or D, as is customary in the objective-based approach

is not good remedy [3]. Each student is a separate entity with

his own knowledge; his own perceptions of things and of

course his own skills. We must therefore associate for each

student his own class which grants him a teaching sequence

for his specific competencies. In a traditional class, this task is

impossible [4] even if we want to benefit the advantages of E-

learning.

How can we lead a teaching sequence that respects the

abilities and the potential of each student? How to choose the

sequence of exercises for the student to pass gradually from

one competency to another? How to maximize his success

rate? In a test choose an exercise that is the first, another is the

second and so on; this sequence plays a very important role in

the learning process. We remember our teachers who

suggested us to start with easy exercises and let the more

complicated till the end. This advice is it fair?

Various approaches to sequencing have been explored in

numerous intelligent tutoring system (ITS) projects [5]. Huang

et al used GA and case based reasoning for constructing an

optimal learning path [6]. Van Den Berg et al constructed

learning path by Particle swarm [7]. Others sequencing

systems were present in literature like [8-12].

In this paper, a sequencing technique that automates the

teacher‟s role is proposed. Tutoring units‟ sequences are

defined in terms of competencies in such a way that generates

a dynamic sequencing curriculum which based on GAs.

GAs approaches [13], which have shown a good performance

for solving a wide variety of problems, are used to find

suitable dynamic personalized curriculum sequence of

exercises within the solution space, respecting the constraints

and maximizing student success. Section II describes the

related literatures review. We present the methodology in

Section III. This is followed by a description of the proposed

method in Section IV. Section V presents the results obtained

when GA is tested in simulated scenarios as well as in a real

world situation (dynamic personalized curriculum sequencing

in remedial education for mathematical in secondary school).

Finally, in Section VI conclusions are summarized and future

research lines are presented.

978-1-61284-732-0/11/$26.00 ©2010 IEEE

II. LITERATURE REVIEW

Competency is an educational term related to the skills,

behaviors and knowledge that are necessary to be successful.

This can be applied to successful completion of a course or

success in a chosen career field. Competencies and learning

objectives are similar. Both are related to the desired learning

outcomes. Competencies are more general and specific relate

to skills, behaviors and knowledge that should be gained

through a course or series of courses. Learning objectives

relate closely to a specific lesson and support the

competencies [14].

Many courses or plans of study in schools are being identified

with core competencies. Core competencies are specifically

what a student must be able to do or understand proficiently to

successfully complete the course.[15]

Competencies can be written by building their structure upon

Bloom‟s taxonomy [16]; from the lowest to the highest level

in the cognitive, psycho-motor and affective domains. Each

one should introduce the skills, knowledge and behaviors

necessary for successful completion and being ready for the

next level of courses or success in a chosen career.

Each individual competency should be specific to the attribute

being described. It should start with an action verb followed

by an object, such as, “Analyze local, regional, national, or

global problems or challenges.” [17,18,19].

Writing competencies can seem a daunting task [20]. When

coupled with the familiar structure of Bloom‟s taxonomy they

become easier to create and identify. Remember to start with

an action verb, followed by an object and complete the

competency with specific information describing the outcome.

The action verb can be changed depending on how high of a

level the student should be able to achieve in the associated

domain.

III. METHODOLOGY

The system architecture of dynamic personalized curriculum

sequencing using the competency learning (DPCSCL)

proposed in this paper has an architecture that is designed as

shown in figure 1. It is an implemented version of the

framework of the genetic-based, a database of pedagogical

items (short lessons in the form of learning activities ended

with quizzes, exercises and multiple choice questions for each

topic) and competency learning approach. When learner has

chosen topic 1 for example, the system generates a test, which

is composed by P pedagogical items then the learner will first

undergo his first formative assessment, and the system will

calculate student‟s scores and analyze their learning situation.

As a result, if he fails to reach competency for topic 1, the

system will recommend an appropriate personalized

curriculum sequencing suggestion based on GA, the sequence

is composed by a certain number of tests, and this number

depends on the competency degree of the learner. The test

consists of P pedagogical items. The sequence 1,2,...,P is

permutation generated by GA respecting the relevance degree

between question i and i+1, 11 Pi . On the other hand, if

they reach competency of the topic 1 then they will continue

with the next topic or extension materials like integration

exercise, i.e. exercise that integrates all the skills already

acquired previously. We have proposed a model that

formulates the DPCSCL under different assessment criteria.

With regard to each test item, this model maintains three

assessment considerations which are the difficulty level of

each test item, the relevance association between each

question and also with each topic and the learner competency

degree. Assume the Pedagogical Item Database (PIDB)

comprises of N pedagogical items, I1, I2,…,IN for topic t. When

i questions are selected to test learners from the PIDB by the

GA in a specific order. These questions will be a subset of P

pedagogical items, Pi . Assume that an examination aims at

P questions; therefore each question selected from the PIDB

should relevant learner competency, relation degree between

the next question and question difficulty. But in case of

summative evaluation, assume that a test at M topics which

consists of i questions, therefore each question selected from

the PIDB should relevant to one or more of these topics, say rj

is the question r of the topic j, 1 ≤ j ≤ M . Moreover, each

topic has a different weight wj , 1 ≤ j ≤ M which is assigned

by the instructor. For example, to test the mathematical

function analysis of learners which consists of Derivative,

Function Domain, and Limits , the teacher can assign different

weights to these topics, such as w1 = 0.18 (weight of

Derivative), w2 = 0.03 (weight of Function Domain), and w3

= 0.1 (weight of Limits). These weights specify that

„„Derivative” is more important than the topics „„Limits” and

„„Domain” in this examination. As mentioned above, with

regard to the relevance between test items and topics, we use

keywords to determine which question is relevant to the

particular topics. The teacher can select different keywords for

an examination via the instructor interface of the dynamic

question generation system. These keywords represent which

topics are involved in this test. Therefore, the dynamic

curriculum sequencing generation system would control the

frequency of each test item. The following describes the

variables used in the dynamic curriculum sequencing:

wj, 0< wj < 1 and 1 ≤ j ≤ M, weight of topic j.

The sum of total wj is 1;

D, 0 ≤ D ≤ 1, competency level for each learner‟s

status, calculated for each test as:

iwP

rkstudent_ma

where P number of question in the test.

dk, 0 ≤ dk ≤ 1, degree of difficulty of test item Ik;

rj, 1 ≤ j ≤ M, relevance of association between

selected question and topic j. rj is 1 if select question

is relevant to the topic j, 0, otherwise;

experts define the curriculum relation degree between

the different pedagogical items of database as a matrix

: E. Then we normalize elements of the matrix P by:

n

=kjk

n

=kik

n

=kjkik

ij

ee

ee

=c

1

2

1

2

1 (1)

The above formula is the fitness function of the dynamic

question generation model, and its three constraints which are

described as follows:

1

1

11

21

n

+M

rw

+Ddα=Ifii

n

=ij

M

=jj

kk (2)

Where 1 ≤ j≤ M, 1≤ k ≤ N.

Ddk

Indicates the difference between the degree of difficulty of

selected test and the student difficulty level.

M

rw j

M

=jj

1

Represents the degree of relevance between the selected

questions and particular topics.

1

21

n

cM

=iii

Represents the concept relation degree of the ( i-1) curriculum

with the i curriculum in the constructed learning path.

α : Degree of competency

Through the computation and iteration each test item has

obtained a fitness value from the fitness function. If a test item

Ik contains minimal fitness value, it will be selected by the

dynamic curriculum sequencing system.

Figure 1 shows the architecture of a DPCSCL system that is

comprised of four components. The following parts can be

described:

Figure 1. System Architecture of DPCSCL for one topic

Pedagogical Item Database: Its contents are

organized in test items. A test item consists of several

pieces of information, such as a question‟s content,

difficulty level, exposure frequency, the weight of

each topic, the test time, response of item and

keywords. Each test item can be defined or associated

with the different topics according to keywords in the

proposed system ;

Student Interface: The Student interface is where

students take tests. The question is displayed and

students can answer it through the XML interface ;

Formative Evaluation: is basically a diagnostic

instrument or process used by the system. In our

context the system compare the student response and

the correct response given by PIDB. Formative

Evaluation is also a principal aid in the planning of

corrective measures to remedy learning errors and a

powerful motivational device by showing students

directly that they can improve their learning and

become successful learners. Therefore, students can

move to the next topic of instruction. Finally, there is

the development of a summative examination.

IV. PROPOSED METHOD

This section explains how to generate the Dynamic learning

path (DLP) for Web-based instruction, utilizing the GA.

We try to find optimal curriculum sequence for maximizing

success degree for student for each topic and finaly for

summative evaluation.

In order to conceive a genetic solution to the problem, we

have to determine the encoding method. Then the fitness

function in equation 2 is used for assessing and comparing the

DLP. Thus, starting from an initial population of randomly

generated individuals, which are DLP, in our case. We

evolved this population toward better solutions according to

the rules of selection strategy, crossover and mutation. The

details are as follows:

Encoding Method: In this study, a serial number is assigned

to each curriculum from 1 to n if there are a total of n

curriculum in the curriculum database for the learning path

generation. Thus, the assigned serial number of each

curriculum is combined directly with the serial number of the

successive curriculum as strings to represent the generated

learning path for the genetic algorithm. The whole individual

represented by the chromosomes of all curriculum parameters

for the genetic algorithm.

Initial population size: Generally, the initial population size

can be determined according to the complexity of the solved

problem. A larger population size will reduce the search speed

of the GA, but it will increase the probability of finding a high

quality solution. To construct a high quality learning path for

an individual learner, the initial population size in this

research is chosen as 50 for the generation of a personalized

curriculum.

Student Interface

Formative Evaluation

Tutoring Unit

Curriculum Sequence based GA

Pedagogical Item Database

Fitness function: The fitness function is a performance index

that it is applied to judge the quality of the generated learning

path for the GA. In order to generate a personalized learning

path for an individual learner based on the pre-test results, the

difficult parameters of the curriculum and the concept relation

degrees of the curriculum must be considered simultaneously

to determine the fitness function. In our method, the learning

path constructed by the GA only considers the curriculum for

which the learner gives incorrect pre-test results. Moreover,

the curriculum with the smallest difficulty parameter is

selected as the first curriculum ranked in the constructed

learning path. Therefore, the fitness function is formulated as

equation 2.

Selection operator: In the selection operation, the

chromosome with the larger fitness function value will have a

higher probability to reproduce the next generation.

The aim of this operation is to choose a good chromosome to

achieve the goal of gene evolution. The most commonly used

method is Tournament Selection. In this study, we used a

modified Tournament Selection, which guards in each

iteration the best individual. The following pseudo code

summarizes the Selection Method.

// N: population size

T_alea: array of integer containing the indices of individuals

(PLP) in the population

T_ind_Winner : an array of individuals indices 's who will be

selected

Lsorted : a list of all individual indices sorted in decreasing

fitness values

l = 0

k=0

For (i=0; i<k; i++)

{

Shuffle T_alea ;

For (j=0; j<N; j=j+k+1)

{

C1 = T_alea(j);

For (m=1; m<k; m++)

{

C2=T_alea(j+m);

if f(C1)< f(C2) C1 = C2

// f(Ci): Fitness of individual Ci

}

T_ind_Winner(l) = C1

T_ind_Winner(l+1) = Lsorted (k)

l=l+2;

k=k+1

}

} Figure 2. Tournament Selection Modified

Crossover operation: In the crossover operation, the two

randomly selected serial numbers of the chromosomes in two

individuals exchange the entire chromosome by probability

decision. This operation aims to combine two parent

chromosomes to generate better child chromosomes.

In our study, the Partially Mapped crossover operation is used.

However, in order to avoid that the generated learning path

has a duplicate serial number of chromosomes or that the

serial number of the curriculum is over the total number of

curriculum after the crossover operation is performed, the

crossover operation will exchange the whole chromosome by

probability decision. With other words, the performed

crossover operation can avoid the generation of an illegal

learning path. In this research, the probability of crossover is

0.7

Mutation operation: we used Inversion Mutation. The

inversion mutation [12] randomly selects a sub sequence,

removes it from the sequence and inserts it in a randomly

selected position. However, the sub sequence is inserted in a

reversed order. The mutation operation will create some new

individuals that might not be produced by the reproduction

and crossover operations.

Generally, a lower probability of mutation can guarantee the

convergence of the GA, but it may lead to a poor solution

quality. On the other hand, a higher probability of mutation

may lead to the phenomenon of a random walk for the GA. In

this research, the probability of mutation is set to be 0.1.

Stop criterion. The genetic algorithm repeatedly runs the

reproduction, crossover, mutation, and replacement operations

until it meet the stop criterion. The stop criterion is set to be

100 generations, because this criterion can obtain satisfied

learning paths for the individual learner

V. EXPERIMENTAL RESULTS

The curriculum organized on a single Web page is a test

composed by 20 pedagogical items in the personalized

curriculum approach. In our approach, the tutoring unit's,

“Mathematical function analysis” for scientific bachelor, is

used to generate dynamic curriculum personalized learning

path. The tutoring unit's includes many topics with different

weight see table I, each topic contains a certain number of

pedagogical items, with various level of difficulty see table II.

In order to prove the efficiency of our approach, we have

chosen two classes with 15 students for each class.

TABLE I. THE CORRESPONDING DIFFICULTY PARAMETER FOR

EACH TOPIC

Topic Title of Topic Weight

I1 Function Domain 0.03

I2 Limits 0.1

I3 Derivative 0.18

I4 Asymptote 0.13

I5 Inflection point 0.14

I7 Tangent 0.07

I8 second derivative 0.19

Topic Title of Topic Weight

I1 Function Domain 0.03

I9 intersection with the main axes 0.04

I10 turning point 0,12

For the first class (Class1) students had the opportunity to

choose the exercises of our database at will. They passed

indifferently a Topic to another, but the only constraint is the

time, the exam lasted two hours.

For the second class (Class2), we have applied our approach.

The student may begin with the topic that suits him, but the

system automatically generates 20 exercises based on GA.

Once the student's submit test response, system based on GA,

suggests another sequence of exercises based on the

competence of the student (see an example of learning path

suggested by our method in figure 4). The system continues to

offer the curriculum sequence until termination of all topics

that make up the tutoring unit's.

The duration of the experiment is 2 weeks to one session of 2

hours each day.

TABLE II. DESCRIPTION OF TOPIC IN PEDAGOGICAL ITEMS IN

DATABASE

Topic Number of Pedagogical Items Average Difficulty

I1 50 0.53

I2 60 0.6

I3 70 0.58

I4 50 0.63

I5 40 0.54

I7 50 0.77

I8 60 0.69

I9 40 0.84

I10 50 0,42

We began our experience with a series of 3 pretest spread over

3 days. The same test is given for each class. The examination

lasts 2 hours including 2 exercises on the analysis of

mathematical functions.

At the end we have proposed 3 post-test spread over 3 days. In

this case also the duration of each examination is 2 hours.

Each exam includes an analysis of 2 mathematical functions.

During the correction of copies of Class1, we found that 6

students have problems with certain topics, so their scores

were low. Directly by asking certain questions, we concluded

that they spent much time in certain topics and have neglected

other topics. The other students had almost identical results in

the pretest and pos-test. For cons students in Class2 have

improved their results, we found a marked difference between

the scores of the pretest and the post-test.

The figure 3 shows the Average percent improvement from

pre-test to post-test results of two classes , we noticed a

marked improvement for the class class2 while for class class1

there is no important difference.

Figure 3: Average percent improvement from pre-test to post test

Figure 4: Test given by GA for one learner

VI. CONCLUSION

This paper proposed a personalized curriculum generation

approach based on GA module for personalized learning path.

The proposed learning path generation approach can

simultaneously consider the curriculum difficulty level and the

sequence exercises in remedial education processes. We used

empirical study to indicate that the proposed approach can

generate appropriate tutoring materials to learners based on

individual learner requirements, and help them to learn more

effectively in a Web-based environment.

In future work we will consider working with a large

Pedagogical Item Database. However the run time of the

algorithm increases with the size of the database. Therefore

we will conduct a prior classification before applying our

method.

REFERENCES

[1] NCES (National Center for Education Statistics),

http://nces.ed.gov/programs/digest/d07/tables/dt07_317.asp, available on April 20, 2008

[2] Jeff E. Hoyt, “Remedial Education and Student Attrition”, Community

College Review, Vol. 27, Issue 2, 1999, pp. 51 [3] F. E. Ritter, J. Nerb, E. Lehtinen, T. M. O'Shea, "In Order to Learn:

How the Sequence of Topics Influences Learning", Oxford University

Press, 2007.

[4] S. Billett, "Learning Through Practice: Models, Traditions, Orientations

and Approaches", Springer 2010.

[5] C. Frasson, G. Gauthier, & G. I. McCalla (Eds.), “Intelligent tutoring

systems” pp. 499–506. Berlin: Springer-Verlag [6] M. J. Huang, H. S. Huang, and M. Y. Chen, “Constructing a

personalized e-learning system based on genetic algorithm and case

based reasoning approach,” Expert Systems with Applications, pp 551-

564, 2007.

[7] B. van den Berg, R. van Es, C. Tattersall, J. Janssen, J. Manderveld, F. Brouns, H. Kurvers, and R. Koper, "Swarm-based sequencing

recommendations in e-learning," in Proceedings 5th International

Conference on Intelligent Systems Design and Applications, 2005.ISDA '05., Wroclaw, Poland, 2005, pp. 488-493.

[8] C.M. Chen, "Intelligent Web-Based Learning System with Personalized

Learning Path Guidance," Computers & Education, vol. 51, pp. 787-814, 2008.

[9] P. Karampiperis, "Automatic Learning Object Selection and Sequencing

in Web-Based Intelligent Learning Systems," in Web-Based Intelligent E-Learning Systems: Technologies and Applications, M. Zongmin, Ed.

London. UK.: Idea Group, 2006

[10] A. Meng, L.Ye, D.Roy, & P.Padilla, Genetic algorithm based multi-

agent system applied to test generation. Computers & Education, 49(4),

1205–1223, 2007. [11] GJ. Hwang, et aI., "An Enhanced Genetic Approach to Optimizing

Auto-Reply Accuracy of an E-Learning System," Computers &

Education, vol. 51, pp. 337-353, 2008 [12] Y.C. Chang, et aI., "A Learning Style Classification Mechanism for E-

Learning," Computers & Education, vol. 53, pp. 273-285, 2009..

[13] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, New York, NY, 1989.

[14] Z. Michalewicz, Genetic Algorithms + Data Structures=Evolution

Programs, 3rd ed., Springer, 1996. [15] J. W. Burke, "Competency Based Education And Training", Routledge,

1989.

[16] P. Brusilovsky, "Adaptive and Intelligent Technologies for Web-based Education," Künstliche Intelligenz, Special Issue on Intelligent

Systems and Teaching, vol. 4, pp. 19-25, 1999.

[17] A. Gonczi, “Competency based education and training: A world perspective,” Mexico City: Grupo Noriega Editores; 2000.

[18] D.C. Leonard, "Learning Theories A TO Z," Greenwood Press, 2002.

[19] J.A. Bowden, F. Marton, "The university of learning: Beyond quality and competence," Oxford: Routledge, 2004.

[20] N.E. Gronlund, "How to write and use instructional objectives",

Englewood Cliffs, NJ: Merrill. 1999.