FIPSE Guide

8/14/2019 FIPSE Guide

1/30

1

A grant for Integrating Electronic Technologyinto Liberal Education & Professional Education Classes

FIPSE Grant 1998-2000 by Dr. Barbara K. Iverson ColumbiaCollege Chicago

If we took the research on persistence seriously, we would,at a minimum, move to forms of academic organizations thatrequire students to become actively involved with others inlearning. We would construct educational settings that

promote shared, connected learning.

Vincent Tinto in Colleges as Communities: Taking Research onStudent Persistence Seriously. Review of Higher Education.21.2(1998) 167-177.

Teaching with S.T.Y.L.E. (situated technologyyielding learning enhancement)


2/30

2

What's Inside

3 ........................................................................................................................................Overview

4 ....................Teaching with S.T.Y.L.E. (situated technology yielding learning enhancement)

5 ...........................................................................................What happens during the semester?5 ....................................................... Measuring Student Teamwork and Student Achievement

6 ......................................................................................................... Why Teach with S.T.Y.L.E?

6 ..........................................................How do Faculty Learn about Technology on-the-job?

6 ...............................................Writing some learning objectives for the collaboration project

7 ....................................................................Project Objectives Evaluation Strategy Summary

8 .................................................................................................................... Digital Skills Invento

10 .......................................................................................................... Product Evaluation Form

10 .......................................................................................................... Evaluation Questionnaire

10 .......................................................................................................... Teacher Evaluation Form

11 .....................Student Evaluation Form -- The L.E.I. or Learning Environment Inventory13 .............................................................................. Summary--Teaching with S.T.Y.L.E. Steps

13 .................................................................. What we have learned this far--benefits and risks

14 ................................................................................................................. Whats best & worst?

14 .......................................................................................Are faculty getting new digital skills?

15 ...........................................................................How did students fare in the collaborations?

15 .................................................................................................Tech Classes vs. Control Group

16 ............................. Latin American Art Music & Culture Content Class vs. Control Group

16 ................................................................Nutrition Science Content Class vs. Control Group

17 ..................................................................................................................................Conclusions

18 .......Highlights of F.I.P.S.E. Annual Progress Report (1998-1999) and Current Semesters

Progress19 ........................................................................................................................... REFERENCES

21 ..................................................................................................................................Appendix A

21 ................................................................................................................... The Grant Proposal

21 .....................................................................................................................I. Project Narrative

23 ......................................................... II. Proposed strategies for improving current practices

27 ...........................III. Institutional commitment--contribution of the College to the project

27 ............................................................................................. IV. Evaluation and Dissemination


3/30

3

Overview

Our project is about active learning -- for faculty and students -- as part of collaborative,team-based projects. We want to use digital technology tools (computers, the Internet, e-mail)to create better learning environments in our classes. The integration of the technology intoteaching creates new ways of organizing instruction, assessing progress and learning, and ofscheduling time and activities. Faculty learn to use the technology on-the-job in specificclasses, which is time-saving and eases their learning curve. Students get to participate in anactive, project-based assignment that draws on multiple learning styles, and opportunities todevelop higher order thinking skills (perspective-taking, synthesis, analysis, evaluation.) Asknowledge makers, students experience a sense of ownership and empowerment that theymiss as passive knowledge consumers.

Our model, funded in part by a Federal Improvement of Post-Secondary Educationgrant sets up a simulation of the contractor/client relationship common in business and

professions today. It pairs up a content class (the clients) with a technology class (thecontractors) to make an interactive hypermedia presentation for the web and CD-ROM. Thecontent teacher gets help from the tech teacher in integrating use of the Internet, elec-tronic presentations, e-mail, electronic forums and bulletin boards into their classroom rou-tine. The students learn in context and by doing, rather than by rote. In general, the team-work inherent in the model increases the cohesiveness of classes involved in the project,which is associated with higher achievement scores. The simulation and project represent ascalable shared assignment developed by the participating teachers to fit the constraints andneeds of their specific classes. The introduction of the technology-supported project intoexisting classes provides a way to update teaching throughout the institution without creat-ing new curricula or hiring new faculty.

The grant provides funding for replacement costs for participating teachers to easedepartmental concerns about workload issues. It funds teaching assistants for each class, aswell. The teacher partners negotiate the specific topics that will be covered and what thefinal outcome project will be to suits their needs. Teachers structure the team-based project toachieve theirsame old learning objectives in new active, collaborative ways through theteam project. They need to meet with the partner class, as well as their own, and to use theirT.A. to help facilitate the collaboration. The project can be continued for several semesters, ifthat is agreeable to the participating faculty, or it might last a single semester.

An outside evaluation consultant conducts qualitative and quantitative assessments ofthe project and its efforts (see Evaluation to Date.) Teachers and students try out tech tools asthey work on the projects. After pariticipating, teachers can continue to collaborate. They can

incorporate internet research into all their classes. They can create team projects presented ascomputer slideshows. They can use electronic forums to stimulate discussion or check on stu-dents understanding of reading assignments. The aim of the project is not to prescribe a singleuse or method of technology, but to give teachers the experience they need to integrate avail-able tools into the dialogue, communication, and teaching/learning process.

Project Director, Dr. Barbara K. Iverson


4/30

4

TeachingwithS.T.Y.L.E. (situatedtechnologyyieldinglearningenhancement)

To participate, you need a partner. If you teach a class with a technological focus, look for apartner whose class focus is on content, and vice versa. For example, a New Media Projects teacherteams with Senior Seminar for a project to create a virtual job placement and information center.

The project director can help you find a partner.

The partnered faculty meet to decide what form your project will take. A poet works with atime-based composing class on interactive literary collages. A psychologist finds multimedia stu-dents who can animate non-locality. The teachers decide if the project will be one of severalassignments, or the main assignment. They set how many weeks to schedule for it. You plan forusing resources like computers and the Internet , to get the assignment coordinated and provideopportunities for teamwork and collaboration within and between classes. The tech teacherwill help the content teacher get up to speed with e-mail attachments, using the internet, andother tech tools. This is an exciting time where you can come up with creative ideas, and work tobring them to life.

You need to share syllabii, and figure out what the project will look like or be. Will it be awebsite for future classes to use? Will it be an interactive compilation of art or poetry (or both?)Will it be a new way of making something? The project outcome is flexible and scalable, and well-suited to a variety of cross-discipline collaborations. The partner teachers should set the dateswhen they will bring the classes together (ideally the class meeting times allow for face-to-facemeetings) or when the teachers will visit their partners classes. How will information be passedfrom clients to contractors? How will clients monitor the progress of the contractors? The facultypartners set this up based on the scope and needs of their specific project. The project director willmeet with as often as you like.

As part of the collaboration, you will be responsible for producing a common product out ofthe collaboration. You and your partner will be expected to use and experiment with e-mail, elec-

tronic forums, computers and ways of sharing information and working together both face-to-face and asynchronously. You can use the tools with each other, and with your students in waysthat work in your situation.

The project is evaluated through collecting faculty and student opinions about the project,and comparing that with a series of questions about the classroom learning environment (howcooperative and cohesive students feel about the class) and an inventory of faculty digital skills.Research shows that classes with a sense of cooperation, cohesiveness as a group, satisfaction, anda sense of having a say in the class (democracy) do better on achievement outcomes than classeswithout a positive sense of groupness.

Part of the reason for building the project around a team-based collaboration is to build up

the learning environment of your class so it will promote higher achievement. The team-basedproject assignments provide opportunities to increase the positive sense of teamness and col-laboration, and by inference achievement. Motivation increases in active-learning situations, whichleads to students spending more time on task, and learning in classes where they are engaged. Youare still the one who evaluates achievment in your subject and class. However, we can comparedisparate classes and draw conclusions about the team-based project approach to integrating tech-nology into classes by measuring the learning environments we create, and attempting to deter-mine if we are changing them in positive ways because of our team-based projects.


5/30

5

Whathappensduringthesemester?

The semester begins and the project is introduced to the classes. When the initial learning inboth classes permits, the project assignment begins. E-mail, electronic forums, networked com-puters and ftp and websites all facilitate how information passes between the two classes. Theproject is built and reviewed by the clients, revised and added to as comments and work comes infrom the client class. Faculty need to remain flexible, as they determine what changes they mayneed to make during the weeks of class as they see how the plans they made play out in theeveryday classroom dynamics.

During the semester the evaluator will take measures via quick questionnaires of studentand faculty opinions and perceptions. The project itself is an additional measure of the progressand success of the collaboration.

MeasuringStudentTeamworkandStudentAchievement

Faculty continue to evaluate the academic achievement for their classes in the manner they

see fit. For project evaluation, students complete a questionnaire which measures six aspects ofthe learning environment we create (see Table 1.) This questionnaire has been carefully developedand studied so that we know which characteristics of a classroom learning environment are asso-ciated with increased student achievement in any class (Fraser, Anderson & Walberg, 1982.) If wetry and measure student achievement directly, and then compare our classes, we will run into anapple and oranges problem how do we compare what students learned in Womens History,with what they learned in CD-ROM Multimedia Production? However, if we measure the learn-ing environment we consciously create, we can make reliable inferences about what is happen-ing with student achievement in our classes.

For example, we know classroom cohesiveness (a feeling of being a team) and student satis-faction are associated with subject-matter achievement gains. We assess the learning environ-

ments of project classes and compare them with similar, but traditionally organized control-groupclasses to see whether our team-based, collaborative approach is associated with higher studentachievement.

We rate the learning environment using a Learning Environment Inventory that measures:cohesiveness (sense of being a team), diversity (extent to which differences between studentsexist and are provided for), material environment, democracy (extent to which students shareequally in decision-making related to the class), and satisfaction (extent of enjoyment of the class.)

Cohesiveness refers to how favorably students see themselves and their work as a team.Diversity reflects whether students with different abilities and interests are encouraged. Thematerial environment refers to availability appropriate of books, equipment, and space. Democ-

racy measures the perceived student input in classroom decision-making. Finally, satisfactiongauges whether the students enjoy the class. These scales were selected to assess the fulfillmentof the project objectives to improve cooperative and active learning.

The Internet is available on campus, but not always integrated into classroom learning.Teacher participants and student participants use e-mail, electronic bulletin boards, listservs,and other Internet-based tools to keep in touch and collaborate. As the content teachers areguided in their use of this technology, they begin to apply what they learn in their research and


6/30

6

other assignments. By learning to use the particular resources at hand while they were teachinge.g. on-the-job, the cost of faculty training is kept low. Teachers who have used technologysuccessfully in their classrooms become better judges of what new innovations might be useful,and which are not important to their discipline or teaching style.

Why

Teach

with

S.

T.

Y.

L.

E?

This model and how we assess it are based on research on what works to improve class-room learning so it provides a framework to guide us in designing the team-based assignmentsthat are flexible, adaptable to existing classes, and effective in promoting student learning andfaculty technology use. It is based on what we know about retention and how learning takesplace.

We know now where we need to be headed (e.g. toward creating a sense of team, towardallowing for diversity within an environment where we have clearly stated learning goals) andwe can structure a collaboration that lets students attain learning objectives in setting that sup-ports active, potent, social learning.

The fear of giving up teacher control is real, but the payback in terms of facilitatingauthentic higher order thinking and enjoyment of learning, coupled with the pleasure arisingfrom collaboration with a fellow teacher in the team-based, collaborative learning environmentmakes it worth the risk.

HowdoFacultyLearnaboutTechnologyon-the-job?

By trying out and using technology while they work, teachers learn tech skills easily, andin context. This is the Trojan-horse approach to teaching faculty to use technology to improve

their teaching -- by focusing on the teaching, we reduce the anxiety around technology andshorten the learning curve. The Digital Skills Inventory we administer measures changes infaculty tech skills. The skills it measures are targeted to correspond to areas where contentfaculty are expected to gain confidence and expertise because of their involvement in theproject.

Writingsomelearningobjectivesforthecollaborationproject

Here are sample objectives that relate to the teamwork objectives of the project. Pleaseinclude something like this on your syllabus. These can be the same for the tech and contentteacher. By including these in your syllabus, they will become a part of the classroom learning

environment:

Students will develop a sense of being part of team as a result of class activities (cohesive-ness)

Students will be encouraged to express and accept a range of opinions about the project andto work on topics of individual interest, as well as achieving group goals (diversity)


7/30

7

Students will participate democratically in goal setting in creating the joint project (democ-racy)

The project will combine goals set by the instructor and goals determined by the students, andthe class will produce a clear list of these goals to guide the project (goal direction)

Students will express satisfaction with their participation in the team-based collaborativeproject as well as other class activities. (satisfaction)

ProjectObjectivesEvaluationStrategySummary


8/30

8

DigitalSkillsInventory

There are no right or wrong answers on this checklist. We are assessing faculty computer skills related to a project we are

conducting to integrate technology into classrooms. This information is provide us with baseline data for assessing our project.

Your responses are appreciated. Please circle the response that best characterisizes your skills at this time. Thanks.

Word Processing and Using Documents

Do you use a computer to:

Write letters, notes, or flyers?

Type academic papers or journal submissions ?

To write press releases and artist notes?

To create syllabi & course handouts?

To complete administrative tasks

Telecommunications

Do you use the on-line catalog of any library?

Do you browse the WorldWide Web (Internet) ?

Do you use the Internet as a research tool in your work?

Have you created a web page?

Do you have a personal webpage or web page for your classes?

Can you annotate bookmarks & make bookmark webpages in a browser?

1 2 3 4

Never Seldom Often Always

1 2 3 4


1 2 3 4


1 2 3 4

Never Seldom Often Always1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4



9/30

9

Do you know what FTP is, and use it?

Do you use e-mail for personal communication?

If no, skip to the next set of questions.

Do you have your own e-mail account?

Use e-mail with friends?

Use e-mail to communicate with students and other professionals?

Do students submit homework for your class electronically?

Do you use a listserv or chatroom with students?

Images & Graphics

Do you scan images to use with a computer?

Have you cropped or resized and manipulated digital images?

Use computer images in your written communications?

Create charts & graphs with a computer and use them in your work?

Use clipart?

Desktop Publishing

Create computer newsletters, flyers, programs, or schedules?

Add and manipulate images in these documents easily?

Usee presentation software like PowerPoint?

1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4


1 2 3 4



10/30

10

ProductEvaluationForm

Name________________________ Date__________________ Dept._______________________

Name of Product being evaluated:_________________________________

Instructions: You should have received a product (CD ROM or website address) produced by the Integrating Electronic

Technology in Liberal and Professional Education Classrooms project. Please respond to the following questions and return thisform to Barbara Iverson, Project Director, within one week. Thank you.

1. What is your opinion about the subject content of the product?

2. What is your impression about the multimedia aspects of the product?

3. What was the best thing about the product?

4. What was the worst thing about the product?

5. Do you have any suggestions to improve the product?

6. What is your overall impression of the product?

7. Any additional comments?

Evaluation

Questionnaire

Instructions: With respect to (course title) please answer the following questions and reply with your answers by (date).

1. In your opinion, what are the goals of the class?

2. What is the best thing about the class?

3. What is the worst thing about the class?

4. What do you suggest be done differently?

5. How would you compare this class to other courses in this discipline?

Teacher

Evaluation

Form

Instructions: Please answer the following questions with respect to, Integrating Electronic Technology in Liberal and

Professional Education Classrooms.

1. What were the goals of the project?

2. Were the goals met?

3. What was the best thing about the project?

4. What was the worst thing about the project?

5. What do you suggest should be done differently?

6. Do you think this project represents a cost-effective method of instruction?

7. Was this project an effective way to integrate computers into the classroom?

8. Were your computer and digital skills improved as a result of this project?

9. Has this project changed your teaching?

10. Did you or do you plan to share project ideas with those outside the project?

11. Do you think this project would benefit other campuses?


11/30

11

StudentEvaluationForm --TheL.E.I. orLearningEnvironmentInventory

Instructions: Please answer the following questions with respect to (course title) using the following scale:

Strongly Agree = 4 Agree = 3 Disagree = 2 Strongly Disagree = 1

Please respond to all the questions and place your answer (whole numbers only 1, 2, 3, or 4) in the left column next to thequestion number. Please reply with your responses by (date). Thank you.

1. Members of the class do favors for one another.

2. The class has students with many different interests.

3. The books and equipment students need or want are easily available to them in the classroom.

4. The class knows exactly what it has to get done.

5. Class decisions tend to be made by all the students.

6. The students enjoy their class work.

7. A student has the chance to get to know all other students in the class.8. Interests vary greatly within the group.

9. A good collection of books and magazines is available in the classroom for students to use.

10. The objectives of the class are not clearly recognized.

11. Decisions affecting the class tend to be made democratically.

12. Personal dissatisfaction with the class is too small to be a problem.

13. Members of the class are personal friends.

14. Some students are interested in completely different things than other students.

15. The students would be proud to show the classroom to a visitor.

16. Students have little idea of what the class is attempting to accomplish.

17. Certain students have more influence on the class than others.

18. Many students are dissatisfied with much that the class does.

19. All students know each other very well.

20. Class members tend to pursue different kinds of problems.

21. The room is bright and comfortable.

22. The objectives of the class are specific.

23. Certain students impose their wishes on the whole class.

24. There is considerable dissatisfaction with the work of the class.

25. Students are not in close enough contact to develop likes or dislikes for one another.

26. The class divides its efforts among several purposes.

27. There are displays around the room.

28. Each student knows the goals of the course.

29. Each member of the class has as much influence as any other .


12/30

12

30. The members look forward to coming to class meetings.

31. The class is made up of individuals who do not know each other well.

32. The class is working toward many different goals.

33. The classroom is too crowded.

34. The class realizes exactly how much work it is required to do.

35. What the class does is determined by all the students.

36. After the class, the students have a sense of satisfaction.

37. Each student knows the other members of the class by their first names.

38. Different students vary a great deal regarding which aspects of the class they are interested in.

39. There is enough room for both individual and group work.

40. Each student in the class has a clear idea of the class goals.

41. A few members of the class have much greater influence than the other members.

42. Students are well-satisfied with the work of the class.


13/30

13

Summary--TeachingwithS.T.Y.L.E. Steps

1. A pair of collaborators decide to work together, and meet with the project director

2. The pair share syllabii, look over suggested learning objectives for collaboration andteam outcomes, and discuss a shared project.

3. The pair meet and create a shared project description. Project Director can write up thedetails as a product descriptor for each teacher to use with their class (the descriptors for eachclass will be different because their activities and contributions to the project are different.)

4. E-mail, electronic forums and FTP and websites are established, and faculty try these outand learn about them prior to meeting with their classes, as part of the planning of the project.

5. Dates for the classes to meet (synchronous or asynchronous) are set. When the teacherswill visit each others classes is set and discussed.

6. Evaluation times are set at the beginning, middle and end of the project.

7. Semester begins, project begins as planned (not usually until the 4th or 5th week of thesemester.)

8. Students share info and input about and for the project. The clients view the work inprogress, and suggest revisions. Contractors prepare the work for shared viewing.

9. The project is completed. Final evaluations take place.

10. Teachers and project director debrief on how the project went. Revisions are made ifthe collaboration is going to be carried out again.

What

we

have

learned

this

far--benefits

and

risks

Since beginning this project at Columbia, teachers have increased and refined how they usee-mail with their students, experimented with electronic forums, and worked to get their de-partments to purchase tech resources they need to support the integration of tech into theirteaching. As a direct result of participation by several faculty from Liberal Arts and the efforts ofthe project director, all Lib.Arts classrooms are wired for internet and two computer/tech cartsare being assembled (computer, internet hookup, scanner, printer, DVD, CD-ROM, PowerPoint,web browsers, easily moveable robust cart with lots of electric outlets.) The lag between theinitial collaborations with the promise of this supportive technology and its acquisition has

attenuated the positive effects of the grant because the teachers who used the technology duringthe grant projects have been unable to continue to use the tools they learned in their subsequentteaching. With the late, but welcome acquisition of supportive hardware and software, thebenefits of the grant will be obvious, and will begin to percolate throughout the institution.

Teachers who have had access to computer and internet use in classrooms continued andhave increased their use. Obviously those whose access is limited have not been able to usewhat they learned in the grant effectively--but new equipment will change this.


14/30

14

This table shows whatteachers who participated inthe program said were thebest and worst aspects of

being in a F.I.P.S.E. collabo-ration. The teachers ratedthe technology, the collabo-ration, and the shift in rolefrom sage on the stage tofacilitator postively. Theyindicated problems or short-comings in the communicationbetween classes, caused by thelack of overlapping meetingtimes, and the necessity for

asychronous meetings facilitatedby internet, e-mail and electroniccommunication tools. On our campus, this is not the common prac-tice, nor within student expectations at this time. The difficulty scheduling these classes inoverlapping timeslots seems insurmountable, so probably the novelty of asynchronous elec-tronic communication will wear off before scheduling is easier, and this kind of communicationwill become routine as it is on many campuses.

This table demonstrates that teachers in the

collaborations were improving theirdigital skills. Little gain in wordprocessing was measuredbecause most teachers enteredwith good skills in wordprocessing. We arerevising this survey to tryand measure the changesin teacher skills in moredetail. Anecdotal evi-dence shows teachers

especially liked theelectronic bulletinboards, but the lack ofcomputer facilitiesavailable to the teach-ers after they partici-pated in the project interfered with the greater use of the this useful tool. In fact,the interactive products have not been used to the best advantage as resources in classes be-cause of the lack of classroom internet access.

Whatsbest&worst?

Arefacultygettingnewdigitalskills?


15/30

15

Howdidstudentsfareinthecollaborations?

First we compared thelearning environment scoresfor all of the grant classesand a comparable controlgroup in traditional classes.Significant differences in thelearning environments werefound for cooperation,cohesiveness, materialenvironment, goal direc-tion, and satisfactionbetween the experimental(content & tech classes) andthe control groups. TheF.I.P.S.E. classes had higher

cooperation, cohesiveness, andmaterial environment scoreswhich is what we would expectbecause of our emphasis on teams and cooperation and collaboration in the S.T.Y.L.E. model.Higher learning environment scores on these variables are indicative of higher achievement. Wecan use the LEI scores because we cant compare achievement scores directly in our variouspartnered situations. The material environment score probably reflects the use of lab and com-puter facilities in the experimental groups, while the control group met in a traditional class-room.

TechClassesvs. Control

Group

The tech classes out-comes were compared tocontrol group outcomes withresults similar to the compari-son of all experimental classesto control groups, except thatthere was no significantdifference between satisfac-tion. Cohesiveness and mate-rial environment perceptions

were higher for students in thetech classes, suggesting higherachievement of cognitiveoutcomes for the students inthese classes. This analysis isalso tentative because the controlgroup consisted of students in content classes, but no tech classes were included in the controlowing to some confusion about what tech classes might constitute comparable groups. In futureanalyses, the control group will include both content type classes and tech type classes.


16/30

16

LatinAmericanArt

Music&CultureCon-

tentClassvs. Control

Group

This class had a signifi-cantly higher cohesivenessscore than the controlgroup. These students wereprimarily Latino and in their

comments expressed pride that otherswere interested in their culture, which could added to the perceptions of cohesiveness whichwould be created through the team approach and group projects. The direction of this correla-tion suggests that achievement was higher for these students than for students in the controlclasses.

NutritionScienceContent

Classvs. ControlGroup

The analysis of results forthe Nutrition Science class reflectthe situation for that class. That itmet in a computer lab as well as atraditional classroom is reflectedin its higher scores for MaterialEnvironment. The sense of coop-eration was higher for this class,indicating a greater sense of teamwork and interdependence.

However, the teacher of classwas enthusiastic, but inexperi-

enced with the use of Internet asa teaching tool in class. Some of the teaching objectives for the Nutrition Science class began todrift, as the class spent time surfing the net, but without guidelines or a concrete set of tasks toaccomplish as a result of the surfing. The project director met with this class halfway throughthe course because of confusion about the nature and amount of work being required in theclass and things went better, but some student dissatisfaction remained.

These problems had not occurred in other collaboration settings because the contentteachers maintained greater focus on the outcomes and objectives for their own classes, and had


17/30

17

used Internet and other digital technology to streamline what they already did. Teachers usedInternet research, bulletin boards and e-mail to carry out their learning objectives. The NutritionScience teacher inadvertently got off-track in her class, focusing on the tools of digital andelectronic technology without guidance of learning objectives for a nutrition science class. TheNutrition Science teacher had assigned students to research nutrition on the Internet, almostexclusively, and because this was new territory, the parameters of what research entailed were

unclear to students, who were browsing and reading sites. In traditional research, studentsidentify relevant sources, read them, and then write or otherwise summarize, generalize, oranalyze what they have learned from their research. For effective use of Internet research asimilar process must be set up, as browsing and reading sites produces no evidence of theresearch or learning. The students felt they were working hard, but had little to show for theirwork, either to share with their collaborators, or to turn in for a grade. The dissatisfaction withthe class was expressed by students, and remediated somewhat in meetings with the teacherand project director during the semester, but it was expected that this class would have lessperceived satisfaction with the class than other experimental classes.

Conclu

sions

The project goals involving teamwork and collaboration appear to be succeeding, asstudent perceptions of cooperation and cohesiveness were greater in experimental than controlgroup classes. This suggests that cognitive learning may be greater in these classes than in thecontrol group, which is consistent with the theoretical underpinnings of the model for improv-ing education which is being tried out and evaluated.

Student satisfaction with the collaborative pairings does appear to be greater for experi-mental classes, except where the teacher got internet infatuation and strayed from her learn-ing objectives during the semester. Future partners will be told to be careful of this pitfall, andthe project director will monitor not only written course materials like the syllabus, but also the

initial class meetings of content teachers to assure the proper use of internet, etc. as tools, andnot as ends in themselves.

This practitioner research effort draws on what works to improve classroom learning. Ithelps faculty understand and use technology effectively as an educational tool. It provides aframework to guide the design of team-based assignments that is flexible, scalable, and adapt-able to a range of teaching styles. By reengineering the teaching actions (e.g. toward creating asense of team, toward allowing for diversity within an environment where we have clearlystated learning goals) teachers can use their experience and judgment to get there. The fear ofnot knowing exactly what the class will end up producing, and of giving up some teachercontrol is real, but the payback in terms of genuine student engagement and enjoyment oflearning, and the pleasure of collaboration with a fellow teacher in the team-based, collaborative

learning environment makes it worthwhile. There is danger in the hype and novelty of digitaland electronic technologies, and in losing ones pedogical way amidst the glitter of tools notserving teaching aims. However, the approach provides guidelines to keep the primary learningobjectives in focus as you new teaching tools. This approach begins to ...move to forms ofacademic organizations that require students to become actively involved with others in learn-ing. and to ... construct educational settings that promote shared, connected learning. (Tinto,1998)


18/30

18

HighlightsofF.I.P.S.E. AnnualProgressReport(1998-1999)andCurrent

SemestersProgress

The technical difficulties of sharing the work between the participating classes when wewere doing CD ROM-based projects in the first semester has caused us to move to the creation

of projects for the internet (http://digdesign.colum.edu/fipse) from the get-go. The work iseasier to share between the classes, and can thus be produced in a more interactive fashion.

During the first year of the project, we completed 3 collaborations. At the end of Fall 1999semester, the number will be six:

Womens History class and Interactive Programming for Business and Training class(website)

a Nutrition Science Class and an Interactive Multimedia Producer Class (website)

a Latin American Art, Music & Literature class and Interactive Programming for Busi-ness and Training class (website)

Latin American Art, Music, & Literature and New Media Projects

Psychology of Creativity and Interactive Multimedia Producer (websites)

Time-based Composing and Literary Collage

The completed projects and some works-in-progress are visible from: http://digdesign.colum.edu/fipse. The site is being renovated by a web consultant who will masteran interactive CD-ROM guide to the projects, information about the model and grant, andresources on teaching collaboratively and related topics.

As a result of the liberal arts/computing collaboration class, the College installed aninternet drop into one liberal arts classroom (NOTE:now all Lib arts classrooms are wiredthrough efforts of project director and Lib Arts chair.

We are completing the ordering and installation of 2 computers, projectors, and carts forthe liberal arts faculty to use in the classroom with the Internet drop. For the first time, thisdepartment has added equipment requests like this to their budget request

One of the non-technical teachers has agreed to design & deliver a presentation to herdepartment demonstrating how to use the Internet hook-up and the computer/projector on acart. There are currently 5 faculty who have expressed an interest in using this technology intheir classes

There are more collaborative projects for use as classroom resources for teachers atColumbia (Womens History part 1, a Nutrition Science site and a site to accompany the LatinAmerican Art, Music & Culture class.)

A collaboration between a Time-based composing class (digital movie production) and apoetry class called Literary Collage is in the works (NOTE: it will be completed this semester.)

Students and teachers are using e-mail to communicate with each other. Electronic


19/30


20/30

20

Jonassen, D.H. & Grabinger, R.S. (1992). Levels of processing in building expert sys-tems. In P. Kommers, D. Jonassen, & T. Mayes (Eds.), Cognitive tools for learning. Heidelberg,FRG: Springer-Verlag.

McLuhan, M. in Southam Interactive Understanding McLuhan: In the Electric WorldChange is the only Stable Factor. Voyager CD-ROM, 1996.

Schank , R. Engines for Education website, 1994 (http://www.ils.nwu.edu/~e_for_e/).

Schank, R.C. Goal-Based Scenarios. The Institute for the Learning Sciences, Northwest-ern University, Evanston, Illinois., Technical Report #36 December 1992.

Tinto, V. Colleges as Communities: Taking Research on Student Persistence Seriously. The Re-view of Higher Education 21.2 (1998) 167-177. URL: http://muse.jhu.edu/journals/review_of_higher_education/v021/21.2tinto.html

Walberg, H.J. (May 1984). Improving the Productivity of Americas Schools. Educational Leader-ship 41, 8: 19-27.


21/30

21

AppendixA

TheGrantProposal

The entire text of the original grant proposal is presented for your convenience in thefollowing sections of this guide to Columbias FIPSE grant.

I. ProjectNarrative

The project that Columbia College Chicago proposes will address three fundamentalproblems facing liberal arts colleges today. First, the problem of how to actively involve stu-dents in their own learning, and prepare them to make use of what they learn later in life isaddressed. Secondly, it addresses the question of how computers and digital technology can beused and integrated into the curriculum rather than just added on to existing course offerings.And finally, the question of how to provide faculty with an effective means of acquiring com-puter and digital skills without adding on to their current workload is addressed.

Pairing up traditional content-rich lecture / discussion classes with new technology-rich computer production classes to create collaborative interactive projects (websites and CD-ROMs) to supplement typical course work is an effective way to integrate digital technology intoclassrooms and to provide students with more opportunities for active learning through partici-pation in the projects. Faculty seeking to learn how to use and integrate computers and digitaltechnologies into their teaching can acquire new skills in a timely and cost effective manner byworking with another teacher and class on a collaborative digital project. By acquiring skills inimage capture, Internet use, and authoring, making knowledge maps (see Appendix A), andcomputer productivity while on the job, these teachers model how one learns to learn for their

students. The ability to retrain oneself and to adapt to new technologies may be novel, but it willbe a core skill in the next century. We will adopt this strategy to help our faculty optimize theirtime by teaching and learning concurrently, and encourage them to pass this skill on as part oftheir teaching repertoire.

The proposed project asks for funding for T.A.s to assist each of the teachers in the pairedclasses. Graduate students are not readily available at our institution owing to its focus primarilyon undergraduate studies. Therefore, the T.A.s will be undergraduate students chosen by the par-ticipating faculty because they have computer experience and demonstrate a degree of leadershipand initiative necessary for the kind of liaison work expected of them. They do not require specialtraining, as their main responsibilities will be in assisting with word processing and Internet useby students who are new to computing, and making sure that non-digital information can be

Iverson, B. K. Digital Technology & the Teaching of Womens History. presentedat TheStuff of Womens History: Using Artifacts, Landscapes & BuiltEnvironments to Research &Teach Womens History in the Classroom, Organization of American Historians & the NationalParkService. Seneca Falls, N. Y.,August 21, 1998 with Teresa Prados-Torreira.

Iverson, B. K. & Prados-Torreira, T. The making of warp & weft,might & magic, mettle &

motherhood: an electronic exploration of AmericanWomens History: 1600s to 1870s.SIGGRAPH98 Education Program,Orlando, Florida, July 1998. in SIGGRAPH98 ConferenceAbstracts and Applications, July, 1998.

Iverson, B. K. & Prados-Torreira, T. Integrating Womens History and CyberTechnology.Lecture & presentation at Womens INterVENTIONS in Science, Art, and Technology,FifthWomens Studies Symposium. Purdue University, March, 1998. URL:http://digesign.colum.edu/FIPSE/WHpaper.html


22/30

22

tive, train faculty in a cost effective, organizationally sound way, strengthen links between thegeneral and professional curricula, and actively involve undergraduate students in research. Themodel can be adapted to various projects so that schools can fit it to available technology and theskills of faculty on their campuses. At Columbia, the project will strengthen and modernize ourliberal and professional education courses which are largely lecture-based and utilize few com-puter-mediated communication tools.

A cost effective feature of this project enabled by the use of computer-mediated communi-cation including electronic mail (e-mail), Listserv technology, and the posting of shared materialson a website, is that the paired classes do not need to meet on the same day, at the same time, orin the same space, in order to collaborate. From any computer on campus, or from any remotecomputer with Internet browsing capability and access to e-mail, students can review each otherswork, suggest revisions, and discuss any aspect of the collaborative project. The project T.A.s canprovide help to students new to computer-mediated communication and can facilitate the ex-change of non-digital course-related material.

Traditional teamwork with its important social and interpersonal dimensions will occurface-to-face in the primary classrooms. The teamwork between the paired classes will occur elec-tronically and be nontraditional, in that students will be not be meeting in person. The faculty willintroduce this idea to students, and where schedules permit, students will have the opportunity tomeet in person at the beginning of the collaboration. The T.A.s and faculty will need to providesupport for and incorporate the information from this new, asynchronous channel of communica-tion into the discourse of the classroom so that it becomes integral to the overall project. Theformative evaluation of the project will provide feedback on how the students relate to their realteam members and to their virtual team members. A presentation session at the end of thesemester will allow the collaborators of the traditional and the virtual kind to meet in personaround a showing of the collaborative project.

With a grant from the American Council of Learned Societies (ACLS) we have been testing

out this model with Womens History classes, and a CD ROM Multimedia production class bycreating a website and related interactive CD-ROM. The pilot project will continue through thisfall. Based on our experience, we are refining the model for this kind of cross-disciplinary, collabo-rative, teaching/learning experiment.

We are requesting support from FIPSE in order to apply, evaluate, refine, and extend thismodel project to a range of classes spanning several departments in our college. The model isdesigned to change the pedagogy of lecture-based classes from passive to active learning environ-ments, to integrate computer-mediated communication technology (Internet, computer presenta-tion stations) into classrooms, and to provide faculty with cost effective, on-the-job computer train-ing. It will benefit our college, and in doing so, will serve as a model that can be easily adapted toother post-secondary institutions facing similar problems and looking for cost effective ways to

transform teaching, integrate computer technology and retrain their faculty without taking themout of their classrooms. The use of computer-mediated communication to conduct a collaborative,team-based project using virtual teams as well as real teams will be new to the proposed project.

The interdisciplinary nature of proposed collaboration between courses will help the col-lege integrate the use of digital technologies and computers throughout the curriculum. Participa-tion faculty will be learning digital skills as they teach, and will become increasingly familiar andable to integrate digital technology like the Internet, electronic presentations, and the incorpora-tion of digital images into their regular teaching. Also, faculty with newly acquired digital skills


23/30

23

can advise their departments regarding digital technology use. The incorporation of computer-mediated communication tools into classes to provide support for project-based learning outsidethe few hours of face-to-face contact time as a flexible extension of class activities can be adaptedto individual courses, as well as those that are paired up for interdisciplinary projects.

The active, constructivist orientation of the model classes stressing teamwork, perspec-

tive-taking in terms of communicating the results of research assignments, and the creation of aknowledge structure we call the knowledge map (see Appendix A) benefits students. Opportu-nities for sustained perspective-taking that come when students must prepare their research foran audience, and annotate and hyperlink it, foster development of higher-order reasoning skills.Research and technology skills transfer to other subjects and classes that students take. When theylearn by doing and think about their thinking, students create cognitive webs or nets thatpromote a deep and full understanding of a subject. Furthermore, because the students will becreating an educational piece, they have a sense of pride and ownership about their abilities as ascholar and their mastery of the topic. Their range of technical skills will be linked to the produc-tion of meaningful learning in their classes.

The Project Director will teach the technical classes involved in the model collaborations,and has the skills to help the students and participating faculty learn to work with digital technol-ogy. The classes that will participate in the project are taken from a range of disciplines, andrepresent the arts, communication, and general studies areas of the college. They are discussed ingreater detail in the next section of the paper.

II. Proposedstrategiesforimprovingcurrentpractices

This model provides the advantages of a relatively low-cost intervention, that will resultin changes in faculty knowledge and teaching strategies. Most faculty are familiar with team-based projects and project-based learning, though in some larger classes they may not have feltthis strategy was practical to implement. This model does not strain organizational structures ofthe college and relies on low-cost infrastructure improvements (computers on carts, projectiondevices, Internet drops where the wiring is already in place) in order to continue beyond thetime of the funding provided by the grant. It looks to the use of computer-mediated communi-cation tools like e-mail, listservs, and websites to provide students with opportunities to col-laborate in a flexible time frame. Faculty need to be shown how to use computer-mediatedinstruction as a means of communication for students both in and outside of the classroom. Thetraining builds on what they know from their personal use of computers for registration and e-mail, and does not require leave time or special training off campus.

The resources the students need in order to use computer-mediated communication arealready in place at our institution and most colleges across the countrycomputer labs with Inter-

net access. All our full-time faculty have computers in their offices with Internet access. The changesin pedagogy from passive to active learning incorporating a constructivist approach to subjectmatter, and the new skills faculty gain from their participation in the proposed grant are notdependent on continued funding from outside sources. We believe that the positive reactions ofstudents to classes where knowledge creation not passive knowledge acquisition is the focus, willreinforce the resolve of faculty to continue to use this method beyond their participation in thegrant. Once the faculty have used computer-mediated communication tools, they will not want torelinquish their use, and they need not, because of the ease of use and access to these tools. Fur-thermore, as this use of computers to communicate proliferates in the workplace, students and


24/30

24

teachers will view it as a natural part of the learning process, not an exotic add-on. Additionallyfaculty become knowledgeable advocates for what kinds of technology are appropriate for theirclasses and department and are less vulnerable to technological fads or bandwagons as they be-come active users of the technology.

The model will be described in general in this section. Its implementation in specific courses

and the differentiation of the model in those settings will be discussed in the next section. Ingeneral, technology-rich classes where students are learning web design or computer produc-tion skills must draw content from outside themselves. Unless the course objectives directly ad-dress instructional design, information engineering, storyboarding or other content creation meth-odologies, the use of content provided to the class from an external source creates a level playingfield as far as evaluating student acquisition of technical skills without controlling for their abilityto be content providers. In many work settings, technical producers must work with clients toproduce the clients vision, not their own, so this experience mirrors real-life experiences the stu-dent will encounter.

In this model, the content is prepared by the students who researched it, but in a formwhere it is ready to be made interactive with the use of media files (images, sound, etc.) The key tothis transformation is the exercise where students work in groups to produce knowledge mapswhich chart out hyperlinks and interactivity in their traditional research papers (see Appendix A.)The project director teaches the content class faculty to use the knowledge map to recast the re-search paper into this annotated, pre-electronic, linked guide for the content producers.

Funding for the faculty who participate in this grant serves as an incentive to meet withthe program director regularly, and to make the required changes to course syllabi to accommo-date the collaborative projects. In some cases, the funding will allow a teacher to have release timeto participate in this special project. In order to get ready for the semester, each teacher needs tomeet 4 to 5 times with the project director, and to exchange syllabi and other course materials (e.g.the topics and order in which they will be covered, required and supplemental readings, and

other resources.) Each syllabus must be altered somewhat to include the collaborative project andprovide for meeting times, computer lab sessions, and discussion of nontraditional assignments.Depending on the faculty members computer skills, the project director may schedule training inusing Internet or e-mail or other computer skills with the faculty before the semester begins. Asthe semester progresses, the faculty member attends the computer lab sessions with the studentsand gets the same training they do. The project director is available to help with any other technol-ogy concerns as the semester progresses.

During the semester, the teachers and project director need to regularly by e-mail andtelephone, and to make sure that any knowledge maps or other materials are being shared in atimely manner. The project director provides an introductory computer lab session for the stu-dents enrolled in the content course, to make sure they can do word processing, use the Internet,

and have an e-mail account. A follow-up session with the T.A.s is provided for students who needa bit of extra help. A series of no-cost workshops offered at Columbia on computer use in differentareas is open to students and faculty. If the content producer class needs to ask content-relatedquestions, the content teacher attends their class to clarify matters.

The content teacher visits and lectures in the technology class at least twice during thesemesterat the beginning to give an overview of the content area, and near the end to trouble-shoot and critique the works-in-progress. Works-in-progress will be evaluated by faculty teachingin relevant disciplines who are not currently part of the project, using a simple evaluation form


25/30

25

(see Appendix B.) This is will provide the students and teachers involved in the project with someconstructive criticism, and also provide a guide for use of the project during the disseminationphase of the project. Each project can be rated in terms of how it fits into the discipline it covers,and its degree of technical completeness. When the projects (CD-ROMS or websites) are com-pleted, the project director, the content teacher, and all the students present a showing of thework for the college community in order to publicize the project and to showcase their project.

Each semester, the faculty of the two classes will summarize their experiences and submit confer-ence proposals to talk about the project and to demonstrate the student projects to appropriateeducational and professional organizations.

Here is a description of the particular classes that will be involved in the pairing of classesduring the term of the grant (see Table 1). The technology classes involved will include a section ofDesigning for the Web, Programming for Interactive Business & Training Uses (formerlyCourseware & Learning.) or other similar classes. In each of these classes the emphasis is onaspects of technical production of digital media, so the use of external content providers can beaccommodated (and in some instances was built-into the course.)

The content-rich classes will include: Womens History I and II., each of which has beena part of a previous, similar grant from the American Council of Learned Societies (ACLS) forcollaborative interdisciplinary work. These classes are the pilot version for the FIPSE grant. Theobjectives for the Womens History class were altered in several ways to enable the collaboration.These alterations did not change the way the topics of Womens History were taught substan-tively, but altered some classroom activities and assignments. For example, the class was shiftedfrom a temporal view of the subject where some students did little writing until the 6th or 7thweek of class, to a theme-based approach which meant students produced content on a moreuniform basis over the semester. This was better for the tech classes and meant that the classescould meet to test and review the collaboration more frequently.

The Womens History students initially wrote two papers. In order to produce content for

the tech classes at a more uniform rate, the requirement was changed to 4 shorter papers, spreadout more equally over the semester. While this change was beneficial to the writing of the stu-dents, the history students as content providers did not get enough opportunity to review theinteractive interpretation of their work by the tech class with only four sessions of meeting to-gether. In the first iteration of this experiment, the tech students were simply showing theirwork to the content students. We redesigned the class so that the content provider students re-worked the material themselves so they would experience the perspective-taking involved ren-dering a work for an audience by creating a knowledge map of their topic (see Appendix A).Additionally, we designed ways for the student writers to have the opportunity to try out thepieces, and suggest revisions to the students who were authoring the interactive pieces on an on-going basis through the use of Internet and e-mail.

With the proposed FIPSE grant, classes involved in the model collaboration project willinclude: Womens History I and II, Nutrition (Science & Math Department), History of AmericanSign Language (Interpreter Training), Latin American Literature, Art & Music (Humanities) andFashion (Fashion Management.) In the Nutrition class, the teacher wants to use certain profes-sionally produced CD ROMs in her lectures. The project director can facilitate this. The collabora-tion project will focus on nutrition knowledge and present public service type web pages aboutgood nutrition for the college community. The Interpreter Training Program (one of the only suchprograms in Illinois) will have its introductory survey course, History of American Sign Lan-


26/30

26

guage (ASL) participate. Students do a research paper in this class, and will adapt their researchfor an audience of laymen to help educate the public about ASL and issues of importance to thisfield. Segments of ASL can be animated as part of this effort.

The Latin American Literature, Music, and Art class currently does one traditional researchassignment and one group project, emphasizing performance. The class objectives already in-

clude items about teamwork and preparing content for an audience. The projects will be enhancedby having students consider how to show highlights of their projects in a context that will explainthem in the interactive digital collaboration piece.

Our college has a strong concentration in Fashion and an additional course that will beinvolved in this grant is a fashion course. Students in this class will transform their drawings andideas about the future of fashion into knowledge maps which will be produced collaboratively bythe tech classes. The team project aspect can include a virtual fashion show which would beinteractive so that viewers look at patterns, drawing, and information about fabrics in the collabo-rative multimedia piece. The area of fashion and rendering fashions for virtual humans is ofgrowing interest in several professions.

For the classes involved in this series of collaborative efforts, there are questions of bal-ancebetween the traditional course content and the changes made to allow for the collaborativeproject and between the importance process and product. The technical classes have a pressfor completion of a polished product, but there is tension in working with student content provid-ers. The quality of the pieces will vary with the student producers, however, issues of visualliteracy, production techniques, and aesthetics are addressed and students learn from near missesas well as from their successes. However, in all cases, the products (CDs or websites) can becomeresources for college. Thus, the CD ROM or website product of the Womens History/CD ROMMultimedia Production class is a resource for the next CD ROM and Womens History classes andfor other production classes, as well as for American History which includes some readings andstudy of the role of women in American History. Similarly, the Sign Language website can become

a resource for the interpreter training class, or for any class which might be considering questionsregarding equity and disabilities. The Nutrition website can be a resource for student groups, andother science and health classes.

It is important to remember all the projects are created by undergraduates in a singlesemester. They may be revised, corrected, and updated regularly by classes in subsequent semes-ters. Much more with than commercially made and produced software and classroom resources,the hallmark of these products is that students are knowledge producers, not simply knowledgeconsumers. There is always the incentive to improve the work, the challenge to take a shot at ityourself, that is missing when students use commercially prepared materials which are profes-sionally edited, finished, and complete. The evaluation of the class products by faculty outside ofthe project will provide information for users within and outside our institution about the nature

and completeness of the CD ROM or website resources produced by for this grant.

This aspect of learning, wherein students are empowered to learn because they want tocommunicate their ideas is powerful and stands in a central role in the transformation of theclassroom from passive to active, with a focus on constructivism. While the proposed interven-tions are not overly complex, nor costly, they are revolutionary, in that the teachers involved be-come active lobbyists for the kinds of technology which will make their classrooms more effectivelearning environments, and students themselves are the active authors of curricular reform and ofclassroom materials. The essence of digital information is that it is assembled rather than looked


27/30

27

up. Being able to put together relevant information, make sense of it, and to communicate aboutit is a central task for the worker, thinker, and citizen of the next century. Students who participatein the proposed model classes practice this, and educational research shows we learn what wepractice.

The payoffs of the transformed classrooms need to be documented. The details of this are

laid out in the evaluation section of this paper, however, it should be noted that dimensions ofcooperative learning environments which will result from this model can be measured. Changesin perceived levels of cooperation can be correlated with achievement gains. This is exciting, be-cause without such an empirically validated measure of effectiveness, it would be difficult tocompare outcomes across such a range of classes with such diverse learning outcomes. In mostcases, there will be comparable classes to use as comparison groups, so that changes in the targetclasses can be more accurately attributed to aspects of the proposed strategies to improve post-secondary instruction.

III. Institutionalcommitment--contributionoftheCollegetotheproject

Columbia College has much of the equipment needed to begin, carry out, and completethese kind of productions, including computer labs, a CD ROM press, and a capable faculty whoare already teaching such classes as Elements of Multimedia, Multimedia Production and CDROM Production. There is growing interest among faculty across departments in learning to useand integrate of digital technologies into the fabric of the classroom. This project will provide theneeded structure within which the faculty collaboration can take place and resources can be usedeffectively. The series of projects produced in the classesthe CD ROMs and websites them-selvesare an additional educational resource and will benefit students and faculty by emphasiz-ing teamwork and project-based learning and providing them with cutting edge digital technol-ogy skills.

We will be working to have the college create smart classrooms for participating subject-matter classes where instructors will have access to an Internet connection, a computer and apresentation device. Currently these classrooms have no computer, projection device, nor Internetdrop and this means the logistics of demonstrating software or showing the work-in-progressprojects are difficult to manage. We hope to have at least one smart classroom by Fall 1998.

IV. EvaluationandDissemination

This section describes the evaluation and dissemination plan for Integrating ElectronicTechnology in Liberal and Professional Education Classrooms. This project has three mainobjectives: to promote active student learning, to integrate computers into the classroom, and toimprove faculty computer and digital skills. The purpose of this evaluation is to assess whether

project goals are met in a cost effective manner. The use of controlled comparisons will helpreveal the impact of the project, disseminate findings, and expand the project to other areas.

This evaluation has two main parts. The formative evaluation is designed to improve imple-mentation and development and reform project activities. The summative evaluation assessescost effectiveness, the fulfillment of project objectives, and project efficacy. Both the formative andsummative evaluations may address administrative planning and policy making, reveal unin-tended consequences, inform the allocation of resources, and identify problems and/or costs.


28/30

28

The formative evaluation is designed to assess quickly and accurately the strengths andweaknesses of the project and provide suggestions for improvement. It is important to capitalizeon the strengths, or address weaknesses, during the early stages of the project to improve devel-opment and implementation. Furthermore, the results of the formative evaluation should be dis-seminated in a timely manner so appropriate adjustments can be made to improve the project bythe Project Director and participating faculty members.

This plan facilitates a timely dissemination of formative evaluation information. The useof electronic technology will be employed for fast and accurate results. The evaluator will be pro-vided with the e-mail addresses for the teachers and students after the first week of classes. Dur-ing the third week of classes the evaluator will e-mail the Evaluation Questionnaire (see AppendixC) to all participating teachers and students. They will have one week to e-mail their responsesback to the evaluator. The evaluator will write a clear and concise formative evaluation based onthe input of the teachers and students. The appendices for these reports will include completetranscripts of the responses for further review. The reports will be sent to the Project Director forfacilitating changes.

The Evaluation Questionnaire is simple and effective. The items are open-ended to allowfor freedom of response. The respondents are asked to state the goals of the project. Participantknowledge about the project objectives is important to promote learning and understanding. Theparticipants are also asked to cite the best and worst things about the project. Obviously, the ProjectDirector could try to enhance or continue the best things while reducing or eliminating the worst.Finally, the respondents are invited to suggest changes. This is an ideal way to identify ways toimprove the project for its consumers.

The summative evaluation, the second part of this plan, has three parts: student question-naires, teacher interviews, and project product evaluations. Initially, the Student Evaluation Form(Appendix D) will be administered at during the last week of classes to the students participatingin the project and, where possible, to students in comparable classes outside the project. This

controlled comparison will identify the strengths and weaknesses of the project.The questionnaire is composed of six scales from the Learning Environment Inventory

(LEI). The LEI measures classroom social climate which research has shown to be one of the keypsychological factors in academic learning (Walberg 1994). If cooperative learning can be enhanced,for example, achievement gains are likely. The LEI consists of Likert scale items with the followingresponses: Strongly Disagree, Disagree, Agree, and Strongly Agree. The scales have been empiri-cally proven to be both valid and reliable (Fraser, Anderson, Walberg, 1991).

Six scales were selected to measure the attainment of the project objectives. Keeping theindividual scales intact preserves their reliability. The scales include: cohesiveness, diversity, ma-terial environment, goal direction, democracy, and satisfaction. Cohesiveness refers to how favor-

ably students see themselves and their work as a group. Diversity reflects whether students withdifferent abilities and interests are encouraged. The material environment refers to availabilityappropriate of books, equipment, and space. Goal direction assesses whether the students knowwhat they are doing. Democracy measures the perceived student input in classroom decision-making. Finally, satisfaction gauges whether the students enjoy the class. These scales were se-lected to assess the fulfillment of the project objectives to improve cooperative and active learning.

The LEI will be administered to the students electronically. The evaluator will e-mail theStudent Evaluation Form to the students during the last week of class. The students will have one


29/30


30/30

similar knowledge construction processes.

For other post-secondary educational institutions, the emphasis in this project on inter-ventions which do not require huge capital outlays, nor profound organizational upheaval, butrely on cost-effective retraining of faculty , the adoption of simple but effective teaching strategies(team-based project assignments) and on the use of resources which are widely available in post-

secondary institutions will be of interest, and through participation in FIPSE sponsored confer-ences, regional and other national conferences, and in publications in academic ( Journal of Edu-cational Research. Educational Technology, etc.) and trade-type journals (SYLLABUS, THE Jour-nal, etc.) the project and its findings will be disseminated.

The detailed and systematic nature of the both the formative and summative evaluation ofthis project will present an important resource in and of itself. Research on the learning impact ofinteractive multimedia materials is sparse. This study, through its use of the LEI which is anestablished tool in educational research and helps provide a way to link changes in student atti-tude and perception of classroom climate to expected changes in intellectual and academic indi-cators, will provide a way of quantifying the effects of collaborative, interdisciplinary digital projectson measures of learning outcomes. The findings of this longitudinal study which will incorporateclasses from a wide range of disciplines, and the provide a first attempt at the evaluation of theeffect of computer-mediated communication tools on classroom climate, and thus student learn-ing outcomes, will be of interest to post-secondary educators in a wide variety of institutions andsettings.

FIPSE Guide

Documents

Transcript of FIPSE Guide