Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and...
-
Upload
frankie-dearborn -
Category
Documents
-
view
216 -
download
1
Transcript of Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and...
![Page 1: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/1.jpg)
Teaching for Fluency with Teaching for Fluency with Information Technology:Information Technology:The Role of Feedback in The Role of Feedback in Instructional Design and Instructional Design and Student AssessmentStudent Assessment
Explorations in Instructional Technology
Mark Urban-Lurain
Don Weinshank
October 27, 2000
www.cse.msu.edu/~cse101
![Page 2: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/2.jpg)
OverviewOverview
Context: Teaching non-CS majors Instructional Design Uses of technology Results Implications
![Page 3: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/3.jpg)
Fluency with Information Fluency with Information TechnologyTechnology
What does it mean to be a literate college graduate in an information age?
Information technology is ubiquitous “Computer Literacy” associated with training Being Fluent with Information Technology (FIT)
Committee on Information Technology Literacy, 1999
CSE 101 MSU introductory course for non-CS majors
![Page 4: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/4.jpg)
Instructional Principles Instructional Principles 1. Concepts and principles promote transfer within domain
Necessary for solving new problems
2. “Assessment drives instruction” (Yelon) “Write the final exam first”
3. Move focus from what is taught to what is learned Student-centered
4. Formative evaluation improves student performance Study – test – restudy – retest
5. Performance assessments evaluate mastery of concepts High inter-rater reliability critical
6. Mastery-model learning ensures objectives met What students can do, not what they can say
![Page 5: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/5.jpg)
Uses of Technology in Uses of Technology in InstructionInstruction
Delivery of content Television CBI / CBT Web-based
Communication E-mail Discussion groups Real-time chat
Feedback and monitoring Formative evaluation Iterative course development and improvement
![Page 6: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/6.jpg)
`Incoming Students
Instruction
Assessment
Outcomes
Design Inputs
Instructional Goals
Instructional Design
Design Phase
Implementation Phase
Course Design & ImplementationCourse Design & Implementation
![Page 7: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/7.jpg)
Discriminant AnalysisDiscriminant Analysis
Multivariate statistical classification procedure Dependent variable: final course grade Independent variables
Incoming student data Classroom / instructional data Assessment performance data
Each student classified in group with highest probability Evaluate classification accuracy
Interpret discriminant functions Independent variable correlations with functions Similar to interpreting loadings in Factor Analysis
![Page 8: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/8.jpg)
3
6
7
5
4
2
FootnoteModify styleWeb formatPrivate folder
New SSChartsFunctions
Path
Find application
TOC
URL
Public folder
Update SS
Boolean search
Skill to Schema MapSkill to Schema Map
LinkWeb search
Find rename file
Computer specs.
![Page 9: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/9.jpg)
SIRS: Course, TA, ATASIRS: Course, TA, ATA
End of semester student survey about course Three Factors
“Fairness” Student preparation and participation Course resources
SIRS for Lead TA One Factor
SIRS for Assistant TA One Factor
![Page 10: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/10.jpg)
Fairness FactorFairness Factor
35.3% of variance on this factor accounted for by: Final grade in course TA SIRS Number of BT attempts ATA SIRS Cumulative GPA ACT Mathematics Computer specifications Incoming computer communication experience
![Page 11: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/11.jpg)
Participation FactorParticipation Factor
19.8% of variance on this factor accounted for by: TA SIRS Attendance ATA SIRS ACT Social Science Number of BT attempts Create chart Incoming knowledge of computer terms ACT Mathematics Find - rename file Path TOC
NO course grade
![Page 12: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/12.jpg)
Course Resources FactorCourse Resources Factor
11.3 % of variance on this factor accounted for by: TA SIRS Attendance ATA SIRS Extension task: backgrounds Web pages in Web folder Number of BT attempts
NO course grade
![Page 13: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/13.jpg)
Lead TA SIRSLead TA SIRS
27.8 % of variance on Lead TA SIRS accounted for by: Fairness factor Preparation and participation factor TA Experience Course resources factor ATA SIRS Attendance Private folder Extension task: excel function Number of BT attempts
NO course grade
![Page 14: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/14.jpg)
Assistant TA SIRSAssistant TA SIRS
13.4 % of variance on ATA SIRS accounted for by: Fairness factor Preparation and participation factor Student E-mail factor TA SIRS Course resources factor Attendance Path TA Experience
NO course grade
![Page 15: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/15.jpg)
Technology in Instructional Technology in Instructional Design and Student AssessmentDesign and Student Assessment
Data-rich instructional system Detailed information about each student CQI for all aspects of instructional system
Performance-based assessments Labor intensive Inter-rater reliability Analyzing student conceptual frameworks
Intervention strategies Early identification Targeted for schematic structures
![Page 16: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/16.jpg)
ImplicationsImplications
Instructional design process can be used in any discipline Accreditation Board for Engineering and Technology CQI
Distance Education Demonstrates instruction at needed scale On-line assessments How to provide active, constructivist learning on line?
![Page 17: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/17.jpg)
Questions?Questions?
CSE 101 Web site
www.cse.msu.edu/~cse101
![Page 18: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/18.jpg)
![Page 19: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/19.jpg)
![Page 20: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/20.jpg)
Instructional design detail slidesInstructional design detail slides
![Page 21: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/21.jpg)
Design Inputs• Literature
• CS0• Learning• Assessment
• Client department needs• Design team experience
Design InputsDesign Inputs
![Page 22: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/22.jpg)
![Page 23: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/23.jpg)
Instructional Goals• FITness• Problem solving• Transfer• Retention• No programming
Instructional GoalsInstructional Goals
![Page 24: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/24.jpg)
Deductive InstructionDeductive Instruction
Concept
Skill 1 Skill 2
Skill 3
![Page 25: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/25.jpg)
Schema 1 Schema 2
Schema 3
Inductive InstructionInductive Instruction
Skill 1
Skill 3
Concept
Skill 2
![Page 26: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/26.jpg)
![Page 27: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/27.jpg)
Instructional Design• 1950 students / semester• Multiple “tracks”
• Common first half• Diverge for focal problems
• All lab-based classes• 65 sections• No lectures
• Problem-based, collaborative learning• Performance-based assessments
Instructional DesignInstructional Design
![Page 28: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/28.jpg)
![Page 29: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/29.jpg)
Incoming Students• Undergraduates in non-technical majors
• GPA• ACT scores• Class standing• Major• Gender• Ethnicity• Computing experience
Incoming StudentsIncoming Students
![Page 30: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/30.jpg)
![Page 31: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/31.jpg)
Instruction• Classroom staff
• Lead Teaching Assistant• Assistant Teaching Assistant
• Lesson plans• Problem-based learning
• Series of exercises• Homework• Instructional resources
• Web, Textbook
InstructionInstruction
![Page 32: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/32.jpg)
![Page 33: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/33.jpg)
Assessment• Performance-based• Modified mastery model• Bridge Tasks
• Determine grade through 3.0• Formative• Summative
• Final project• May increase 3.0 to 3.5 or 4.0
AssessmentAssessment
![Page 34: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/34.jpg)
Bridge Task Competencies Bridge Task Competencies in CSE 101in CSE 101
1.0 E-mail; Web; Distributed network file systems; Help 1.5 Bibliographic databases; Creating Web pages 2.0 Advanced Word-processing 2.5 Spreadsheets (functions, charts); Hardware; Software 3.0Track A
Advanced Web site creation; Java Applets; Object embedding
3.0 Track C Advanced spreadsheets; Importing; Data analysis; Add-on tools
3.0 Track D Advanced spreadsheets; Fiscal analysis; Add-on tools
![Page 35: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/35.jpg)
![Page 36: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/36.jpg)
Bridge Task Detail Drilldown 1Bridge Task Detail Drilldown 1
![Page 37: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/37.jpg)
BridgeTask (BT)Database
• Each Bridge Task (BT) has dimensions (M) that define
the skills and concepts being evaluated.
• Within each dimension are some number of instances
(n) of text describing tasks for that dimension.
• A bridge task consists of one randomly selected
instance from each dimension for that bridge task
Dim 1Instance i
Instance i+1Instance i+2Instance i+n
Dim 2Instance i
Instance i+1Instance i+2Instance i+n
Dim MInstance i
Instance i+1Instance i+2Instance i+n
Creating AssessmentsCreating Assessments
![Page 38: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/38.jpg)
BridgeTask (BT)Database
Dim 1Instance i
Instance i+1Instance i+2Instance i+n
Dim 2Instance i
Instance i+1Instance i+2Instance i+n
Dim MInstance i
Instance i+1Instance i+2Instance i+n
Dim 1Instance 1
Criteria iCriteria i+1Criteria i+2Criteria i+n
Dim 2Instance i+2
Criteria iCriteria i+1Criteria i+2Criteria i+n
Dim MInstance i+n
Criteria iCriteria i+1Criteria i+2Criteria i+n
Student EvaluationPASS or FAIL
Evaluation CriteriaEvaluation Criteria
![Page 39: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/39.jpg)
![Page 40: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/40.jpg)
Bridge Task Detail Drilldown 2Bridge Task Detail Drilldown 2
![Page 41: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/41.jpg)
Web Server
Student Enters: Pilot ID
PIDPW
StudentRecordsDatabase
Submits to Create Query
BridgeTask (BT)Database
RequestNew BT
Dim 1Instance i
Instance i+1Instance i+2Instance i+n
Dim 2Instance i
Instance i+1Instance i+2Instance i+n
Dim MInstance i
Instance i+1Instance i+2Instance i+n
Randomly select one instance from each of M dimensions for desired BT
Assemble Text
Dim 1 (i+1)Dim 2 (i+n)Dim M (i)
Web Server
IndividualStudent
BT Web PageReturns
Delivering AssessmentsDelivering Assessments
![Page 42: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/42.jpg)
![Page 43: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/43.jpg)
StudentRecordsDatabase
Grader Queuing Create Query
BridgeTask (BT)Database
RequestCriteria
Dim 1Criteria i
Criteria i+1Criteria i+2Criteria i+n
Dim 2Criteria i
Criteria i+1Criteria i+2Criteria i+n
Dim MCriteria i
Criteria i+1Criteria i+2Criteria i+n
Provide criteria for instances used toconstruct student’s BT
IndividualStudent
BT ChecklistReturns
Grader evaluateseach criteria
PASS or FAIL
StudentBridge
Task
Dim
1
Cri
teri
a
Dim
2
Cri
teri
a
Dim
M C
rite
ria
StudentRecordsDatabase
Record PASS / FAIL for each criteria
Evaluating AssessmentsEvaluating Assessments
![Page 44: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/44.jpg)
![Page 45: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/45.jpg)
![Page 46: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/46.jpg)
OutcomesOutcomes
Outcomes• Student final grades• SIRS• TA feedback• Oversight Committee: Associate Deans• Client department feedback
![Page 47: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/47.jpg)
![Page 48: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/48.jpg)
![Page 49: Teaching for Fluency with Information Technology: The Role of Feedback in Instructional Design and Student Assessment Explorations in Instructional Technology.](https://reader036.fdocuments.in/reader036/viewer/2022062417/55173555550346fe558b5eea/html5/thumbnails/49.jpg)