TSDS Pilot – Lubbock ISD Train the Trainer September 26, 2011.
-
Upload
douglas-parker -
Category
Documents
-
view
214 -
download
0
Transcript of TSDS Pilot – Lubbock ISD Train the Trainer September 26, 2011.
Texas Student Data System n Lubbock ISD n 2©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Expected Learning Outcomes
Gain an understanding of the goals of the Texas Student Data System dashboard project
Understand your role and expectations as a trainer Be able to access, log-in, and navigate the
dashboards Engage in hands on activities to understand student
data presented in dashboards Know how to contact customer support and access
materials needed for training
Texas Student Data System n Lubbock ISD n 3©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Today’s Agenda
Topic Timing
Background - TSDS Dashboards 5 min
Your Roles as Trainers 10 min
Review Dashboard Training Module 2 hrs
Data Inquiry Framework 10 min
Going Forward – Rollout Plan, Trainer Support, and Feedback 10 min
Q&A 20 min
Total 3 hrs
Texas Student Data System n Lubbock ISD n 4©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Texas Student Data System (TSDS) Vision
A data system for teachers, designed by teachers.
Delivers relevant, timely and actionable student data back to educators to continually improve performance
A “One stop shop” for student data – brings together student information from multiple sources
Reduces reporting and collection burden to districts
Requires no additional data input
User friendly and intuitive and accessible from any location
Available free of charge to all Texas districts
Enable 100% of educators to have access to timely, relevant, actionable data to drive classroom and student success
Texas Student Data System n Lubbock ISD n 5©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards are Grounded in Best Practices and Vetted by Educators
Nationwide Best Practices Audit
Academic Research
Focus Groups with 2,600 Educators
• Initial dashboards based on national education research and review of best practices across the country
• Received and incorporated feedback on dashboard from 2,600 educators in Texas
• Enhancements to dashboards based on stakeholder feedback
Texas Student Data System n Lubbock ISD n 6©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards Consist of Four Levels Driven by Individual Student Data
District/LEA
Campus
Classroom
Student
Drill down to the individual
Student level from any
aggregate view
Data originates at the Student
Level and rolls up to aggregate
views
Texas Student Data System n Lubbock ISD n 7©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards Piloted in Six Districts to get Input and Inform Statewide Release
20142013201220112010
Initial DCD prototype
Draft data standard defined
Field test and acceptance of data standard
State-Sponsored SIS requirements
released
District Connections
Database Implemented
Transfer tested data standard
Late-2009Draft data
snapshots defined
Stakeholder engagement
Snapshots finalized based on feedback
Integrate SSIS and
DCD
Limited Product Release PhaseStatewide implementation of
DCD and dashboards
Texas Student Data System n Lubbock ISD n 8©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Today’s Agenda
Topic Timing
Background - TSDS Dashboards 5 min
Your Roles as Trainers 10 min
Review Dashboard Training Module 2 hrs
Data Inquiry Framework 10 min
Going Forward – Rollout Plan, Trainer Support, and Feedback 10 min
Q&A 20 min
Total 3 hrs
Texas Student Data System n Lubbock ISD n 9©2011 Michael & Susan Dell Foundation. All Rights Reserved.
After Today’s Session, You Should Be Able To …
Understand the scope and functionality of the Dashboards
Navigate and leverage the Dashboards to understand the needs of your students and campus
Provide introductory training to Teachers and Administrators at your campus
Know how to direct people to the proper support channels
Texas Student Data System n Lubbock ISD n 10©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Training Best Practices
Train in small groups – 15 people maximum
If possible, group into functional areas – e.g., administrators, teachers, or subject area
Ensure everyone has access to a computer (e.g., computer lab, laptops)
Review the pre-training checklist to ensure no technology issues arise
Review the training materials and video shortly before the training session- Log into and get comfortable with TSDS- Watch the demo video and review the FAQs so you can respond to users- Review the Classroom, Student and Campus exercises and select the examples
you plan to highlight from your campus and classroom
Use the training materials during the session
Capture questions you can’t answer and work with the data steward to ensure there is follow-up with the users
Make sure participants complete the feedback forms - during session or send email with URL following session: https://www.surveymonkey.com/s/LubbockQ2Users
Let us know if there are any other tools or materials that can help you!
Texas Student Data System n Lubbock ISD n 11©2011 Michael & Susan Dell Foundation. All Rights Reserved.
We’re Counting On You!
Ultimately, the Dashboards should save educators time and result in better outcomes for Students
Without You, Training will not take place
Without Training, Dashboard benefits will be limited
Your valuable feedback will ensure training tools will meet the needs of districts statewide
Texas Student Data System n Lubbock ISD n 12©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Today’s Agenda
Topic Timing
Background - TSDS Dashboards 5 min
Your Roles as Trainers 10 min
Review Dashboard Training Module 2 hrs
Data Inquiry Framework 10 min
Going Forward – Rollout Plan, Trainer Support, and Feedback 10 min
Q&A 20 min
Total 3 hrs
Texas Student Data System n Lubbock ISD n 13©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Goals of Reviewing the Training Module
Help you learn to use the system
Build your familiarity with training content
Build your knowledge base so you can answer questions that might be asked of YOU
Texas Student Data System n Lubbock ISD n 14©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboard Training Module
The training materials are designed for a 1 hour training session
Today, we will spend more time today to familiarize you with the system and prepare to deliver training
Topic Today End-User Training
Background - TSDS Dashboards 5 min 5 min
Student Dashboard - Functionality Overview and Exercises- Real Time Inquiry
45 min 30 min
Campus Dashboard - Functionality Overview and Exercises- Real Time Inquiry
30 min 15 min
Creating a Data Inquiry Mindset / Best Practices 20 min 5 min
Going Forward – Support and Feedback, Q&A 20 min 5 min
Total 2 hours 1 hour
Texas Student Data System n Lubbock ISD n 16©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Expected Learning Outcomes
Gain an understanding of the goals of the Texas Student Data System dashboard project
Be able to access, log-in, and navigate the dashboards
Engage in hands on activities to understand student data presented in dashboards
Start to create action plans for your students or your campus based on the dashboard data
Know how to contact customer support
Texas Student Data System n Lubbock ISD n 17©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboard Training Module Outline
Topic Timing
Background - TSDS Dashboards 5 min
Student Dashboard - Functionality Overview and Exercises- Real Time Inquiry
30 min
Campus Dashboard - Functionality Overview and Exercises- Real Time Inquiry
15 min
Creating a Data Inquiry Mindset / Best Practices 5 min
Going Forward –Support and Feedback 5 min
Total 1 hour
Texas Student Data System n Lubbock ISD n 18©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards are Grounded in Best Practices and Vetted by Educators
Nationwide Best Practices Audit
Academic Research
Focus Groups with 2,600 Educators
• Initial dashboards based on national education research and review of best practices across the country
• Received and incorporated feedback on dashboard from 2,600 educators in Texas
• Enhancements to dashboards based on stakeholder feedback
• Your feedback in this pilot will drive further enhancements to dashboards
Texas Student Data System n Lubbock ISD n 19©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards are Targeted Toward Improving Student Outcomes
Positive Interactive Involvement with Students
More Teaching Time
Informed Decision Making
User Friendly Data System
“After the information has been gathered, hopefully I would have a user-friendly service that would allow me to look at the information, serve my students and meet their needs more effectively and efficiently.”
Texas Student Data System n Lubbock ISD n 20©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards Consist of Four Levels Driven by Individual Student Data
District/LEA
Campus
Classroom
Student
Drill down to the individual
Student level from any
aggregate view
Data originates at the Student
Level and rolls up to aggregate
views
Texas Student Data System n Lubbock ISD n 21©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboard Training Module Outline
Topic Timing
Background - TSDS Dashboards 5 min
Student Dashboard - Functionality Overview and Exercises- Real Time Inquiry
30 min
Campus Dashboard- Functionality Overview and Exercises- Real Time Inquiry
20 min
Best Practices, Support and Feedback 5 min
Total 1 hr
Demonstration of the DashboardsStudent and Classroom
https://www.districtconnections.org/LubbockISD/Docs/ Training Materials-Training Demo Video
Texas Student Data System n Lubbock ISD n 23©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Exercises - Classroom and Student
Log-in and let’s go through some examples together:
1. Classroom: go to the Subject-Specific view and sort your students based on benchmark scores or the Assessment page and sort by TAKS objectives. Traverse this list and identify groups of students for further instruction based on strengths and areas for growth.
2. Classroom: identify a student having attendance problems. Navigate to their student dashboards. Are the absences mostly excused or unexcused? Is the attendance trend improving or declining? Find their parent’s contact information so you can call to discuss your concerns.
3. Student: Use the search function to search for a student that you suspect has not performed well on their last benchmark assessment
4. Student: For the student you suspect having low benchmark assessment scores in your subject area of interest, determine the trend of their benchmark assessment scores and which TEK items they struggled with on one of their past benchmark administrations.
Explore - Spend 10 - 15 minutes of real time inquiry on your own
Texas Student Data System n Lubbock ISD n 24©2011 Michael & Susan Dell Foundation. All Rights Reserved.
During the Real Time Inquiry Please Complete the Personal Action Plan
Peruse the system looking at YOUR students and use the Action Plan to record your observations.• Inquire – What do I want to
know? What jumps out?• Acquire – Find the data on
the Dashboard• Analyze – Test your
hypothesis by driving into detail
• Share – We’ll share some examples together
Texas Student Data System n Lubbock ISD n 25©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboard Training Module Outline
Topic Timing
Background - TSDS Dashboards 5 min
Student Dashboard - Functionality Overview and Exercises- Real Time Inquiry
30 min
Campus Dashboard- Functionality Overview and Exercises- Real Time Inquiry
20 min
Best Practices, Support and Feedback 5 min
Total 1 hr
Demonstration of the DashboardsCampus View
https://www.districtconnections.org/LubbockISD/Docs/ Training Materials-Training Demo Video
Texas Student Data System n Lubbock ISD n 27©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Exercises - Campus
Log-in and let’s go through some examples together:
1. All: Go the Advanced Academics page and look at the TAKS Commended performance. Use the More…Historical drill down feature to determine the change in your campus’ TAKS performance from last year to this year.
2. All: Go to the Attendance & Discipline page and look at either the Daily or Class Period measures. Which grade has the lowest daily attendance? Which students have the lowest daily attendance?
3. All: Let’s use the search function to navigate to a particular student or teacher (campus administrators).
4a. Teachers: find the names of 5 students who scored the lowest on their last TAKS.
4b. Administrators: Go to the Operational Dashboard page and look at teacher attendance. Which teachers have a high absence rate. Click a teacher to see if their absences are affecting classroom performance.
Explore - Spend 10 minutes of real time inquiry on your own
Texas Student Data System n Lubbock ISD n 28©2011 Michael & Susan Dell Foundation. All Rights Reserved.
During the Real Time Inquiry Please Complete the Personal Action Plan
How will you use the Texas Student Data System to improve your campus?
Record your observations using the TSDS Personal Action Plan
Prepare to share an example with the group
Texas Student Data System n Lubbock ISD n 29©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboard Training Module Outline
Topic Timing
Background - TSDS Dashboards 5 min
Student Dashboard - Functionality Overview and Examples- Real Time Inquiry
30 min
Campus Dashboard- Functionality Overview and Examples- Real Time Inquiry
20 min
Best Practices, Support and Feedback 5 min
Total 1 hr
Texas Student Data System n Lubbock ISD n 30©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Best Practices - Ways to Use Dashboards
There are several critical times to use the dashboards
- At the beginning of the school year or a new grading period
- After the results of a benchmark assessment are returned
- To identify students for intervention or to assess intervention effectiveness
- When you observe problems in your classroom or with a specific student
- In preparation for parent – teacher conferences
You can begin exploring the dashboards in several different ways
- Use hypothesis-driven analysis, e.g., “I think A and B are causing Jeffry’s performance issues … let me test that hypothesis with real data”
- Look for trends and alerts
- Look for groupings of students who have similar results
- Just explore!
Catch students early before problems become large and pervasive
Print dashboards and share with parents and students
Texas Student Data System n Lubbock ISD n 31©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Several customer support resources
Websitehttps://
www.districtconnections.org/LubbockISD/
Dashboard Support Form
1 2 3 District Data Steward
(Customer Support)
Texas Student Data System n Lubbock ISD n 32©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Pro’s• Access to TSDS before others• Opportunity to provide input
and guide development
Con’s• May experience occasional
system glitches• Some data may appear
questionable
Feedback - Your Charge: Beta Test!
Beta Testing means …
- Accessing real data about your students, updated nightly
- Testing and perfecting the system before it is rolled out to the entire state
Texas Student Data System n Lubbock ISD n 33©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Workshop Feedback Form
Providing feedback will help future training sessions
Complete the survey online - https://www.surveymonkey.com/s/LubbockQ2Users
It will take less than 5 minutes to complete the survey
Texas Student Data System n Lubbock ISD n 35©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Trainer References and FAQ
The dashboard FAQs summarize responses to questions from other training session
It is a reference to help trainers prepare for questions they may encounter
Review in detail prior to the first end user training session
Texas Student Data System n Lubbock ISD n 36©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Metric Definitions - Algebra I
Why measure Algebra I vs. other courses?- Research indicates that students who enroll and complete Algebra I by the 9 th grade (a key
prerequisite for higher math) will graduate in higher numbers and are more likely to be college ready.
Algebra I Metrics-High School: - Taking or have Taken: % of students who have taken Algebra I by the 9th grade. In the case
of students who have never enrolled, teachers and counselors can quickly identify students who need to be enrolled and, based on each student’s academic history, provide the necessary support to register and prepare them for Algebra I.
- Passing or have Passed: % who are passing/have passed by the 9th grade. For students who are currently enrolled in Algebra I, teachers can quickly identify students who need additional academic support to successfully complete the course. In the case of former Algebra I students, teachers of advanced math courses can view Algebra I performance to evaluate the level of support students may require as they engage in more rigorous coursework.
Algebra I Metrics - Middle School:- This measure, defined as enrolling and successfully completing Algebra I by the 8 th grade,
helps campuses identify the level of students who show potential for more advanced coursework and their ability to master higher level math skills to be successful at the post-secondary level.
Texas Student Data System n Lubbock ISD n 37©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Metric Definitions - Attendance (Campus Level)
Common measure used by districts to track attendance and for state reporting
Use to pinpoint specific students negatively impacting ADA. Value often lower than ADA.
Catch students with attendance problems that aren’t reflected in daily attendance measures above
Texas Student Data System n Lubbock ISD n 38©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Metric Definitions – Discipline (Campus Level)
All Discipline Incidents: percentage of students with repeat occurrences (5 or more) of discipline incidents representing minor infractions (school code of conduct incidents) or one or more discipline incidents representing the most serious incidents, excluding school code of conduct incidents.
School Code of Conduct Incidents: percentage of students who have one or more minor infractions (school code of conduct incidents) in a given grading period.
Identifies students with chronic discipline issues
Identifies students with early signs of discipline problems
Texas Student Data System n Lubbock ISD n 39©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Today’s Agenda
Topic Timing
Background - TSDS Dashboards 5 min
Your Roles as Trainers 10 min
Review Dashboard Training Module 2 hrs
Data Inquiry Framework 10 min
Going Forward – Rollout Plan, Trainer Support, and Feedback 10 min
Q&A 20 min
Total 3 hrs
Texas Student Data System n Lubbock ISD n 40©2011 Michael & Susan Dell Foundation. All Rights Reserved.
A Structured Data Inquiry Process can help you get the most out of the Dashboards
•What do I want to know?•Why is this student having issues?Inquire•Focused data search on DashboardAcquire Data•Keep asking, “What does the data suggest?”•‘Peel the onion’ – continuous cycle of inquiryAnalyze Data
•Intervention, Instruction, DiscussionTake Action•Did the data support the situation?•Did my action have the desired effect?Evaluate
* Model based on work done by Jeffrey Watson, Wisconsin Center for Education Research
Texas Student Data System n Lubbock ISD n 41©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Inquire and Acquire Data
• Which students are consistently missing class?
• From the Subject Specific Class Overview page, sort by Current Class Absences
• Seven students have a high number of absences
Texas Student Data System n Lubbock ISD n 42©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Analyze Data – Layer 1
• Look across each student’s metrics
• Most have a troubling attendance trend and poor performance
• Ryan Plata’s data look a little different
• Troubling attendance trend
• Declining benchmark and grades
• But, meeting TAKS standard
Texas Student Data System n Lubbock ISD n 43©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Analyze Data – Layer 2
• Go to Ryan’s student page to investigate
• Attendance and Course Grades are flagged telling you to dig deeper in these areas
• Interestingly, Advance Academics shows a red stoplight
• Go to the detailed metric pages to learn more
Texas Student Data System n Lubbock ISD n 44©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Analyze Data – Layer 3
• Start by looking at the attendance page
• The daily attendance rate detail available in the more menu shows us Ryan has just started having unexcused absences this grading period
Texas Student Data System n Lubbock ISD n 45©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Analyze Data – Layer 4
• The class period absence detail available in the more menu shows us Ryan recently had a death in the family
Texas Student Data System n Lubbock ISD n 46©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Analyze Data – Layer 5
• Exploring the Advance Academic page shows us Ryan is not meeting his potential
• Ryan scored commended on TAKS ELA, but is not enrolled in an advanced class
Texas Student Data System n Lubbock ISD n 47©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Take Action and Evaluate
Speak with Ryan about the recent loss in his family
Contact the counselor and/or parents for support
Potentially, change Ryan’s schedule to an advanced ELA course
Check back to the dashboards frequently to monitor his attendance and academic metrics
Texas Student Data System n Lubbock ISD n 48©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Today’s Agenda
Topic Timing
Background - TSDS Dashboards 5 min
Your Roles as Trainers 10 min
Review Dashboard Training Module 2 hrs
Data Inquiry Framework 10 min
Going Forward – Rollout Plan, Trainer Support, and Feedback 10 min
Q&A 20 min
Total 3 hrs
Texas Student Data System n Lubbock ISD n 49©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Next Steps and Roll-out Schedule
Trainers are expected to share responsibility for training all educators on their campus - Campus administration – principals, assistant principals, counselors- Department heads- Specialists - Teachers
Plan for one hour to cover training material- Workshop materials and attendance forms - Kathy Rollo ([email protected])
Training should be complete by November 29th
Feedback session will be held on November 29th to share findings, observations and suggestions
Texas Student Data System n Lubbock ISD n 50©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Dashboards Available to Q2 September 26th!
Data will reflect current school year – 2011-2012
Some data will be blank until the current year data becomes available:- Class grades-after the first six-week grades are final- District assessments-after the first administration
You can start using the dashboards today!- Review student dashboards and academic history- Monitor attendance and discipline data- Analyze TAKS results to group students and guide
instruction; address campus-wide improvement areas- Identify students with credit deficiencies- Identify students with potential for advanced courses
Texas Student Data System n Lubbock ISD n 51©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Reminder - Training Best Practices
Train in small groups – 15 people maximum
If possible, group into functional areas – e.g., administrators, teachers, or subject area
Ensure everyone has access to a computer (e.g., computer lab, laptops)
Review the pre-training checklist to ensure no technology issues arise
Review the training materials and video shortly before the training session- Log into and get comfortable with TSDS- Watch the demo video and review the FAQs so you can respond to users- Review the Classroom, Student and Campus exercises and select the examples
you plan to highlight from your campus and classroom
Use the training materials during the session
Capture questions you can’t answer and work with the data steward to ensure there is follow-up with the users
Make sure participants complete the feedback forms - during session or send email with URL following session: https://www.surveymonkey.com/s/LubbockQ2Users
Let us know if there are any other tools or materials that can help you!
Texas Student Data System n Lubbock ISD n 52©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Training Resource Summary
Document Description
Training Setup Checklist Tasks to coordinate prior to training session(s)
Registration Form Used to track who has registered for the training (if required)
Training Attendance Form
Used to log who received training. After each training session, this must be completed and sent to Kathy Rollo – [email protected]
Training Presentation-Train the Trainer
Copy of today’s presentation – Dashboard Training Module slides (18-35) to be used for your training sessions.
Training Demo Video for Trainers
Link to download video for your training sessions
Training User Guide-FAQ Reference document with common questions and known issues
Training Template-Participant Action Plan
Each trainee will have a copy of this ‘exercise’ to complete as part of the training. These should be sent to Kathy [email protected].
http://www.districtconnections.com/Dashboards.htm: link to metric reference library with information on metric definitions, how to use metrics and practitioner research.
All materials are available for download at www.districtconnections.org/LubbockISD
Texas Student Data System n Lubbock ISD n 53©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Instructions to download the training video
Go to URL:
https://www.districtconnections.org/LubbockISD/Docs/
Go to Training Material section and mouse over the Trainer Demo Video item (text turns blue)
Right click this item and select ‘Save target as…’ Select the folder you wish to store this file Open the file to test and make sure it is running If you have any problems with running or downloading the
file, contact the Data Steward – [email protected] Do not play the video from the website during
training, download to PC/laptop projecting training materials
Texas Student Data System n Lubbock ISD n 54©2011 Michael & Susan Dell Foundation. All Rights Reserved.
CTE and Attendance Credits
LISD representative to review
Texas Student Data System n Lubbock ISD n 55©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Today’s Agenda
Topic Timing
Background - TSDS Dashboards 5 min
Your Roles as Trainers 10 min
Review Dashboard Training Module 2 hrs
Data Inquiry Framework 10 min
Going Forward – Rollout Plan, Trainer Support, and Feedback 10 min
Q&A 20 min
Total 3 hrs
Texas Student Data System n Lubbock ISD n 56©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Workshop Feedback Form
Providing feedback will help future training sessions
Complete the survey online - https://www.surveymonkey.com/s/Lubbock926
It will take less than 5 minutes to complete the survey
Texas Student Data System n Lubbock ISD n 58©2011 Michael & Susan Dell Foundation. All Rights Reserved.
Handouts