BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester...
Transcript of BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester...
![Page 1: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/1.jpg)
Brandan Keaveny EdD Rochester City School DistrictBrandan Keaveny, EdD Rochester City School District
Manjeet Rege, PhD Rochester Institute of Technology
![Page 2: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/2.jpg)
Our Route for this Morning
Stop 1) The extreme case for justifying the racejustifying the race.
Stop 2) Intervention or condition?
Stop 3) Intelligence Gathering- At Present
Stop 4) Data MiningStop 4) Data Mining
Stop 5) Data Mining with K-12 Data
Stop 6) The Power of Partnerships
![Page 3: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/3.jpg)
STOP 1: AN EXTREME CASESTOP 1: AN EXTREME CASE JUSTIFYING THE RACE
![Page 4: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/4.jpg)
About RochesterAbout Rochester
•Ranked 180th by NCES for size of district: 33 000 students•Ranked 180t by NCES for size of district: 33,000 students•60% African American, 20% Hispanic/Latino, 10%White, 10% Other•220,000 residents•11th Most impoverished city for students under the age of 12
![Page 5: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/5.jpg)
Split‐Graduates to Non‐GraduatesSplit Graduates to Non Graduates
YesNo
![Page 6: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/6.jpg)
Connection Between HS ELA, HS Math, and Graduation Rate Cohorts 2001‐2004
100%
60%70%80%90%
30%40%50%60%
0%10%20%30%
2001 2002 2003 2004 2005
HS ELA HS Math HS Graduation
![Page 7: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/7.jpg)
Graduation Rates Adjust to Changes in State Policy
Total Cohort, Graduation Rates, Timeline of Policy Changes in 75%
to Changes in State Policy
Determining August Graduation Rates
50%
rcen
t
25%
Per
0%2000 2001 2002 2003 2004 2005
Cohort Year
b b d h b
Method for calculation of graduation rate changed.
Local Diploma Requirements Changed
*Adjusted to include summer graduates, number is approximation
Changes in NYSED RequirementsNumbers based on cohort members through August four years later.
Changes in NYSED Requirements
![Page 8: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/8.jpg)
Change, at any time?
The State has changed the proficiency threshold
Change, at any time?
The State has changed the proficiency threshold (as known as a cut score) to a higher target.
Change Zone 201Did Not Meet State Standards Met State Standards
Range of Raw Test ScoresLow High
10
Did Not Meet State Standards Met State Standards
g
200
Did Not Meet State Met State Standards
09Standards
![Page 9: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/9.jpg)
Grades 3‐8 ELA Achievement 2007‐2010
Percentage of Students Scoring in
Levels 3 and 4
Recalibration
![Page 10: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/10.jpg)
Grades 3‐8 Math Achievement 2007‐2010
Percentage of Students Scoring in
Levels 3 and 4
Recalibration
![Page 11: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/11.jpg)
STOP 2: INTERVENTION ORSTOP 2: INTERVENTION OR CONDITION?
![Page 12: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/12.jpg)
What is reality?What is reality?
• Intervention:Intervention: “to interfere with the outcome or course especially of a condition or process (as toespecially of a condition or process (as to prevent harm or improve functioning)” Merriam Webster Online, Def. 3b, retrieved July 28, 2010
• Condition: “ a state of being”Merriam Webster Online, Def. 3b, retrieved July 28, 2010
![Page 13: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/13.jpg)
STOP 3: INTELLIGENCE GATHERINGSTOP 3: INTELLIGENCE GATHERING AT PRESENT
![Page 14: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/14.jpg)
We are data rich…..We are data rich…..
• Attendance• Behavior• Course/Teacher• Enrollment• GPA• Grades 3‐8 ELA/Math testing data• Graduation Cohort DataP S i D• Program Service Data
• Regents Examination Test Scores
![Page 15: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/15.jpg)
We are system rich…We are system rich…
NYS Data Warehouse
D t tiData‐cation
System/process forSystem/process forInstructional Use
Local Data
Chancery SMSOperational Use Knowledge Discovery
Warehouse
IEP Direct
Peoplesoft
![Page 16: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/16.jpg)
Use of DataUse of Data
• Instructional UseInstructional Use• Operational Use
l d i• Knowledge Discovery
![Page 17: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/17.jpg)
Capacity Inhibits Rate of AnalyticsCapacity Inhibits Rate of Analytics
• Our focus needs to become cohort specific and Ou ocus eeds to beco e co o t spec c a dexamine factors that promote/deter learning within a short time period….How short, this is left
b k h h ld kto be known, however, it should not take an entire school year. W k h t k d ’t k h t• We know what we know, we don’t know what we don’t know. There lies the danger. The higher danger is being able to constructively respond todanger is being able to constructively respond to intervene when what we don’t know becomes known.
![Page 18: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/18.jpg)
STOP 4: DATA MININGSTOP 4: DATA MINING
![Page 19: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/19.jpg)
Why do we collect data?Why do we collect data?
• Historically– Maintain Database Consistency
• By checking and maintaining the Logs
– Proof of EvidenceR d d t ti t d• Records and transactions are stored
• RecentlyA l h– Analyze the Data
• Draw inference
![Page 20: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/20.jpg)
Data FloodingData Flooding
• The amount of data that is generated has increased gtremendously.
• The increase is going to continue as the data storage b his becoming cheaper.
• We need to extract useful information from the stored datastored data.
• How can we achieve that?
![Page 21: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/21.jpg)
What is Data Miningg
P t di lid• Process to discover valid, novel, nontrivial,
t ti ll f l dStatistics
Machine Learning &
lpotentially useful and understandable patterns f l d t t Data Mining
Visualization
from large data sets. g
DatabaseDatabase systems
Adapted fromTan, Steinbach, Kumar book
![Page 22: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/22.jpg)
Data Mining examples in other domains
• Google –Google – wonder wheel
• Amazon• Amazon –– Collaborative Information Filtering
• Groceries –– diapers and beer bought together
![Page 23: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/23.jpg)
STOP 5: DATA MININGWITH K‐12STOP 5: DATA MINING WITH K 12 DATA
![Page 24: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/24.jpg)
How can we benefit by implementing Data Mining in Education?
P f di ti• Performance prediction• At risk student identification• Forecasting drop out rate• Alumni donationsAlumni donations • Clustering students and teachers together
Co relating student performance with teacher– Co‐relating student performance with teacher performance
![Page 25: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/25.jpg)
Data Mining Stagesg g
I t t ti /
KnowledgeKnowledgeData Mining
Interpretation/Evaluation
PatternsPreprocessing
Data Target
SelectionPreprocessed
Data
Data
adapted from:U. Fayyad, et al. (1995), “From Knowledge Discovery to Data Mining: An Overview,” Advances in Knowledge Discovery and Data Mining, U. Fayyad et al. (Eds.), AAAI/MIT Press
![Page 26: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/26.jpg)
StagesStages
• SelectionSelection– Identifying the requirements to create target data sets
• Pre‐processingp g– To perform data cleaning on the target dataset
• Data Miningg– To design & implement the algorithms
• Interpretation / Evaluation – Analysis of the generated results
![Page 27: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/27.jpg)
Data Mining ArchitectureData Mining Architecture
Graphical user interface
Data mining engine
Pattern evaluation
DATA
Data mining engine
Knowledge-base
D t
Data cleaning & data integration
Filtering
DATA
Data WarehouseDatabases
Adapted fromTan, Steinbach, Kumar
![Page 28: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/28.jpg)
Individual StepsIndividual Steps
• Data Bases or Data Warehouse– Stores the data
• Data Mining Engine– Implement the Mining Algorithms
• Pattern EvaluationTh diff h d l l d d l d– The different graphs and plots are analyzed and evaluated
• Graphical User Interface– Visualization– Visualization
![Page 29: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/29.jpg)
Techniques of Data MiningTechniques of Data Mining
• ClassificationClassification• Clustering
Co clustering– Co‐clustering– Clustering of evolving data
![Page 30: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/30.jpg)
ClassificationClassification
Major Graduating Undergrad/ Donor
Alumni Database • A method for predicting the
GPA GradIT 3.3 U YesMath 3 8 G N
instance class from pre‐
Math 3.8 G NoBusin 3.2 U YesBusin 3 5 G Yes
labeled (classified)
Busin 3.5 G YesIT 3.0 G Yes……. ………. ………… ……
instances
![Page 31: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/31.jpg)
We need to classify new students
The system needs to “learn” from the historical data, in order to
Psy 3.3 U ?
classify new data.
Chem 3.8 G ?
Math 3.2 U ?
Busin 3.5 G ?
IT 3.0 G ?
![Page 32: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/32.jpg)
ClusteringClusteringClustering means naturally grouping data instancesdata instances
attendance grades
Each row is a student
![Page 33: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/33.jpg)
Student Cl 1Student matrix
Cluster 1
Cluster 2
Cluster 3
![Page 34: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/34.jpg)
Co‐clusteringCo clustering
• Clustering only provides us with groups of dataClustering only provides us with groups of data instances (students).
• It does not provide us information about why certain p ystudents are similar.
• Simultaneous clustering of students and their features gcan help us discover this.
![Page 35: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/35.jpg)
XX
53 1 154
students
45
4 4
5
5
5
3
3
1 1
5
4
34
4
4
354
Cluster 1 Cluster 2 Cluster 3
skills
![Page 36: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/36.jpg)
What can we discover?What can we discover?
St d t th t t fi i t i th kill• Students that are most proficient in the skills.• Students that are not proficient in some skills.
I t ti i d– Intervention required.
![Page 37: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/37.jpg)
OR
![Page 38: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/38.jpg)
Can we cluster more information together?g
killskills
students
teacher
In general, we can cluster different types of information together to discover co-relations.
Clustering at regular time intervals can help us learn how the correlationsClustering at regular time intervals can help us learn how the correlations evolve.
![Page 39: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/39.jpg)
STOP 6: POWER OF PARTNERSHIPSSTOP 6: POWER OF PARTNERSHIPS
![Page 40: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/40.jpg)
Formula for Reaching Higher Levels of Analytic C i h R h S dCapacity that Reaches Students
• Use quantitative accountability data to continue the q yneeds analysis process through the use of Data Mining Techniques.O i h l d l l t b d d b i• Organize schools, grade levels etc. by need and begin a qualitative analysis of the factors that bound such grouping with coded observational data that intensifies g p gsupport of the quantified data.
• Use Educational Data Mining Techniques to create a profile of the students ultimately leading to customprofile of the students ultimately leading to custom interventions for groups or individual students.
![Page 41: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/41.jpg)
Selected Relevant PublicationsSelected Relevant Publications– M. Rege and M. Dong, “A Graph Theoretic approach to Heterogeneous Data Clustering: New
Research Directions and Some Results,” VDM Verlag, 2009, ISBN: 978‐3‐639‐11658‐8. – Y. Chen, M. Rege, M. Dong and J. Hua, “Non‐negative Matrix Factorization for Semi‐supervised
Data Clustering”, Knowledge and Information Systems Journal (KAIS), Vol. 17, No. 3, 2008.– M. Rege, M. Dong and F. Fotouhi, “Bipartite Isoperimetric Graph Partitioning for Data Co‐
clustering”, Data Mining and Knowledge Discovery Journal (DMKD), Vol. 16, No. 3, 2008. g , g g y ( ), , ,– M. Rege, M. Dong and J. Hua, “Graph Theoretical Framework for Simultaneously Integrating
Visual and Textual Features for Efficient Web Image Clustering”, in proceedings of 17th International Conference on World Wide Web (WWW), 2008.M Rege M Dong and F Fotouhi “Co clustering documents and words using Bipartite– M. Rege, M. Dong and F. Fotouhi, Co‐clustering documents and words using Bipartite Isoperimetric Graph Partitioning”, in proceedings of IEEE International Conference on Data Mining (ICDM), 2006
![Page 42: BrandanBrandan Keaveny EdD Rochester City School District ... · Manjeet Rege, PhD Rochester Institute of Technology. Our Route for this Morning Stop 1) The extreme case for justifying](https://reader035.fdocuments.in/reader035/viewer/2022070111/60517a15155ec00c8e63f1b3/html5/thumbnails/42.jpg)
Brandan Keaveny, Ed.D.Director of Accountability
Manjeet Rege, Ph.D.Assistant ProfessorDirector of Accountability
Office of AccountabilityRochester City School Districtbrandan keaveny@rcsdk12 org
Department of Computer ScienceRochester Institute of [email protected]://www cs rit edu/[email protected] http://www.cs.rit.edu/~mr