Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth...

12
W hat should we spend on students to ensure they suc- ceed? Who should teach our children to help them achieve? How should we allocate these monetary and staffing resources to be effective? These questions are basic to providing children a good education. Answering them, however, is not that easy, especially as education policy- makers rely more on data to make instructional changes needed to improve student performance. The data must be accessible, of high quality, and easily understood. They must also be broad enough in scope to respond to the diversity of instructional policy issues, yet have ample detail to accurately answer specific questions. Can policymakers rely on existing state education databases to find the answers they need? This was the focus of a new study conducted by Southwest Educational Development Laboratory (SEDL). Our results reveal an important message for state policymakers: state education databases are critical but underutilized to inform and support policy decision making and research. Further, the quality of the data necessitates that states make ongoing improvements. This issue of Insights features what we discovered about existing state education data, guidance for policy audiences about the instructional resource allocation questions that can and cannot be answered with existing data, and our recommendations for state data system reform. DECEMBER 2005 NUMBER 18 SOUTHWEST EDUCATIONAL DEVELOPMENT LABORATORY SOUTHWEST EDUCATIONAL DEVELOPMENT LABORATORY / 211 E. 7TH STREET, SUITE 200 / AUSTIN, TEXAS 78701 / 800-476-6861 / http://www.sedl.org Enhancing Data Use and Quality to Shape Education Policy Insights ON EDUCATION POLICY, PRACTICE, AND RESEARCH In This Issue SEDL investigated state education data in four states to determine whether research can be conducted to find answers to education resource and student performance policy questions. This issue high- lights study findings that policy- makers will find informative in their efforts to meet standards and use data effectively. f

Transcript of Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth...

Page 1: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

What should we spend on students to ensure they suc-ceed? Who should teach our

children to help them achieve? Howshould we allocate these monetary andstaffing resources to be effective?

These questions are basic to providing children a good education.Answering them, however, is not thateasy, especially as education policy-makers rely more on data to makeinstructional changes needed toimprove student performance. Thedata must be accessible, of high quality, and easily understood. They must also be broad enough inscope to respond to the diversity ofinstructional policy issues, yet haveample detail to accurately answer specific questions.

Can policymakers rely on existingstate education databases to find theanswers they need? This was the focusof a new study conducted bySouthwest Educational DevelopmentLaboratory (SEDL).

Our results reveal an importantmessage for state policymakers: stateeducation databases are critical butunderutilized to inform and supportpolicy decision making and research.Further, the quality of the data necessitates that states make ongoing improvements.

This issue of Insights features whatwe discovered about existing stateeducation data, guidance for policy

audiences about the instructionalresource allocation questions that canand cannot be answered with existingdata, and our recommendations forstate data system reform.

D E C E M B E R 2 0 0 5

N U M B E R 1 8

S O U T H W E S T E D U C A T I O N A L

D E V E L O P M E N T L A B O R A T O R Y

SOUTHWEST EDUCATIONAL DEVELOPMENT LABORATORY / 211 E. 7TH STREET, SUITE 200 / AUSTIN, TEXAS 78701 / 800-476-6861 / http://www.sedl .org

Enhancing Data Use and Quality to ShapeEducation Policy

InsightsON EDUCATION POLICY, PRACTICE, AND RESEARCH

In This IssueSEDL investigated state educationdata in four states to determine

whether research can be conductedto find answers to education

resource and student performancepolicy questions. This issue high-lights study findings that policy-

makers will find informative in theirefforts to meet standards and use

data effectively.

f

Page 2: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

State EducationData SystemDevelopmentLooking back at the roots of statedata system development shows usmuch has changed. Yet we have torecognize it has been a slow processand one that will be ongoing. The firstevidence that education data were collected dates back to the early1800s when school administrativerecords contained enrollment, atten-dance, and literacy figures (Goldin,1999). The data were unreliable, butserved as a springboard for future data collection.

In 1867, Congress legislated aDepartment of Education to “collect suchstatistics and facts as shall show the condition and progress of edu-cation in the several states and territo-ries” (see An Act to Establish aDepartment of Education, Ch. 158. 39thCongress., 2nd Sess., 14 Stat. 434 of1867). States now needed to provide thefederal government with public school data on students, teachers, and schools, as well as basicfinances. At the time, you could get thenumber of students and staff in a town-ship and compare it to another, but littleelse was possible. Not only did thesedata have little detail, they were notconnected. For example, you could notcompute students by grade until around1910 or relate a teacher’s education levelto income until three decades later.

Data must beaccessible, ofhigh quality,and easilyunderstood.

Database, Data System, Data WarehouseWhat’s the Difference?

Database: an organized collection of information or data elements, typicallystored in a computer, that can be searched, sorted, reorganized, and analyzedrapidly. The following are database models:• Flat file: data in one record that cannot be linked to other records (a single

table format)• Hierarchical: data in separate records that are attached to one root (one-to-

one relationship)• Network: data in separate records that can be attached to multiple other

records (many-to-many relationship)• Relational: data in a collection of tables without any hierarchy and that are

physically independent• Object-oriented: data in separate records that can be linked to a variety of

data objects like text, graphics, photos, video, and sound

Data system: a collection of computer programs that enable you to store, mod-ify, and extract information from a database. One such data system is the TexasPublic Education Information Resource.

Data warehouse: a combination of many different databases across an entiresystem to present an entire picture. Data are added but never removed. Anexample of a data warehouse is the Louisiana Educational Accountability DataSystem.

Sources:Date, C. J. (2003). An Introduction to Database Systems, 8th Ed. Boston, MA: Addison-Wesley.http://www.frick-cpa.com/ss7/Theory_Models.asp

2 Insights | Southwest Educational Development Laboratory

Page 3: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

Insights | Southwest Educational Development Laboratory 3

In the early twentieth century,when data collection was becomingmore sophisticated, states were alsogrowing more interested in studentproficiency. A number of states begantesting students using state or nation-al standardized tests and collecting theresults. This was an important firststep toward having data to assess student performance; however, student demographic data, such asrace/ethnicity, gender, and age, were not collected until much later(Dorn, 2003).

These early attempts to collectdata served as a basis for creatingmany current statewide education datasystems. However, it was not until the1970s or 1980s that most of these datasystems were actually established,some a decade later. They weredesigned to collect data for specificpurposes, most often in response tofederal reporting requirements, budgetmanagement, and district compliancetracking. As state accountability priori-ties took precedence in education decision making, new data collectionand management became necessary.

The majority of state data systemsestablished were composed of distinctdatabases focused on one level ofdata, i.e., fiscal, student, teacher, orschool data. This still holds true inmany education data systems today.In addition, not all of the databasesare necessarily housed or managed bythe same department in the state edu-cation agency (SEA) or even withinthe agency itself. These separate data-bases are full of useful information,but many challenges exist to link thedata. Linking the data is essential ifwe want to answer current educationpolicy questions, as well as meet stateand federal standards.

As the type of SEA data and datamanagement have changed, so has ouruse for the data. Not only are boardsof education, legislatures, and fundingsources requesting quality data-basedanswers to important policy questions,but school personnel, courts, and thegeneral public are relying on data

more in their decision making. Someof this upsurge in data use reflectsfederal No Child Left Behind (NCLB)legislation related to measuring adequate yearly progress, choosingacademic programs, setting studentimprovement goals, and keeping parents informed. For example, statesare providing publicly available reportcards that include data on school anddistrict accountability and limited fiscal and/or staff resources.

School finance lawsuits have alsonecessitated increased use of statedata. Nationally, 45 states, includingall in our region, have been, or arecurrently, engaged in court cases. Thecourts have asked pointed questionsthat require data-based answers. Onepervasive question is, What resources,fiscal and staffing, are needed toimprove performance in all students?This question, as well as questions onhow to effectively allocate thoseresources, would be best answeredwith SEA data that links individualstudent data to staff, school, district,fiscal, and assessment data.

Do state databasesallow the investigation of the relationshipbetween fiscal andstaff instructionalresources and student performance?

Page 4: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

4 Insights | Southwest Educational Development Laboratory

SEDL Study on StateData SystemsFindings from our 2003 policy study,Examination of Resource Allocation inEducation: Connecting Spending toStudent Performance, served as thebasis for our current study on stateeducation databases. These findingsfor Arkansas, Louisiana, New Mexico,and Texas are below:

• High-performing school districtsput more fiscal and staff resourcesinto instructional areas than dolow-performing districts.

• Districts with increasing studentachievement use data-driven decision making to supportresource allocation.

In discussions with policymakersabout these results, they asked forgreater detail on instructionalresources linked to student perfor-mance. To pursue this, our lateststudy, Investigation of EducationDatabases in Four States to SupportPolicy Research on Resource Allocation,examined Arkansas, Louisiana, NewMexico, and Texas education data collected and managed by the fourstate education agencies to determinewhether research can be conducted tofind these links.

Our goal was to understand thescope, quality, and availability of eachstate’s data to support its instruc-tional resource decisions. Specifically,we addressed the question, Do statedatabases allow the investigation ofthe relationship between fiscal andstaff instructional resources and student performance?

Four years of state instructionalexpenditure; staff characteristic; student performance; and student,school, and district characteristic data were the focus of our study. We assessed how the four states utilize their data and determined

state-specific policy concerns by the following:

• Examining public reports, summaries, and research

• Discussing data management withstate education staff and otherstate policymakers

• Reviewing state policy

After identifying key variables, weassembled and examined the data forusability on five criteria:

• Availability and accessibility• Completeness• Accuracy• Consistency• Alignment

Next, we gauged commonalitiesacross the state data and performeddescriptive statistics to assess dataquality. An important last step was tomeet with state policymakers to dis-cuss our findings and formulate ideasfor data system reform specific totheir state.

The State of StateEducation DataWe focused on instructional expenditure data, instructional andadministrative staffing data, schooland district demographic data, andstudent demographic and performancedata. The four state data systems gen-erally have separate databases forthese different data. However, nonewere set up to link individual studentdata directly to teacher and fiscal data.Since our initial investigation in 2003,Arkansas and Louisiana have movedtoward connecting each student’s datawith his or her teacher’s data.

Instructional Expenditure DataInstructional expenditures are fundsspent to support teaching and

States have been consistent over time in their categorization and collection ofinstructional expenditures.

Page 5: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

Insights | Southwest Educational Development Laboratory 5

learning that occur in the classroom,with the majority going to teachercompensation. All four study states,similar to states across the nation,have detailed instructional expendi-tures for school districts. Texas alsohas school expenditure data. Arkansas has just begun to collectschool-level fiscal data, and Louisianauses a statistical method to estimateschool expenditures based on district data.

Each of the instructional expenditure databases are set up witha limited number of functions, e.g.,instruction and student support. Ingeneral, each state has similar instruc-tion function categories; however,they are not necessarily named ordefined exactly the same way. Thefunctions are divided into object cate-gories, such as salaries, benefits, and

supplies. Often additional subfunc-tions and subobjects are also designat-ed. As shown in Table 1, several of thestates also have instructional expendi-ture data by programs, such as specialeducation and Title I. We found thatthe states have been consistent overtime in their categorization and collection of instructional expendi-tures, contributing to greater datareliability, accuracy, and completeness.

Salary data was of particular interest, and in all four states are doc-umented in both the state’s fiscal andteacher databases. The fiscal databasehas district or school salary averages,while the staff database recordssalaries for individuals. Using the indi-vidual data allows greater flexibility inlooking at particular subgroups ofstaff, which may be beneficial in making policy decisions. Generally, the

salary data is reliable; however, it isoften impossible to discern someincentives, bonuses, or other reim-bursements staff may receive. Thismakes it difficult to get a completecompensation picture. To this sameend, we also investigated staff bene-fits. As seen in Table 2, unlikesalaries, actual benefit costs per individual are not recorded by threestates. Rather they are prorated using district benefit data. This further complicates decision makingregarding total teacher compensation.

Instructional and AdministrativeStaffing DataData on teachers are more extensivethan other staff data. In addition tothe individual salary data in thestaffing databases, the following dataare available on individual teachers:

Table 1SEA Instructional Expenditure Data in Four Study States

Arkansasa Louisianaa New Mexico Texas

Instruction- • Instruction • Instruction • Direct instruction • Instruction andrelated function • Student support • Student support • Instructional instruction-relatedcategories • Instructional staff • Instructional staff support services

services services • Instructional and school leadership

• Support services–student

Object categories • Salaries • Salaries • Personnel services • Payroll costs• Benefits • Benefits • Employee benefits • Professional and• Professional • Professional • Purchased services • Payroll costs

purchased services purchased services • Supplies and • Supplies and materials• Supplies and • Supplies and materials • Other operating costs

materials materials • Travel and training • Debt service• Other objects • Other objects • Capital outlay • Capital outlay

Unit of analysis • Program (for • Program • District • Programinstruction only) • District • School

• District • District

a Function and object categories align with federally defined functions and objects (Census form F-33).

Page 6: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

6 Insights | Southwest Educational Development Laboratory

• Basic demographics, such as gender and race/ethnicity

• Educational attainment• Years of experience• Certification• Teacher test scores• Position, e.g. role, school, district• Full-time equivalent (FTE) or

percent time

When relying on these staffingdata for decision making, we facedseveral challenges. First, data man-agers across all four states report limited reliability with teacher experi-ence data since they often depend onunverified self-reports from individualteachers or school districts. Cross-checking the data over a span of years would be beneficial. Second, certification databases, often housedand managed separate from otherstaffing databases, are cumulative and not always easily aligned withother data. And last, not all statesreliably document FTE or percenttime data for each position a staffperson holds.

School and District DemographicDataAll four states have extensive schooland district demographic databases.Much of the data in these databasesare averages; therefore, some cautionmust be taken when using school versus district data to look at a partic-ular characteristic. For example, if wewanted to know the percent of low-income students in XYZ district, wecould go directly to the district data-base and find the answer. We couldalso go to the school database, find all the schools in XYZ district, thenaverage their student income data forthe answer. The problem is that thetwo answers may be different. Still,the school and district databases offerinformation useful for school or district report cards, annual state education reports, descriptive research,and required funding reports.

School databases in the four statesincluded the following data:

• School characteristics, e.g., typeschool and grade range

Data on teachersare more extensivethan other staffdata.

Table 2SEA Individual Salary Data in Four Study States

Arkansas Louisiana New Mexico Texas

What salary measure(s) are available? • Total salary • Base pay • Base pay • Base pay• Additional • Supplemental

compensation pay(3 types)

Can partial salaries be determined for Yes Yes Yes Yespart-time staff? (since 2003)

Do salary data align with actual Yes Yes No Yesexpenditures? (since 2003)

Are individual benefit expenditures Yes No No Noavailable?

Page 7: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

• Student population, e.g., numberof low-income and minority-statusstudents

• Attendance, graduation or completion, and dropout rates

• Special program participation• Per-pupil expenditures• Accountability ranking

The district databases often includedadditional information about districtwealth. It was relatively easy to accessschool and district databases sincemany are available and downloadableon SEA Web sites.

We also went to the school anddistrict SEA data to get a broader pic-ture of the educational environmentof students and staff. However, it wasnecessary to get additional data thatstates do not collect through federaldatabases, such as the U.S. CensusBureau and the National Center forEducation Statistics. For example, federal databases designate localeinformation for a school or district,e.g., urban, suburban, or rural, thatmost of the states do not include intheir databases. Also, the federal data-bases contain information abouthousehold characteristics of families inthe school or district’s vicinity, suchas household income or parent level ofeducation. In order to use these datawith our SEA data, we had to findcommon identifiers on which theycould be merged. It takes a bit ofeffort but provides more detail forspecific groupings or geographic areas.

Student Demographic and Performance DataEach state has different ways of complying with federal and staterestrictions to ensure confidentiality ofstudent information. Some have com-puterized methods to scramble studentidentities before providing the data.Others provide only aggregated studentdata at the grade or school level.

In each state a student’s datarecord includes characteristics, such asrace/ethnicity, gender, and age. The

student’s participation in programs,such as free and/or reduced pricelunch, preschool, afterschool, and special education, is also available.Some states include family informa-tion, such as whether the student isin foster care or homeless. All statesconnect the student to his or hergrade level, school, and district.

Data on each student also includeperformance measures. For our study,we sought only student achievementdata. Other measures of student performance, such as attendance, grad-uation, and dropout rates, are alsoavailable. All four states have beenimproving their capacity to measurestudent performance through standard-ized tests. While this process hasimproved the quantity of these datawith regard to the number of testsoffered and the grades tested, it alsohas resulted in inconsistency in thetest scores available from year to year.In each of the four study states,changes were made to the tests admin-istered, grades tested, or scoring stan-dards during the study period. Thesechanges hampered getting a completepicture of student achievement over time.

Using the StateInstructionalResource DataAfter getting a better understandingof the data in the four states andassessing the relative quality of thosedata, we wanted to know if the datacould be used to answer policy ques-tions so prevalent today. What shouldbe spent on students to ensure theysucceed? Who should teach children tohelp them achieve? How should thesemonetary and staffing resources beallocated to be effective?

The state data are both availableand accessible to answer basic, butimportant, questions about the adequacy and equity of state funding formulas and compensation for teach-ers and other staff. Searching the datato find out about teacher characteris-tics can be done relatively easy. Thiscan be followed by an investigationinto what impact these characteristics may have on student success. It’s possible to get a good grasp of whattype of teachers might be allocated to

Insights | Southwest Educational Development Laboratory 7

SEA Staffing, School, District, and Student Data

The four states in SEDL’s study have a variety of useful staff, school,district, and student data. Much of the data are available on SEAWeb sites, and some must be directly obtained from the SEA to

ensure confidentiality is maintained. The SEA data include the following:

Staffing data: data for individual teachers as well as other staff, such asbasic demographics, education, experience, and certification

School and district data: data for individual schools and districts, such asstudent population, per-pupil expenditures, and accountability rankings

Student data: data for individual students, such as race/ethnicity, specialprogram participation, grade, and student achievement scores

Page 8: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

different schools to impact studentoutcomes. That is, teachers with cer-tain qualifications may be betterplaced at schools with high-need student populations or schools in particular locales.

About SpendingHow instructional dollars are spentand how the spending varies acrossdistricts can be studied using state fiscal data. These data are available tolearn about district spending in allfour states and school spending inTexas. To see if funds allocated are inany way connected to studentachievement, the fiscal data needs tobe merged with performance data.Specific questions that can beanswered with the state data includethe following:

• What are the differences ininstructional spending across districts in the state?

• Do districts that perform well allocate more instructional dollarsto salaries and benefits?

• How do districts of varying levels of performance allocateadministrative versus instructional dollars?

More detailed questions aboutschool and district spending onsalaries can also be answered if indi-vidual staff data are merged with student performance data. Forinstance, determine how teacher paycan impact student performance withthe merged data or find if salaries aredistributed equitably between schoolsand districts. Also, using these data,see what effect salary has on theretention of qualified teachers. Toanswer this particular question, itwould be necessary to have data onteacher mobility, which most of ourstates do not collect. However, it isnot difficult to calculatemobility using existing statedata on school and districtteacher assignment over multiple years. Unfortunately,no current data in the statedatabases can tell us whyteachers stay or leave.

Although answers to anumber of important policyquestions about instructionalspending can be found withexisting state data, additionaldata would increase ourlearning, especially in regardto teacher compensation. Itwould be helpful to have better measures for all funds paid to staff, such asindividual data on the cost ofbenefits, bonuses, and incentives. With this moreaccurate depiction of totalcompensation, it’s feasible toask about the influence ofbenefits and incentives on

teacher recruitment or retention, particularly in shortage areas such asbilingual and special education.

Another instructional area whereadditional data would be helpful isprofessional development. States havelittle data on professional develop-ment, so it is impossible to know if their investments produce results.Each state collects data on the hours of professional developmentcompleted, but not data on actual ordollar-equivalent measures for teachertime, stipends, travel expenses, andcosts for teacher substitutes.Information on the content would alsobe beneficial for addressing questionsabout the effectiveness of professionaldevelopment, its relative costs, andthe distribution of professional development resources across schoolsand districts.

About Teacher QualityTeacher quality has always been animportant policy issue, but NCLB hasheightened our need to use good data

8 Insights | Southwest Educational Development Laboratory

Merging data fromstate databases,you can understandwhether allocatingmore teachers,administrators, or aides impactsstudent achievement.

Page 9: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

to determine the quality of our teaching staff. We found only certain questions about teacher quality can beanswered using the existing statedata, i.e., data on teacher experience,education, and certification. Questionsabout other measures of quality, suchas application of pedagogical tech-niques, teacher motivation, and classroom management skills requireinformation not collected in statedatabases. Changes are being made,however. For example, New Mexico hasbegun to collect data on evaluationsof teachers as one measure of teacherquality. Using all of the availableteacher data and merging them withother databases, finding answers toquestions about teacher quality, itsrelationship to teacher salary, and its impact on student performance are possible.

Specific questions to ask using thedata are as follows:

• How does teacher experience,education, and/or certificationrelate to student achievement?

• Do higher teacher salaries buyteachers with more experience,higher education levels, andadvanced certification?

• How are teachers who are educated at different teacher education institutions distributedacross the state?

• Do rural areas have a higher rateof uncertified teachers?

• What is the connection betweenteacher retention and route tocertification?

Although you can use currentteacher quality data to find answers tomany policy questions, improvingthese data are important to ensureaccurate answers. For instance, wefound it would be important to helpdistricts better understand definitionsfor reporting experience, especially forteachers who transfer between dis-tricts or from other states. Also, ifdata on teacher degree major were

collected, questions on how many in-field teachers you have and how theseteachers are distributed across thestate could be answered. Combiningthese data with performance data, itis also possible to seek information onthe impact of these teachers, especial-ly in comparison to those not teach-ing in their field of study.

About Staffing PatternsWe found that using the existing statedata, you can determine what staffresources comprise each school andhow those resources differ acrossschools with varying levels of studentperformance. These staffing patternprofiles can be detailed for specifictypes of schools or teachers. For exam-ple, it’s easy to see how beginningteachers or administrators in small,medium, or large schools are distrib-uted across the state. Or maybe findout about what teaching patterns existfor rural and urban schools, especiallythose that are low performing.

Another important education policy issue is class size, not only itsrelative cost but its connection tostudent performance. Merging datafrom state databases, you can understand whether allocating moreteachers, administrators, or aidesimpacts student achievement. Beaware, however, that using a ratio ofthe number of students in a school tothe number of teachers in that schoolis the least accurate measure of classsize. It would be better to use datathat directly links students to teachersand teachers to specific classes.

Data System Reform:What Can Be DonePolicymakers, state data managers,and researchers need to work togetherto expand the use and quality of stateeducation data to learn more aboutinstructional resource allocation andits impact on student performance.

Insights | Southwest Educational Development Laboratory 9

States have little data on professional development, so itis impossible toknow if theirinvestments produce results.

Page 10: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

Specifically, policymakers andresearchers should become more familiar with state data and regularlydiscuss, with SEA data managers, howthe data could be improved to betteranswer policy questions. This wouldallow SEAs to expand the use of theirdata beyond traditional reports andmonitoring purposes. Also, statesshould increasingly work with nationaldata centers, such as the NationalCenter for Education Statistics, to further establish and adopt nationaldata standards that would enhancecommonalities across states for a morecomplete picture of instructionalresource allocation.

Some targeted improvements werecommend SEAs and other educationpolicymakers consider are below:

• Add school-level detail for instructional expenditures.

• Institute more accurate ways to get teacher years of experience data.

• Ensure teacher certification datacan be easily aligned to teachers’subject areas and grade levels.

• Create databases that link individual teachers to their students and classrooms.

• Enhance collection of data on thecosts, content, and quality of professional development.

Our research also highlighted vary-ing SEA data accessibility issues acrossthe states. As a result we recommendSEAs consider the following:

• Combine separate databases in acentralized data warehousehoused and managed by onedepartment.

• Establish clear procedures for datarequests, including the time andcost to provide the data.

• Find ways to share individual-leveldata while maintaining confidentiality.

• Have ample and knowledgeablestaff in place to assist data users.

• Provide documentation explainingdetails of the data, i.e., defini-tions, calculations, and year-to-year changes. Post this documen-tation on agency Web sites.

We recognize that SEA data arecollected for competing needs, such asfederal reporting, tracking stateaccountability goals, and supportingstate funding formulas. Consequently,reforming state education data systems takes careful planning andcollaboration between those who manage the data and those who usethe data. Critical to this process is forSEAs to balance the time and resourceburdens that changes to their datasystems create for schools, districts,and their own agency staff. Continuedwork remains to be done to ensurehigh-quality, user-friendly data areaccessible that can be used to answerimportant policy questions about education resources and studentachievement. Such efforts wouldsupport the creation of more reliableinformation needed for more effectivedecision making on the resourcesneeded to help children succeed.

ReferencesDorn, S. (2003). High-stakes testing

and the history of graduation.Education Policy Analysis Archives,11(1). Accessed October 3, 2005,from http://epaa.asu.edu/epaa/v11n1/

Goldin, C. (1999). A brief history ofeducation in the United States.(Historical Working Paper 0119).Cambridge, MA: National Bureau ofEconomic Research. RetrievedOctober 3, 2005, from http://ideas.repec.org/p/nbr/nberhi/0119.html

For a copy of SEDL’s researchreport, go tohttp://www.sedl.org/rel/IES-report.html.

10 Insights | Southwest Educational Development Laboratory

Reforming stateeducation data systems takes careful planningand collaborationbetween those whomanage the dataand those who usethe data.

Page 11: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

Author: Zena H. RudoContent Editors: Joan Buttram, Karen Herbert, Michael Vaden-Kiernan,Diane Pan, Tara Leo ThompsonCopy Editor: Shaila Abdullah, Laura Shankland Design and Layout: David Timmons

Insights on Education Policy, Practice, and Research may be reproducedand copies distributed to members by educational agencies, organiza-tions, and associations.

This publication was produced in whole or in part with funds fromthe Institute of Education Sciences, U.S. Department of Education,under contract number ED-01-CO-0009. The content herein does notnecessarily reflect the views of the U.S. Department of Education, anyother agency of the U.S. Government, or any other source.

Insights | Southwest Educational Development Laboratory 11

Using the SEA data collected fromArkansas, Louisiana, and Texas, weat SEDL are currently conducting a

study to investigate teacher resources andstudent achievement. Teacher resourcesincludes teacher salary, level of educa-tion, and years of experience. We are particularly interested in looking closelyat these teacher resources in high-needschools, i.e., schools in rural or urbanareas and schools with high studentminority and poverty enrollments.

The goals for this study are as follows:

• To learn about the extent to whichdistricts pay teachers based on yearsof experience and degree level

• To determine whether teacherresources are distributed differentlyacross schools depending upon theirlevels of need

• To see if funds expended on teacherresources are connected to studentachievement

Our study includes 191,813 core teachersand 6,618 public elementary and middleschools. We are using a variety of analysistools and the array of data collected toachieve these goals.

In our final study report, we will discussthe states’ use of a single salary schedulebased on teacher education and experi-ence and patterns of teacher resource allocations in schools. Our focus will beon helping policymakers in the threestates better understand the relationshipsbetween teacher salaries, teacher experience, teacher education, and student achievement.

For more information about this study,contact SEDL policy staff at 1-800-476-6861.

Additional information about SEDL’s policywork can be found on our Web site athttp://www.sedl.org/rel/pr_overview.html.

What Can the SEA Data Tell Us?

Page 12: Insights - SEDL · Insights | Southwest Educational Development Laboratory3 In the early twentieth century, when data collection was becoming more sophisticated, states were also

fAT A GLANCEIN

SIGHT

S Efforts to base education policy and practice on reliable data are increasing.The heightened need for answers about

the allocation of resources to improve student performance have prompted policymakers and researchers to ask, Can existing state education data support research that will provide this information?

This edition of Insights on Education Policy,Practice, and Research summarizes a recentSEDL study that explored what can be learnedfrom state education data systems about theallocation of instructional resources and howthis may impact student achievement.Specifically, this brief gives policymakers aclearer picture of the capacity and quality ofthe state data to answer questions aboutinstructional spending and teacher qualitybased on four states in the SEDL region.

Instructional resource data is available and accessible in separate state databases.• Detailed instructional expenditures for

school districts are in a fiscal database.• Average staff salary data are in a fiscal

database and individual salary data in astaffing database.

• Teacher quality data, e.g., teacher experience and education, and teachercharacteristic data, e.g., position andrace/ethnicity, are in a staffing database.

• Teacher certification data are in a distinctdatabase not generally linked to otherteacher data.

• School and district databases with demographics and other descriptive dataare often on state agency Web sites.

• Student databases include characteristics,program participation, and performancemeasures for each student and are highly confidential.

State data is highly useful for policymaking and research.• See how instructional dollars are spent and

how the spending varies across districts. • Find answers to questions about teacher

quality, its relationship to teacher salary,and its impact on student performance.

• Determine what staff resources compriseeach school and how those resources differacross schools with varying levels of student performance.

Ongoing reform of state education datasystems is needed.Collect new and improved data, includingschool expenditures; more accurate staff years of experience; teacher certification alignment to subject and grade being taught;and professional development costs, content,and quality.• Link data on individual teachers to

their students.• Combine fiscal, staff, school, and

student databases.• Improve data access, including clear

procedures for data requests and detaileddata documentation.

Southwest Educational Development Laboratory211 E. 7th Street, Suite 200Austin, Texas 78701 800-476-6861 http://www.sedl.org