The Integrity of Data Management Responsible Conduct of Research Workshop

Howard UniversityFebruary 8, 2011The Integrity of Data ManagementResponsible Conduct of Research WorkshopPeggy Carr, Ph.D.Associate CommissionerNational Center for Education Statistics

1Overview of Statistical and Research Standards2OMB Guidelines for Federal Statistical Agencies16 Federal Statistical Agencies adhere to Office of Management and Budget Guidelines to maintain high standards of performanceDevelopment of concepts and methodsPlanning and design of surveys and other means of collecting dataCollection of dataProcessing and editing of dataOMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, February 22, 2002, (67 FR 8752-8460)3Best Practices for the Conduct of Research4Best Practices for the Conduct of ResearchConceptualize Research ProblemOperationalize Study DesignEngage in Peer Review and Networking ActivitiesPublish and Publish Early/OftenBe Ethical51. Conceptualize Research Problem6Literature ReviewStatement of ProblemHypothesesResearch QuestionDefinition of Terms, Concepts, and ConstructsConceptualOperational2a. Operationalize Study DesignThe Concept of Study DesignHypothesis 1Explained VarianceUnexplained VarianceSampling ErrorMeasurement ErrorValidityReliabilityValidation of ConstructsFace Content validation across judgesCriterion Predictive and concurrentConstruct Methodological approaches to validation8JealousyEnvy2b. Operationalize Study Design2c. Operationalize Study DesignReliability of MeasuresTest and Retest stability over timeEquivalency consistency across formsInternal internal consistency

9UNRELIABLERELIABLE2c. Operationalize Study DesignResearch DesignInternal Validity refuting rival plausible hypothesesExternal Validity generalizability of findingsChoice of Designs experimental, quasi, survey, correlational10XYQuality Assurances of DataData Checks (e.g., out of range variance, criterion reference checks, replication)Thoughtful and Appropriate Data AnalysisThe nature of your dataLimitations of statistical techniqueDiscuss Findings and Conclusions with ColleaguesRelate back to literature reviewImplications for future researchFlaws and limitations11NEXT2c. Operationalize Study DesignTrend in fourth-grade NAEP mathematics average scores122c. Operationalize Study Design

132c. Operationalize Study Design

Trend in fourth-grade NAEP mathematics percentile scores4. Publish and Publish Early/OftenStart your publication record early in peer reviewed journalsAs first, second, or third author

Publish and/or present oftenRegardless of professional focusAnnually155. Be EthicalBe committed and follow through

Adhere toSubjects RightsInstitutional Review BoardConfidentiality and Privacy Rights16

Questions?17