ONSITE TRAC TRAININGMay 16, 2014

Joanne Mahonymahonyj@michigan.gov (517) 335-0405

TRAC OVERVIEW*Technical Review, Assistance, and Compliance

The purpose of TRAC is:To provide technical assistance, and Monitor for compliance with State and Federal laws

Each Region Every Five YearsAdded CostCTE PerkinsCTE Instructional Programs

*OVERVIEW OF TRAC PROCESS

MAJOR STEPSPre-visit preparation and desk reviewOnsite visitReport to regionCompliance plan submitted by regionCompliance plan and evidence approved by OCTE*

REVIEW CATEGORIESDataFinanceGrantsCTE Instructional ProgramsNOTE: The Office of Professional Preparation Services (OPPS) will join OCTE on TRAC reviews to complete OPPS review of Teacher Certification. More information will be given during OPPS Presentation this afternoon.

*

WARNING: Submission of documents that are subject to confidentiality and privacy laws should NOT be submitted electronically. These documents will be reviewed onsite.*

QUESTIONS?

Elizabeth Collinscollinse2@michigan.gov (517) 335-3149

TRAC TIMELINE (OCTE)Whats happened:Region selection Onsite visit notificationTRAC trainingWhats ahead:Grant activity selectionsFiscal agency selectionsCTE instructional program selections *

TRAC TIMELINE (Region)Document Submission for Desk Review:CIP Self-Review documentation for selected programs is submitted through NavigatorData, Finance, and Grant documents submitted through GEMS (checklists)All documents are due September 8, 2014Agenda (due three weeks prior to onsite)*

TRAC TIMELINEDay 1 - Program reviews beginDay 2 - Program reviews continueIntroduction meetingData/Finance/Grant reviews beginDay 3 Program reviews continueInterviewsWrap Up meeting*

TRAC TIMELINEDesk Review and Onsite Visit OCTE Report Of Findings in GEMSNot In Compliance - Action RequiredRecommendations (comments, technical assistance, observations)Commendations (exemplary program, best practices)*

TRAC TIMELINE

Appeals Process Compliance Plan (created by Region) Compliance Plan and Evidence (Approved by OCTE) Final Letter

*

IMPORTANT CHANGES Expanded State Desk ReviewDocuments RequiredNo Compliance Plan WorkshopNo Priority (I/II) FindingsImplementation of GEMSNavigator CIP Programs

*

QUESTIONS?

Joanne MahonymahonyJ@michigan.gov(517) 335-0405

GRANTSALL desk review documents for Grant review must be uploaded to GEMS by September 8, 2014.Personally identifiable information must be redacted from documents before upload or documents must be provided onsite.

*

GRANTS G01 20% of CPI Indicator Activities from 2013-14 applicationFactors considered in selection process:CPI DeficienciesRegional Improvement Plan ActivitiesAmount of grant funds budgetedRelevancy of activity

*

GRANT G01 DOCUMENTATIONEvidence as identified and approved in the application must be providedEvidence must reflect the expected outcomes as identified in the applicationEvidence must be consistent with the scope of the activity and outcome*

GO1 Evidence ExamplesG01 evidence may include: Meeting minutes and sign-in sheetsPre- and post-test scoresRevised curriculumPlanning documentsDocumentation of services provided to students (payroll records and work logs)Dated vendor paymentsOther*

EXAMPLEEXPECTED OUTCOME

CPI: 1S1 Academic Attainment/ReadingActivity: Curriculum Integration

Expected Outcome: 60% of the region's CTE teachers will participate in professional development related to the integration of reading concepts and will integrate contextual reading activities into curriculum to support student growth in reading.*

EXAMPLE*

G01 - EvidencesProvide Regional/CEPD/District documents (as appropriate) listed under DESK REVIEW for review items (participation may have been at the regional, CEPD or district level).

No personally identifiable information*

*Evidences Supporting:

Regional Long Range PlanEligibility for Perkins Funding

(G02 G06)

G02 - Equity and Access Documents that demonstrate:Students have equal access to CTE programsAll students are included in outreach activitiesCompliance with federal law and regulations (includes eligible charter schools and home-schooled students)*

G02 - EvidencesEvidences may include:Counseling, advising and admission policies and procedures demonstrating equal accessOutreach materials which may include brochures, videos, letters or flyers (including ELL languages as applicable)

*

G03 Regional Planning/StakeholderDocuments that demonstrate:Grant priorities Joint planning and coordination of CTE programsWDB/EAG was provided the opportunity to review and provide input

*

G04 Special Population ServicesDocuments that demonstrate:Region/CEPD/District plan is in place to identify and provide services to SPOPS (LRP)Region/CEPDs/Districts have assessed and met the needs of SPOPSRegion/CEPDs/Districts have conducted outreach targeting SPOPS

*

G05 Professional DevelopmentDocuments demonstrating Professional Development:Secondary teachers and personnelAdministratorsCareer Guidance and Academic Counselors

*

G06 Nonpublic and Charter SchoolsDocuments that demonstrate compliance with federal laws and regulations regarding participation by students, teachers and administrators in:Nonprofit, nonpublic schools and home-schooled studentsCharter schools

*

G07 Previous OnsiteOnly provide documents if requested after the desk review is completed.*

David MacQuarriemacquarried@michigan.gov(517) 241-6202

Krishnan Sudharsansudharsank@michigan.gov(517) 241-7652

Valerie Felderfelderv@michigan.gov(517) 335-1066

*Focus of Data Review

DATA - Common FindingsCourse section minutes or weeks reported in CTEIS and shown on X0107 report, do not match district documentation:Master scheduleBell scheduleDistrict calendar

Failure to provide bell schedule, master schedule or district calendar for ALL PSNs specified by OCTE for review

*

DATA - Common FindingsCourse section titles often dont match those in CTEIS and shown on the X0107 report.Could result in findings of noncomplianceIf master schedule course names do not match course section names in CTEIS, please provide a crosswalk showing both course names

Enrollments reported to OCTE dont match teacher class/course records

Please be sure to provide documents for the year under review (usually prior school year)

*

DATA - Common FindingsDocumentation not provided to demonstrate that:CPI reports were shared with instructorsCPI reports were used for program improvementFollow Up Survey results, including related placement data shared with instructors

(Examples of adequate evidence include meeting minutes or memos)

*

DATA - Common FindingsData processes and procedures documentation such as data security procedures, not provided

*

DATA - Common FindingsEvidence not provided for all CEPDs or districts in Region.PSN-specific documentation is required for data reported at the course-section level (and is only required for the specified PSNs):Examples:Class listsInstructional DesignMinutesWeeks

*

DATA - Common FindingsEvidence not provided for all CEPDs or districts in Region (continued):Documentation demonstrating effort across the entire region is required for non-PSN-specific data activities:Examples: Sharing CPI results, Sharing follow-up results, Data security proceduresDocumentation must be provided demonstrating that activity was carried out at the regional level or, if carried out at CEPD or district level, a representative sample of documentation must be provided for every CEPD.*

TIPS FOR REVIEW ITEMSD01-Perkins quality data:Provide teacher class lists for specified PSNs on-site (do NOT upload Personally-Identifiable Information (PII) to GEMSProvide PII onsiteExamples: documents with student names, UICs, information on individual students, IEP (even without name)*

TIPS FOR REVIEW ITEMSD02-Accurate CPI data:Teacher class lists for specified PSNs (see D01). Provide onsite only. Do not upload to GEMS.List of Students showing Special Populations category From DISTRICT records (not exported from CTEIS).Provide onsite only. Do not upload to GEMS.*

TIPS FOR REVIEW ITEMSD02-Accurate CPI data:Teacher instructional design document for every PSN identified by OCTE (not printed from CTEIS)Articulation agreements for PSNs identified by OCTERegional list of PSNs that have articulation agreements

*

TIPS FOR REVIEW ITEMSSee review checklist for recommended evidence. Findings of noncompliance are common in:D03-CPI Reports/CPI ActivitiesD04-Added Cost Quality DataTip: provide counts of days of instruction on calendars and minutes of instruction on schedules.

*

TIPS FOR REVIEW ITEMSSee review checklist for recommended evidence. Findings of noncompliance are common:D05-Placement data (follow-up survey)Tip: Remember to provide evidence for:Total placementRelated placementCPIs used for program improvementCPIs shared with staff*

TIPS FOR REVIEW ITEMSD06-CPI Report availabilityTip: Provide web links to reports for all districts in the regionD07-Data securityTip: Provide district-level policy documents for all districts in region.Tip: Agendas, meeting minutes or memos can be used to demonstrate that policies and procedures were shared with staff.

*

TIPS FOR REVIEW ITEMSD08-Previous monitoring reviewOnly provide evidence if requested from OCTE after desk reviewTip: Review prior TRAC Report to avoid repeat findings.*

Naming Evidence Files In GEMS

Label each file uploaded to GEMS with:Review item code (example: D02)PSN (for PSN-specific evidence)-or- District (for District-level evidence)-or- CEPD (for CEPD-level evidence)Description: (example: InstructDesign)EXAMPLE FILE NAME:Example: ReviewItemCode_PSN_DescriptionPSN example: D02_12509_InstructDesignBuilding example: D04_Randolph_MasterScheduleDistrict example: D04_DetroitPS_DistrictCalendarCEPD example: D03_CEPD41_AgendaWhereCPIsShared

*

For Evidence Provided OnsiteDont forget to use:A sticky note on a pageHighlight a sentence, paragraph, etc.Draw an arrow to the sentence, picture, etc.Provide crosswalks between district documents and or with CTEIS reports, if different*

QUESTIONS?

BREAK TIME!Pleasereconvene in15 minutes

Christine Blackblackc11@Michigan.gov(517) 241-0260

FINANCE AGENCY REVIEWS2013-2014 CTE Perkins fiscal agencyand1-3 additional participating agencies*

F01 Level of Effort: Supplement Not SupplantDocumentation demonstrating that federal funds were used to supplement, not supplant, nonfederal funds expended to carry out career and technical education activities.

*

F01 - EvidencesDesk Review documents:Board policy prohibiting supplantingOnsite documents:Interview with business officialInterview with CTE administratorOCTE documentation*

F02 - ProcurementDocumentation that region maintains a written code of standards of conduct governing the performance of its employees engaged in the award or administration of contracts; and maintains records sufficient to detail the history of a procurement transaction; and procurement procedures provide for full and open competition to the extent practicable, and no contract is made with a vendor that has been suspended or debarred.

*

F02 - Evidences

Desk Review documents:Procurement/purchasing policies and procedureBoard-approved Code of Conduct or Conflict of Interest policy for employees and officers involved in procurementOnsite documents:Evidence that the System for Award Management was checked ( www.sam.gov )Employee-signed copy of Code of Conduct or Conflict of Interest (required if Board-approved Code of Conduct or Conflict of Interest policy is not evident)Interview with business officialInterview with CTE administratorOCTE documentation

*

F03 Expenditure ReviewDocumentation demonstrating fiscal control and fund accounting policies and procedures ensure proper expenditure of and accounting for federal and nonfederal funds expended for career and technical education.

*

F03 Evidences

Desk Review documents:General ledger summary and transaction details (or equivalent) for federal CTE grant expendituresGeneral ledger summary and transaction details (or equivalent) for nonfederal expenditures used to support required CTE Perkins grant activitiesBudget policies and proceduresAccounting policies and proceduresProcess, procedures and forms used to monitor reimbursements to participating agencies (fiscal agency only)Chart of accounts (or equivalent) that defines accounting codes used in general ledger

*

F03 Evidences Continued

Onsite documentation:Supporting documentation for selected sample.Signed Cooperative Agreements for Designation of Fiscal Agency for ALL participating agencies generating CTE Perkins funds (fiscal agency only)Interview with business officialInterview with CTE administrator or CTE Perkins federal grant coordinatorOCTE documentation*

F04 Time and Effort ReportingDocumentation demonstrating employees whose salaries and benefits are paid in whole or in part with federal funds maintain time distribution and personnel activity reports (PARS) in accordance with OMB Circular A-87.

*

F04 Evidences

Desk Review documents:List of employees and titles whose salary and benefits were paid in whole or in part with CTE Perkins funds.Payroll ledgerPolicy and procedures complying with federal time and effort reporting requirements

*

F04 Evidences ContinuedOnsite documentation:Reconciliation of actual payroll costs to budgeted amountsSemi-annual certifications (100%)Personnel Activity Reports for split-funded personnelInterview with business officialInterview with CTE administrator or CTE Perkins federal grant coordinatorOCTE Documentation

*

PAR vs. Semi-Annual CertificationThere are only four (4) possible options to consider when determining whether to complete a semi-annual certification or personnel activity report. All possible circumstances will apply to one of the four options.

*

Timekeeping Decision Making Rubric*

F05 Property ManagementDocumentation demonstrating prior written approval is required for purchasing equipment for state-approved career and technical education programs and that recipient's inventory management system meets the requirements in EDGAR. ONLY applicable if equipment was purchased with 2013-2014 Perkins funds.Equipment is defined as tangible, nonexpendable property having a useful life or more than one year and an acquisition cost of $5,000 or more per unit.

*

F05 - EvidencesDesk Review documentation:Equipment inventoryBoard policy for capital outlay purchases (if not submitted for F02)Onsite documentation:Accounting records for instructional equipment purchasesInterview with business officialInterview with CTE administrator or CTE Perkins federal grant coordinatorOCTE documentation

*

F06 - Previous Onsite Monitoring VisitProvide evidence for F06 only if requested by OCTE after the desk review.*

FINANCE INTERVIEWSInterviews as needed with:Business office representativesCTE directors or administrators*

FINANCE CHECKLIST Provide DISTRICT documents listed under DESK REVIEW for all review items Due to OCTE on or before September 8, 2014ONSITE documents should be provided at the site.

*

QUESTIONS?

DESK REVIEW DOCUMENTSBefore Onsite VisitCollect and organize materials electronicallyVirtual folders to prepare for GEMSLabel with review item number (D01, F01, G01, etc.) and Document Name including CEPD, PSN, Building as necessaryMark pages/highlight sections (if possible)Provide all requested documentationSubmit Desk Review documentation in GEMS on or before September 8, 2014.*

Example of Virtual Folders*

GEMS AccessEach Region Liaison will need access to GEMS. This access is granted by your regions GEMS Sub-recipient administrators. Information about this was sent to each Region Liaison via email on May 13, 2014.*

GEMS Demonstration*

ONSITE DOCUMENTS Only one copy of each itemDocuments should be placed in the document review room at the homebase location for day two of the agendaMaterials remain with region

*

FINAL REMINDERSDocumentation should represent the whole Region - not just one CEPD or district unless otherwise specifiedDont treat each CEPD separately; put all evidence for each review item togetherHave someone available on day two and three of the onsite visit who knows the documentsWatch for program and grant selection notification via email on June 2, 2014.

*

Onsite Visits Dates

Please see the TRAC Schedule in your folder for your regions scheduled visit date.*

OCTE TRAC COORDINATOR

Elizabeth Collins (517) 335-3149collinse2@michigan.gov

*

QUESTIONS?

Debbie RyanOffice of Professional Preparation Servicesryand@michigan.gov517-335-0585

Norma Timstimsn@michigan.gov517-241-2091

Program Selection CriteriaAdded Cost funding amountsEnrollment and completion dataGeographical locationNew program established within last five yearsNew teacher for programOCTE program file informationProgram and building level core performance indicator data including special populationsRegional CIP Summary information and CIP Improvement Plans

*

Program Selection NotificationOCTE notifies Regional TRAC Liaison Regional TRAC Liaison notifies CEPD administratorsCEPD administrator notifies building administrator, teacher(s), and OCTE consultantTeacher uploads all information into MCCTE Navigator by September 8, 2014OCTE consultant contacts teacher(s) to finalize visit details and copies the CEPD administrator, building administrator, Regional TRAC Liaison, and OCTE TRAC Coordinator*

Program Selection Follow-Up2014-15 Master schedule highlighting program teacher(s) scheduleProgram Location - Building addressExceptions - Notation of teacher on leave, retired; program relocated, closed, etc.

*

CIP Self-ReviewComplete and upload in Navigator by PSN (http://ctenavigator.org)Single review for each PSN-all teachers Evidence for 2013-2014 school yearMust be complete by September 8, 2014

*

CO1 Program Teacher Certification Enter legal name(s) exactly as printed on teaching credential(s)Upload: Copy of state professional license, if applicableCopy of industry certification, if applicableCopy of Community College Instructor letter, if applicable

*

CO2 - Program Teacher PDTo stay current with all aspects of program-related industry

That promotes the integration of coherent and rigorous content aligned with challenging academic standards and relevant CTE.Effective integration and use of challenging academic in CTE provided jointly with academic teachersEffective research-based teaching skillsEffective practices to improve parental and community involvementEffective use of research and data to improve instructionProvide relevant experience in the related business or industry (internship)Training in the effective use and application of technology to improve instruction.*

Upload Professional Development RecordsTo show:DateSponsor(s)Topic(s) or focus of Professional Development

*

CO6 - Program Advisory CommitteeUpload:Advisory Committee roster-appropriate businesses, parent, and postsecondaryDocumentation of two meetings for year.Evidence of Advisory Committee assessment of and input on facilities, equipment, materials, and resources that reflect current industry standards and technology.

*

C10 Implementation of Program Standards/Course ContentUpload:Program Gap AnalysisSyllabi for all courses within the programProgram Instructional Design Form (Segments)Record of safety training (CIP specific)Documentation of work-based learning opportunities for students (CIP specific)Documentation of student leadership opportunities *

Upload:Program Facilities (evidence of accessibility)Bias review resultsRecruitment materials/brochuresProgram applications/prerequisitesEnrollment demographics (over/under representation of sub groups)*C13 Strategies to Eliminate Barriers to Program Access

Upload one or more of the following: Program of Study (CIP specific)Articulation AgreementConcurrent/Dual enrollment agreement, if applicableDirect Credit agreement, if applicable *C16 Secondary-Postsecondary Connections

QUESTIONS?

ONSITE TRAC TRAININGMay 16, 2014

Handout #14We need to be able to determine the total number of minutes programs were delivered. To do that we need a master calendar which will tell us the number of weeks of delivery, and the bell schedule including the actual times to compute minutes per week.We need to be able to determine the total number of minutes programs were delivered. To do that we need a master calendar which will tell us the number of weeks of delivery, and the bell schedule including the actual times to compute minutes per week.Documentation must either demonstrateDocumentation must either demonstrateHandout #13The required documents are listed on the checklists for Data, Finance, Grants, LTCS, and CIP Self Review in your TRAC handouts. Again, ALL of the grant items are due for desk review. Reminder: The Compliance Plan Evidence of Completion must be kept onsite for six years. This evidence is not the documentation you submitted for the desk review or onsite reviews.