Commission on aCCreditation

implementing regulations

Section C: IRs Related to the Standards of

Accreditation for Doctoral Graduate Programs for Internship Programs for Postdoctoral Residency Programs

Section C: Doctoral Programs IR Name Old # New # SoA location Conduct of Doctoral Reviews C-31 C-1 D N/A Definition of "Developed Practice Areas" for Doctoral Programs and the Process by which Areas May be Identified as Such C-14 C-2 D I.A.2 Review of Applications for the Recognition of Developed Practice Areas C-14(a) C-3 D I.A.2 Appeal of Decisions for Areas Seeking to be added to the Scope of Accreditation as Developed Areas C-14(b) C-4 D I.A.2 Academic Residency for Doctoral Programs C-2 C-5 D I.C.2 Record of Student Complaints in CoA Periodic Review C-3 C-6 D I.D.4 Discipline-Specific Knowledge C-7 D II.B.1.a Profession-Wide Competencies C-8 D II.B.1.b Diversity Education and Training C-23 C-9 D II.B.1.b Positive Student Identification Consistent with Higher Education Opportunity Act C-25 C-10 D II.B.2 Distance and Electronically Mediated Education in Doctoral Programs C-27 C-11 D II.B.2 Practicum Guidelines for Doctoral Programs C-26 C-12 D II.B.3 Telesupervision C-28 C-13 D II.B.3 Direct Observation C-14 D II.B.3.d

Awarding the Doctoral Degree Prior to Completion of the Internship C-5 C-15 D II.B.4 Affiliated Internship Training Programs C-10 C-16 D II.B.4 Expected Internship Placements for Students in Accredited Doctoral Programs C-31(c) C-17 D II.B.4 Outcome Data for Doctoral Programs C-32 C-18 D II.D.1 Licensure Rates for Doctoral Programs C-31(d) C-19 D II.D.1.b Selection and Admission of Students into Accredited Doctoral Programs C-31(a) C-20 D III.A Diversity Recruitment and Retention C-22 C-21 D III.A.1.b Student Attrition Rates for Doctoral Programs C-31(b) C-22 D III.C.1 Faculty Qualifications C-23 D IV.B.2 Program Names, Labels, and other Public Descriptors C-6(a) C-24 D V.A Accreditation Status and CoA Contact Information C-6(b) C-25 D V.A.1.b

Disclosure of Education/Training Outcomes and Information Allowing for Informed Decision-Making to Prospective Doctoral Students C-20 C-26 D V.A.2.c Notification of Changes to Accredited Programs C-19 C-27 D V.B.2 “Intent to Apply” C-28 D N/A “Accredited, on Contingency” C-29 D N/A Partnership/Consortium C-30 D N/A

C-1 D. Conduct of Doctoral Reviews (formerly C-31; Commission on Accreditation, July 2012; revised November 2015)

A number of programs have sought clarification regarding how the CoA reviews key markers of student progress that are assessed both at the time of the full CoA review as reflected in decision letters and during the annual review process based upon data provided by programs in completing the Annual Report Online (ARO). The four key markers of student progress are: IR C-17 D: Internship Placement; IR C-19 D: Licensure; IR C-20 D: Student Selection and Admission; and IR C-22 D: Student Attrition. For each of these issues, the CoA reviews programs for their quality in a comprehensive manner through the review of the self-study and site visit, as well as monitoring continued adherence to providing educational quality. These two processes - periodic review and annual review - are discussed below. Periodic review - In reaching a decision about the accreditation status of a program, the CoA does not have a set number of issues or concerns that automatically leads to an adverse decision; rather the professional judgment of the Commission is based on the overall review of the program's adherence to the accreditation standards - the SoA and related Implementing Regulations. In making an accreditation decision, the CoA looks at the program's entire record to determine whether or not, as a whole, the program achieves an appropriate level of quality to be accredited, and that it meets its own stated aim(s). The SoA do allow for some flexibility in the professional judgment of the CoA based upon the program's stated aim(s). However, the purpose of the CoA's accreditation review at the doctoral level is to evaluate "preparation for entry-level practice in health service psychology, regardless of the program's aims. At this time, "entrance to the profession" involves the completion of the doctoral program in a timely manner and attainment of licensure. In addition, review of licensure rates is required by the U.S. Department of Education. As a result, for the purposes of evaluating entrance to the profession, the CoA evaluates the proportion of students entering a doctoral program who complete it, the time-to-degree, and the proportion of students completing the doctoral program who attain licensure. Annual review - In its annual monitoring of accredited doctoral programs (as articulated in Implementing Regulation D 4.7), the CoA has set a series of parameters by which it annually reviews programs’ adherence to general quality assurance indicators each year. At this time these include: internship placement; time to degree; annual attrition within the student body; and changes in core faculty as related to total students in the program. The CoA requires programs to provide annual report data each year, and uses these data to monitor program quality indicators during those years the program is not engaged in periodic review. Thus, if a program meets the IR D.4-7 threshold as determined by the Annual Report Online (“ARO”) in a given year, it means that the program does not need to provide additional reports on that specific threshold in that year. It is important to understand that meeting these thresholds simply means that the program's reported data will not trigger a fuller review in connection with the annual report. This does not mean that these outcome data will dictate reaccreditation during the periodic review, which is based on a more comprehensive analysis of the program, including a broader review of the data, the program's outcomes, and other factors bearing on the program's consistency with the Standards of Accreditation.

C-2 D. Definition of “Developed Practice Areas” for Doctoral Programs and the Process by which Areas May be Identified as Such

(formerly C-14; Commission on Accreditation, October 2006; pursuant to changes in the scope of accreditation approved by the APA Council of Representatives in August 2006; revised November 2015) Scope of Accreditation for Doctoral Programs: The Commission on Accreditation reviews doctoral programs in psychology that provide broad and general training in clinical psychology, counseling psychology, school psychology. And other developed practice areas. The CoA also reviews programs that combine two or three of the above-listed practice areas. Definition Developed practice areas of psychology have all of the following characteristics:

• National recognition of the practice area by a national organization(s) whose purpose includes recognizing or representing and developing the practice area, by relevant divisions of the APA, or by involvement in similar umbrella organizations;

• An accumulated body of knowledge in the professional literature that provides a scientific basis for the practice area including empirical support for the effectiveness of the services provided;

• Representation by or in a national training council that is recognized, functional, and broadly accepted;

• Development and wide dissemination by the training council of doctoral educational and training guidelines consistent with the SoA;

• Existence of the practice area in current education and training programs; • Geographically dispersed psychology practitioners who identify with the practice area and provide

such services.

Process Steps in the identification process are:

1. Application by the training council will be initially reviewed by the CoA based upon the criteria defined above to determine the eligibility of the area for public comment on its inclusion;

2. If in this initial review, the area meets the criteria for eligibility, the CoA will invite subsequent public comment as well as inviting letters of support or concern from relevant organizations;

3. Final decision by the CoA. 4. In the case of a decision to not include the area in the scope of accreditation, the training council

may file an appeal using an appeal process parallel to the current procedures for the appeal of program-level decisions. Specific procedures for that appeal will be developed.

(See Implementing Regulation B-2 for more information about changes in the scope of accreditation)

C-3 D Review of Applications for the Recognition of Developed Practice Areas (formerly C-14(a); Commission on Accreditation, October 2007; revised October 2008, November 2015)

A program cannot be reviewed for accreditation in a developed practice area until that area has been added to the scope of accreditation. An area applying for recognition must first demonstrate training in that area at the doctoral level before programs will be recognized in that area at the internship level. Application Areas seeking to become included in the scope of accreditation must provide all information requested in the application, which is available from the Office of Program Consultation and Accreditation. Applications not following the required format will be returned without review. Staff members of the Office of Program Consultation and Accreditation will confirm receipt of the application and ensure that all required information has been provided. Staff members may request the submission of any missing information, and the application will not be reviewed by the CoA until all required materials have been provided. Areas may submit their applications at any time. However, in order to be reviewed during a specific CoA meeting, applications must be received at least 2 months prior to that meeting. A list of CoA meeting dates is available at http://www.apa.org/ed/accreditation/calendar.aspx. Applications received after that deadline will be reviewed during the next available meeting. Review Upon receipt of the area’s completed application materials, the Executive Committee of the CoA will be charged with the review of the application. The Executive Committee maintains the right to seek additional consultation and expertise in the area as necessary. Based upon its review of the record, the Executive Committee will develop a recommendation for action by the full CoA. If the full CoA believes the area meets the criteria outlined in Implementing Regulation C-2 D, then the CoA will invite public comment on inclusion of the area in the scope of accreditation as a Developed Practice Area. After review of any public comments, the CoA will make its final decision on inclusion of the area as a Developed Practice Area. However, if the area wishes to be specified by name as part of the scope of accreditation, then the application and CoA recommendation will be forwarded to the APA Council of Representatives for review.

C-4 D. Appeal of Decisions for Areas Seeking to be added to the Scope of Accreditation as Developed Practice Areas

(formerly C-14(b); Board of Educational Affairs, November 2007) A decision by the CoA not to recommend an area for inclusion in the scope of accreditation as a Developed Practice Area may be appealed to the APA Board of Educational Affairs using the process outlined for appeals of program review decisions (see Implementing Regulations D5-1 and D5-2). The Chief Executive Officer of the group or training council petitioning for recognition of the area, or the responsible administrative officer of the group may challenge a CoA decision not to recognize a proposed Developed Practice Area. Such an appeal must be received within 30 days of receipt of written notice of the CoA decision. The appeal must specify the grounds on which the appeal is made, which must be either a procedural violation or substantive error by the CoA in its review of the area’s consistency with the provisions of Implementing Regulation C-2 D. The appeal should be addressed to the President of the APA. A nonrefundable appeal fee will be charged to the appellant group or training council, such fee to be submitted with the letter of appeal. Appointment of Appeal Panel Within 30 days of receipt of the area’s letter of appeal, the APA Board of Educational Affairs will provide the group or training council with a list of six potential appeal panel candidates, no one of whom will have had affiliation with the proposed Developed Practice Area filing the appeal or with the accreditation process related to the non-recognition of the area. The Office of Program Consultation and Accreditation will determine the willingness of the potential panel members to serve, and notify the group or training council to that effect. Within 15 days, the group or training council may select three panel members from this list to serve as its appeal panel. If the group or training council does not notify the Office of Program Consultation and Accreditation of its selection within 15 days, the Board of Educational Affairs will designate three members to serve on the appeal panel. Scope and Conduct of Appeal An appeal is not a de novo hearing, but a challenge of the decision of the CoA based on the evidence before the CoA at the time of its decision. The CoA’s decision should not be reversed by the appeal panel without sufficient evidence that the CoA’s decision was plainly wrong or without evidence to support it. Accordingly, the appeal panel should not substitute its judgment for that of the CoA merely because it would have reached a different decision had it heard the matter originally. The procedural and substantive issues addressed by the appeal panel will be limited to those stated in the area’s appeal letter. If an issue requires a legal interpretation of the CoA’s procedures or otherwise raises a legal issue, the issue may be resolved by APA legal counsel instead of the appeal panel. Only the facts or materials that were before the CoA at the time of its decision may be considered by the panel. The panel will be provided with only those documents reviewed by the CoA in making its decision, the letter that notified the group or training council of the CoA’s decision, the letter of appeal, written briefs submitted by the group or training council, and reply briefs submitted by the CoA. The letter of appeal and written briefs shall not refer to facts or materials that were not before the CoA at the time the decision was made. The appeal panel will convene a hearing at APA during one of three pre-scheduled appeal panel hearing dates. In addition to the three members of the appeal panel, the appeal hearing will be attended by one or

more representatives of the group or training council representing the proposed Developed Practice Area, one or more representatives of the CoA, and staff of the Office of Program Consultation and Accreditation. APA’s legal counsel will also attend the hearing. In addition to advising APA, counsel has the responsibility to assure compliance with the above procedures and may resolve legal or procedural issues or can advise the panel regarding those issues. Decision and Report of Appeal Panel The CoA’s decision should be affirmed unless (a) there was a procedural error and adherence to the proper procedures that would dictate a different decision; or (b) based on the record before it, the CoA’s decision was plainly wrong or without evidence to support it. The appeal panel has the options of: (a) upholding the CoA decision; or (b) returning the matter to the CoA for reconsideration of its decision in light of the panel’s ruling regarding procedural violations or substantive errors. The report of the appeal panel will state its decision and the basis of that decision based on the record before the panel. The report of the panel will be addressed to the President of the APA and sent within 30 days of the hearing. Copies will be provided to the Chief Executive Officer or to the responsible administrative officer of the group or training council whose appeal was heard, the Chair of the CoA, the Chair of the Board of Educational Affairs, and the Office of Program Consultation and Accreditation.

C-5 D. Academic Residency for Doctoral Programs (formerly C-2; Commission on Accreditation, July 2007; revised November 2015)

The doctorate is the highest degree of educational accomplishment in health service psychology. The level of sophistication in thought and behavior required for the degree is attained in part through full-time study in residence at an institution of doctoral education. To this end, the Standards of Accreditation (Standard I.C.2) requires of each student:

a. A minimum of 3 full-time academic years of graduate study (or the equivalent thereof) plus an internship prior to receiving the doctoral degree;

b. At least 2 of the 3 academic training years (or the equivalent thereof) within the program from which the doctoral degree is granted;

c. At least 1 year of which must be in full-time residence (or the equivalent thereof) at that same program. Programs seeking to satisfy the requirement of one year of full-time residency based on “the equivalent thereof” must demonstrate how the proposed equivalence achieves all the purposes of the residency requirement.

Residency has two primary purposes: student development and socialization, and student assessment. With regard to student development, residency allows students (1) to concentrate on course work, professional training and scholarship; (2) to work closely with professors, supervisors and other students; and (3) to acquire the attitudes, values, habits, skills, and insights necessary for attaining a doctoral degree in psychology. Full-time residence provides students other opportunities, including obtaining fluency in the language and vocabulary of psychology as enhanced by frequent and close association with, apprenticing to, and role modeling by faculty members and other students; obtaining valuable experience by attending and participating in both formal and informal seminars; colloquia; discussions led by visiting specialists from other campuses, laboratories, or governmental research and/or practice organizations; and, obtaining support in thesis, dissertation, or doctoral project work through frequent consultations with advisors. An equally important purpose of the residency requirement is to permit faculty, training staff, supervisors, and administrators to execute their professional, ethical, and potentially legal obligations to assess all elements of student competence. Executing these obligations is an essential aspect of assuring quality and protecting the public. These elements include not only student-trainees' knowledge and skills, but also their emotional stability and well-being, interpersonal competence, professional development, and personal fitness for practice. Through such student assessment, accredited programs can ensure—insofar as possible—that their graduates are competent to manage relationships (e.g., client, collegial, professional, public, scholarly, supervisory, teaching) in an effective and appropriate manner. This capacity for managing relationships represents one of the competencies that define professional expertise. Programs seeking to satisfy the requirement of one year of full-time residency based on "the equivalent thereof" must demonstrate how the proposed equivalence achieves all of the purposes of the residency requirement, as articulated above. In evaluating whether the residency requirement is satisfied, the Commission will consider processes and indicators related to the elements of student development and socialization and student assessment detailed in paragraphs 2 and 3 of this Implementing Regulation. Note: The above statement on the purpose of full-time residency is drawn substantially from the Policy Statement of the Council of Graduate Schools titled "The Doctor of Philosophy Degree" (Council of Graduate Schools, 2005), the statement of the Council of Chairs of Training Councils (December, 2003) titled “Comprehensive Evaluation of Student Competence,” and the APA Policy Statement on Evidence-Based Practice in Psychology (August, 2005).

C-6 D. Record of Student Complaints in CoA Periodic Review (formerly C-3; Commission on Accreditation, October 1998; revised November 2015)

Standard III.B.2 of the Standards of Accreditation for doctoral programs addresses the need for accredited programs to recognize the rights of students to be treated with courtesy and respect, to inform them of the principles outlining ethical conduct of psychologists, and to ensure that they are aware of avenues of recourse should problems with regard to these principles arise. In accordance with Standard III.B.2 of the Standards of Accreditation for doctoral programs, a program is responsible for keeping information and records of all formal complaints and grievances, of which it is aware, filed against the program and/or against individuals associated with the program since its last accreditation site visit. These records will be reviewed by the Commission on Accreditation (CoA) as part of its periodic review of programs. The CoA expects a program to keep all materials pertaining to each of the complaints/grievances filed against it during the aforementioned time period. The site visitors shall review the full record of program materials on any or all of the filed complaints/grievances.

C-7 D. Discipline-Specific Knowledge (Commission on Accreditation, November 2015; revised, July 2017)

Discipline-specific knowledge serves as a cornerstone of identity as a psychologist and orientation to health service psychology. Therefore, all students in accredited doctoral programs shall demonstrate knowledge in the discipline of psychology, broadly construed. This discipline-specific knowledge base shall include: 1) the history and systems of psychology, 2) basic knowledge in scientific psychology, 3) integrative knowledge in scientific psychology, and 4) methods of inquiry and research. Discipline-specific knowledge, as it is articulated in the Standards of Accreditation (Doctoral Standards, II.B.1.a):

a. Discipline-specific knowledge represents the requisite core knowledge of psychology an individual must have to attain the profession-wide competencies. Programs may elect to demonstrate discipline-specific knowledge of students by:

i. Using student selection criteria that involve standardized assessments of a

foundational knowledge base (e.g., GRE subject tests). In this case, the program must describe how the curriculum builds upon this foundational knowledge to enable students to demonstrate graduate level discipline-specific knowledge.

ii. Providing students with broad exposure to discipline-specific knowledge. In this case, the program is not required to demonstrate that students have specific foundational knowledge at entry, but must describe how the program's curriculum enables students to demonstrate graduate-level discipline-specific knowledge.

For purposes of this Implementing Regulation, there are four categories of discipline-specific knowledge.

Category 1: History and Systems of Psychology

• History and Systems of Psychology, including the origins and development of major ideas in the discipline of psychology.

Category 2: Basic Content Areas in Scientific Psychology.

• Affective Aspects of Behavior, including topics such as affect, mood, and emotion. Psychopathology and mood disorders do not by themselves fulfill this category.

• Biological Aspects of Behavior, including multiple biological underpinnings of behavior, such as neural, physiological, anatomical, and genetic aspects of behavior. Although neuropsychological assessment and psychopharmacology can be included in this category, they do not, by themselves, fulfill this category.

• Cognitive Aspects of Behavior, including topics such as learning, memory, thought processes, and decision-making. Cognitive testing and cognitive therapy do not, by themselves, fulfill this category.

• Developmental Aspects of Behavior, including transitions, growth, and development across an individual’s life. A coverage limited to one developmental period (e.g., infancy, childhood, adolescence, adulthood, or late life) is not sufficient.

• Social Aspects of Behavior, including topics such as group processes, attributions, discrimination, and attitudes. Individual and cultural diversity and group or family therapy do not, by themselves, fulfill this category.

Category 3: Advanced Integrative Knowledge in Scientific Psychology.

• Advanced Integrative Knowledge of Basic Discipline-Specific Content Areas, including graduate-level scientific knowledge that entails integration of multiple basic discipline-specific content areas identified in Category 2 (i.e., integration of at least two of: affective, biological, cognitive, social, or developmental aspects of behavior). Advanced integrative knowledge in Category 2 areas can be acquired in either of two ways: 1) an evaluated educational1 experience that integrates at least two Category 2 content areas that have been previously covered through other methods; or 2) an evaluated educational experience that provides basic coverage in two or more areas and integration across those areas.

Category 4: Research Methods, Statistical Analysis, and Psychometrics

• Research Methods, including topics such as strengths, limitations, interpretation, and technical aspects of rigorous case study; correlational, experimental, and other quantitative research designs; measurement techniques; sampling; replication; theory testing; qualitative methods; mixed methods; meta-analysis; and quasi-experimentation.

• Statistical Analysis, including topics such as quantitative, mathematical modeling and analysis of

psychological data, statistical description and inference, univariate and multivariate analysis, null-hypothesis testing and its alternatives, power, and estimation.

• Psychometrics, including topics such as theory and techniques of psychological measurement,

scale and inventory construction, reliability, validity, evaluation of measurement quality, classical and contemporary measurement theory, and standardization.

Overarching considerations that apply to all aspects of DSK Several aspects of this IR are intentionally written broadly in order to allow programs to design curricula that are consistent with their aims, the training needs of their students, and evolutions in the field. The narrative descriptions provided within the bullet points above for each of the discipline-specific content areas are not checklists of required topics; rather, they are examples of the sorts of topics that may be included. For example, under Category 4, all programs are expected to provide evaluated doctoral-level experience in research methods, statistical analysis, and psychometrics; however, different programs may elect to include customized topics within those broad headings. Considerations specific to Category 1 (History and Systems) The History and Systems requirement is the only portion of the DSK that may be accomplished entirely prior to matriculation into the doctoral program and/or through undergraduate-level work after matriculation into the doctoral program. Alternatively, programs may choose to cover this domain of knowledge at the graduate rather than the undergraduate level. Refer to the section below entitled Foundational knowledge attained outside of the doctoral program for information about evaluation of these types of educational experiences.

1 Evaluated educational experience: a learning experience (e.g., course, parts of courses, or independent study) the outcome of which is assessed by a person recognized as having current knowledge and expertise in the area of the learning experience.

Considerations specific to Category 2 (Basic Content Areas in Scientific Psychology) The SoA distinguishes between 1) foundational knowledge of DSK, which may be acquired prior to matriculation into the doctoral program, at the undergraduate level after entering the doctoral program, or through graduate-level training, and 2) graduate-level knowledge of DSK. As required by the SoA, programs must demonstrate that students are provided with the opportunity to acquire and be evaluated on Category 2 discipline-specific knowledge at the graduate level. In evaluating whether a program has provided sufficient coverage of the DSK, the CoA will require documentation that, at program completion, each of its students has demonstrated sufficient knowledge in each Category 2 area to allow 1) graduate-level interaction with the scientific literature that draws on these categories and 2) an understanding of the scientific foundations of the Profession-Wide Competencies. Refer to the section below entitled Graduate-Level Training for information about how the curriculum will be evaluated by CoA to ensure sufficient graduate-level coverage. It is understood that accredited programs will vary in the amount of foundational knowledge of the DSK that is expected at program entry; also, within a single program, students may have variable knowledge bases at program entry. For some programs, rigorous entry criteria will result in the need for less emphasis on foundational content within the doctoral program and more extensive coverage of graduate-level knowledge of DSK. By establishing foundational knowledge in this manner, trainees will demonstrate considerable depth of knowledge when the entirety of their educational records are considered, despite relatively less emphasis on foundational material during doctoral training. In contrast, for programs that admit students with less undergraduate education in foundational knowledge of the DSK, the entire curriculum (both foundational knowledge and graduate-level knowledge) may be taught during doctoral training. It is not consistent with the SoA for the entirety of a student’s education in the DSK to occur prior to matriculation into the doctoral program or through undergraduate coursework following matriculation into the doctoral program. Coverage of graduate-level discipline-specific knowledge within an accredited program may be provided through coursework (e.g., individual courses or material infused across multiple courses) or through other evaluated educational experiences (e.g., research requirements, qualifying examinations, or other methods). Programs must provide a minimum of one integrative evaluated educational experience (Category 3: Advanced Integrative Knowledge), but it is permissible to achieve multiple required graduate-level competencies in DSK through one or more integrative experiences. Regardless of the method by which a program chooses to satisfy the discipline-specific knowledge requirement, the program must document how each student demonstrates graduate-level knowledge in the relevant content areas. The program must also document procedures for ensuring the curriculum plan in these content areas is developed, provided, and evaluated by faculty who are well qualified in the content areas as specified in IR C-23D. Evaluating graduate-level training Graduate-level training must include evidence of graduate students’ exposure to knowledge through a curricular experience that utilizes primary source materials (including original empirical work that represents the current state of the area), emphasizes critical thinking and communication at an advanced level, and facilitates integration of discipline-specific knowledge with the program’s substantive area(s) of practice.

As programs work to confirm that their graduate-level training and evaluation is sufficient to meet these criteria, they are advised to ensure that students are interacting with current primary source materials and that they are evaluated in part on their ability to communicate critical thinking at an advanced level. Evaluating foundational knowledge attained outside of the doctoral program Programs that permit the attainment of foundational Category 1 and/or Category 2 knowledge through experiences that were not acquired within the accredited program bear a significant responsibility for documenting the quality/rigor, currency, standardization, and fairness of the method for establishing students’ knowledge. If programs permit students to attain foundational knowledge of Category 1 or 2 areas of DSK outside of their doctoral training (i.e., prior to matriculation or through undergraduate coursework they may enroll in while they are also doctoral students), it is incumbent upon programs to develop and implement systematic processes to evaluate each individual student’s foundational knowledge. The CoA will assess the extent to which these systematic processes are:

• Relevant to the required discipline-specific knowledge areas (i.e., history and systems; affective, biological, cognitive, social, or developmental aspects of behavior).

• Sufficiently rigorous to demonstrate students’ substantial understanding of discipline-specific knowledge.

• Appropriate for the program’s intended use. • Free from discrimination on bases irrelevant to success in the doctoral program. • Based on a substantial educational experience that included evaluation of knowledge

contemporaneous with the experience (e.g., a course for which the instructor assigned a grad at course completion, rather than an activity completed in the remote past that was evaluated post hoc by a member of the doctoral faculty).

The SoA lists the GRE subject test as an example of a standardized test; however, the CoA does not mean to imply that this is the only or the preferred method of evaluation. The Major Field Test or other standardized evaluations of knowledge in scientific psychology may also be appropriate, as may evaluations developed at the program level (e.g., tests of knowledge at program entry designed by the doctoral program). In addition, there are several instances in which the GRE subject test may not be an appropriate evaluation method for a program (e.g., if it does not evaluate the required areas of knowledge, is not considered appropriate for the program’s use, or discriminates against specific applicants on bases irrelevant to success in the program). The CoA anticipates that assessment methods will evolve as demand for them increases.

At times a program may determine that its evaluation methods or minimum criteria could inadvertently discriminate against an individual student on the basis of issues irrelevant to success in the doctoral program. In this case, the program should utilize alternative methods and corresponding criteria and document this determination process and the specific criteria used. Considerations specific to Category 3 (Advanced Integrative Knowledge in Scientific Psychology) The Advanced Integrative Knowledge category must be achieved entirely at the graduate level.

Considerations specific to Category 4 (Research Methods, Statistical Analysis, and Psychometrics) The Research Methods, Statistical Analysis, and Psychometrics category of DSK must be achieved entirely at the graduate level. It is not required that coverage of Statistical Analysis or Psychometrics include original source materials.

C-8 D Profession-Wide Competencies (Commission on Accreditation, October 2015; draft revised for public comment, November 2016; revised

July 2017) Introduction The Commission on Accreditation (CoA) requires that all trainees who complete accredited training programs, regardless of substantive practice area, degree type, or level of training, develop certain competencies as part of their preparation for practice in health service psychology (HSP). The CoA evaluates a program’s adherence to this standard in the context of the SoA sections that articulate profession-wide competencies at the doctoral (Section II.B.1.b), internship (Section II.A.2), and post-doctoral (Section II.B.1) levels. This Implementing Regulation refers specifically to aspects of a program’s curriculum or training relevant to acquisition and demonstration of the profession-wide competencies required in all accredited programs. The CoA acknowledges that programs may use a variety of methods to ensure trainee competence, consistent with their program aim(s), degree type, and level of training. However, all programs must adhere to the following training requirements:

• Consistency with the professional value of individual and cultural diversity (SoA Introduction, Section II.B). Although Individual and Cultural Diversity is a profession-wide competency, the CoA expects that appropriate training and attention to diversity will also be incorporated into each of the other profession-wide competencies, consistent with SoA Introduction, Section II.B.2.a.

• Consistency with the existing and evolving body of general knowledge and methods in the science and practice of psychology (SoA Introduction, Section II.B.2.d). The CoA expects that all profession-wide competencies will be grounded, to the greatest extent possible, in the existing empirical literature and in a scientific orientation toward psychological knowledge and methods.

• Level-appropriate training. The CoA expects that training in profession-wide competencies at the

doctoral and internship levels will provide broad and general preparation for entry level independent practice and licensure (SoA Introduction, Section II.B.2.b) Training at the postdoctoral level will provide advanced preparation for practice (SoA Introduction, Section II.B.2.c). For postdoctoral programs that are accredited in a specialty area rather than a developed practice area of HSP, the program will provide advanced preparation for practice within the specialty.

• Level-appropriate expectations. The CoA expects that programs will require trainee demonstrations of profession-wide competencies that differ according to the level of training provided (i.e., doctoral, internship, post-doctoral). In general, trainees are expected to demonstrate each profession-wide competency with increasing levels of independence and complexity as they progress across levels of training.

• Evaluation of trainee competence. The CoA expects that evaluation of trainees’ competence in each required profession-wide competency area will be an integral part of the curriculum, with evaluation methods and minimum levels of performance that are consistent with the SoA (e.g., for clinical competencies, evaluations are based at least in part on direct observation; evaluations are consistent with best practices in student competency evaluation).

I. Research

This competency is required at the doctoral and internship levels. Demonstration of the integration of science and practice, but not the demonstration of research competency per se, is required at the post-doctoral level The CoA recognizes science as the foundation of HSP. Individuals who successfully complete programs accredited in HSP must demonstrate knowledge, skills, and competence sufficient to produce new knowledge, to critically evaluate and use existing knowledge to solve problems, and to disseminate research. This area of competence requires substantial knowledge of scientific methods, procedures, and practices. Trainees are expected to: Doctoral students:

• Demonstrate the substantially independent ability to formulate research or other scholarly activities (e.g., critical literature reviews, dissertation, efficacy studies, clinical case studies, theoretical papers, program evaluation projects, program development projects) that are of sufficient quality and rigor to have the potential to contribute to the scientific, psychological, or professional knowledge base.

• Conduct research or other scholarly activities. • Critically evaluate and disseminate research or other scholarly activity via professional publication

and presentation at the local (including the host institution), regional, or national level. Interns:

• Demonstrates the substantially independent ability to critically evaluate and disseminate research or other scholarly activities (e.g., case conference, presentation, publications) at the local (including the host institution), regional, or national level.

II. Ethical and legal standards This competency is required at the doctoral, internship, and post-doctoral levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees at all levels are expected to demonstrate competency in each of the following areas:

• Be knowledgeable of and act in accordance with each of the following:

o the current version of the APA Ethical Principles of Psychologists and Code of Conduct;

o relevant laws, regulations, rules, and policies governing health service psychology at the organizational, local, state, regional, and federal levels; and

o relevant professional standards and guidelines.

• Recognize ethical dilemmas as they arise, and apply ethical decision-making processes in order to

resolve the dilemmas.

• Conduct self in an ethical manner in all professional activities. III. Individual and cultural diversity This competency is required at the doctoral, internship, and post-doctoral levels.

Effectiveness in health service psychology requires that trainees develop the ability to conduct all professional activities with sensitivity to human diversity, including the ability to deliver high quality services to an increasingly diverse population. Therefore, trainees must demonstrate knowledge, awareness, sensitivity, and skills when working with diverse individuals and communities who embody a variety of cultural and personal background and characteristics. The Commission on Accreditation defines cultural and individual differences and diversity as including, but not limited to, age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and socioeconomic status. The CoA recognizes that development of competence in working with individuals of every variation of cultural or individual difference is not reasonable or feasible. Trainees at all levels are expected to demonstrate:

• an understanding of how their own personal/cultural history, attitudes, and biases may affect how they understand and interact with people different from themselves;

• knowledge of the current theoretical and empirical knowledge base as it relates to addressing

diversity in all professional activities including research, training, supervision/consultation, and service;

• the ability to integrate awareness and knowledge of individual and cultural differences in the

conduct of professional roles (e.g., research, services, and other professional activities). This includes the ability to apply a framework for working effectively with areas of individual and cultural diversity not previously encountered over the course of their careers. Also included is the ability to work effectively with individuals whose group membership, demographic characteristics, or worldviews create conflict with their own.

Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence as they progress across levels of training. Trainees are expected to: Doctoral students:

• Demonstrate the requisite knowledge base, ability to articulate an approach to working effectively with diverse individuals and groups, and apply this approach effectively in their professional work.

Interns:

• Demonstrate the ability to independently apply their knowledge and approach in working effectively with the range of diverse individuals and groups encountered during internship.

Post-doctoral residents:

• Demonstrate the ability to independently apply their knowledge and demonstrate effectiveness in working with the range of diverse individuals and groups encountered during residency, tailored to the learning needs and opportunities consistent with the program’s aim(s).

IV. Professional values and attitudes This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training.

Doctoral students and Interns are expected to:

• behave in ways that reflect the values and attitudes of psychology, including integrity, deportment, professional identity, accountability, lifelong learning, and concern for the welfare of others.

• engage in self-reflection regarding one’s personal and professional functioning; engage in activities

to maintain and improve performance, well-being, and professional effectiveness.

• actively seek and demonstrate openness and responsiveness to feedback and supervision.

• respond professionally in increasingly complex situations with a greater degree of independence as they progress across levels of training.

V. Communication and interpersonal skills This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. The CoA views communication and interpersonal skills as foundational to education, training, and practice in health service psychology. These skills are essential for any service delivery/activity/interaction, and are evident across the program’s expected competencies. Doctoral students and interns are expected to:

• develop and maintain effective relationships with a wide range of individuals, including colleagues, communities, organizations, supervisors, supervisees, and those receiving professional services.

• produce and comprehend oral, nonverbal, and written communications that are informative and well-integrated; demonstrate a thorough grasp of professional language and concepts.

• demonstrate effective interpersonal skills and the ability to manage difficult communication well. VI. Assessment This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees demonstrate competence in conducting evidence-based assessment consistent with the scope of Health Service Psychology. Doctoral students and Interns are expected to: • Demonstrate current knowledge of diagnostic classification systems, functional and dysfunctional

behaviors, including consideration of client strengths and psychopathology.

• Demonstrate understanding of human behavior within its context (e.g., family, social, societal and cultural).

• Demonstrate the ability to apply the knowledge of functional and dysfunctional behaviors including context to the assessment and/or diagnostic process.

• Select and apply assessment methods that draw from the best available empirical literature and that

reflect the science of measurement and psychometrics; collect relevant data using multiple sources and methods appropriate to the identified goals and questions of the assessment as well as relevant diversity characteristics of the service recipient.

• Interpret assessment results, following current research and professional standards and guidelines, to

inform case conceptualization, classification, and recommendations, while guarding against decision-making biases, distinguishing the aspects of assessment that are subjective from those that are objective.

• Communicate orally and in written documents the findings and implications of the assessment in an

accurate and effective manner sensitive to a range of audiences.

VII. Intervention This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees demonstrate competence in evidence-based interventions consistent with the scope of Health Service Psychology. Intervention is being defined broadly to include but not be limited to psychotherapy. Interventions may be derived from a variety of theoretical orientations or approaches. The level of intervention includes those directed at an individual, a family, a group, an organization, a community, a population or other systems. Doctoral students and Interns are expected to demonstrate the ability to:

• establish and maintain effective relationships with the recipients of psychological services. • develop evidence-based intervention plans specific to the service delivery goals. • implement interventions informed by the current scientific literature, assessment findings, diversity

characteristics, and contextual variables. • demonstrate the ability to apply the relevant research literature to clinical decision making. • modify and adapt evidence-based approaches effectively when a clear evidence-base is lacking, • evaluate intervention effectiveness, and adapt intervention goals and methods consistent with

ongoing evaluation. VIII. Supervision

This competency is required at the doctoral and internship level. The CoA views supervision as grounded in science and integral to the activities of health service psychology. Supervision involves the mentoring and monitoring of trainees and others in the development of competence and skill in professional practice and the effective evaluation of those skills. Supervisors act as role models and maintain responsibility for the activities they oversee. Trainees are expected to:

Doctoral students:

• Demonstrate knowledge of supervision models and practices.

Interns:

• Apply this knowledge in direct or simulated practice with psychology trainees, or other health professionals. Examples of direct or simulated practice examples of supervision include, but are not limited to, role-played supervision with others, and peer supervision with other trainees.

IX. Consultation and interprofessional/interdisciplinary skills This competency is required at the doctoral and internship level. The CoA views consultation and interprofessional/interdisciplinary interaction as integral to the activities of health service psychology. Consultation and interprofessional/interdisciplinary skills are reflected in the intentional collaboration of professionals in health service psychology with other individuals or groups to address a problem, seek or share knowledge, or promote effectiveness in professional activities. Trainees are expected to: Doctoral students and Interns:

• Demonstrate knowledge and respect for the roles and perspectives of other professions. Doctoral students:

• Demonstrates knowledge of consultation models and practices. Interns:

• Apply this knowledge in direct or simulated consultation with individuals and their families, other health care professionals, interprofessional groups, or systems related to health and behavior.

Direct or simulated practice examples of consultation and interprofessional/interdisciplinary skills include but are not limited to:

• role-played consultation with others. • peer consultation, provision of consultation to other trainees.

C-9 D. Diversity Education and Training (formerly C-23; Commission on Accreditation, November 2009; revised March 2013, November 2015)

In accordance with Standard II.B.1.b of the doctoral Standards of Accreditation (SoA), a program has and implements a thoughtful and coherent plan to provide students with relevant knowledge and experiences about the role of cultural and individual diversity in psychological phenomena and professional practice. Although the Commission asks for demographic information about faculty/staff and students in the tables of the self-study and annual report, the information requested is limited to the data collected in federal reports, which is not sufficient in demonstrating a program’s compliance with Standard II.B.1.b. Consistent with Standard I.B.2, as described in the doctoral program SoA, cultural and individual diversity includes but is not limited to age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and social economic status.

An accredited program is expected to articulate and implement a specific plan for integrating diversity into its didactic and experiential training. This training should be based on the multicultural conceptual and theoretical frameworks of worldview, identity, and acculturation, rooted in the diverse social, cultural, and political contexts of society, and integrated into the science and practice of psychology. Programs are expected to train students/interns/residents to respect diversity and be competent in addressing diversity in all professional activities including research, training, supervision/consultation, and service. Programs are expected to train students to be competent not only for serving diverse individuals present in their local community and training setting, but also for working with diverse individuals they may encounter when they move to other locations after completion of their training. The program should demonstrate that it examines the effectiveness of its education and training efforts in this area. Steps to revise/enhance its strategies as needed should be documented.

C-10 D. Positive Identification of Students Consistent with Higher Education Opportunity Act (formerly C-25; Commission on Accreditation, November 2009; revised 2015)

Consistent with the 2008 Higher Education Opportunity Act, all accrediting agencies recognized by the U.S. Department of Education are required by federal law to engage in a review of the methods used by its accredited programs for positive identification of students who are enrolled in any form of distance/online/electronically mediated education. As such, the APA Commission on Accreditation (CoA) requires that if a student in an APA-accredited program is engaged in any form of distance, online, or electronically mediated education for any part of their educational sequence (doctoral, internship, residency), the program must provide CoA with information in its self-study regarding the methods it and its host institution use to identify that student. In particular, the program must provide CoA with information about how it ensures that a student who registers or receives credit for a course that uses any form of distance, online, or electronically mediated education is the same student who participates in and completes that course. Whatever methodology is used must clearly protect student privacy. Finally, students must be provided with information at the time of registration or enrollment of any projected additional student charges associated with verification of student identity.

C-11 D. Distance and Electronically Mediated Education in Doctoral Programs (formerly C-27; Commission on Accreditation, July 2010; revised November 2015)

The APA Commission on Accreditation (CoA) is recognized as an accrediting body by both the U.S. Department of Education and Council of Higher Education Accreditation. The CoA’s recognized scope of accreditation does not include distance education. However, the CoA understands that the growth of technology has increased the options for how instruction can be delivered within psychology doctoral programs. Traditional methods of teaching and interaction in the same time and place are no longer the only available approach to instruction. The CoA recognizes that some accredited doctoral programs may elect to utilize distance and electronically mediated delivery formats in a supplemental or adjunctive role within their programs. The CoA defines distance and electronically mediated education in the following manner, consistent with definitions from the APA Task Force on Distance Education and Training in Professional Psychology (June 2002, page 4) as well as definitions specified by other higher education accreditation organizations.

Distance education is defined as a formal educational process in which the majority of the instruction occurs when student and instructor are not in the same place. Instruction may be synchronous (students and instructors present at the same time) or asynchronous (students and instructors access materials on their own schedule). Distance education may employ correspondence study, or audio, video, or computer technologies. Electronically mediated education covers a wide set of electronic applications and processes such as Web-based learning, computer-based learning, virtual classrooms, and digital collaboration. It includes the delivery of content via Internet, intranet/extranet (LAN/WAN), audio and videotape, satellite broadcast, interactive TV, and CD-ROM.

Although the Standards of Accreditation (SoA) do not set a pre-determined limit on the extent of distance education that is permitted, a doctoral program delivering education and training substantially or completely by distance education is not compatible with the SoA and could not be accredited. This is because face-to-face, in-person interaction between faculty members and students is necessary to achieve many essential components of the SoA that are critical to education and training in professional psychology, including socialization and peer interaction, faculty role modeling, and the development and assessment of competencies. The following elements are specifically noted for all accredited and applicant doctoral programs:

• Practicum experiences must be conducted face-to-face, in-person, and cannot be completed through

distance education (i.e., virtual clients) or other electronically mediated education; • Telesupervision of students within practicum experiences is governed through Implementing

Regulation C-13 D; • All programs are expected to follow generally accepted best practices and utilize evidence-based

methods in distance education and electronically mediated delivery; • All programs are expected to clearly describe to the CoA in their self-studies which aspects of their

education and training utilize distance or electronically mediated delivery formats; and

• All programs are expected to clearly disclose to the public which aspects of their education and training utilize distance or electronically mediated delivery formats.

Programs delivering any amount of distance education or utilizing any electronically mediated formats are expected to describe to the CoA how they meet all standards of the SoA, as is true of all programs that are accredited or are seeking accreditation. In their self-studies, such programs are expected to pay particular attention to how distance or electronically mediated delivery is related to ALL parameters of the SoA.

C-12 D. Practicum Guidelines for Doctoral Programs (formerly C-26; Commission on Accreditation, January 2010; revised November 2015)

Standard II.B.3 of the Standards of Accreditation for Health Service Psychology (SoA) for doctoral graduate programs identifies practicum as a required training element. In reviewing practicum experiences within doctoral programs, the CoA looks to determine that the program is responsible for identifying how the practicum helps to realize the educational aims identified in the program’s curriculum plan. This curriculum plan should

1. Include a clear statement of how practicum training provides opportunities for students to achieve and demonstrate profession-wide competencies, as well any program-specific competencies for which practicum is a relevant curricular element.

2. Document outcome measures used within practicum to evaluate profession-wide and any relevant program-specific competencies ; and

3. Specify how practicum is clearly integrated with other elements of the program. This includes a description of how academic knowledge is integrated with practical experience through forums led by psychologists for the discussion of the practicum experience, as well as a description of how practicum training is sequential, cumulative and graded in complexity, and designed to prepare students for further organized training.

Further, each accredited doctoral program is expected to have clearly defined administrative policies and procedures in place for both internal and external practicum settings. The guidelines below clarify the CoA’s expectations as to how programs demonstrate and provide documentation of adherence to the required practicum training elements specified in Standard II.B.3 of the SoA during periodic program review (i.e., review of the program since its last self-study).

• The CoA recognizes that practicum training and experiences can include psychological testing, consultation, program development, outreach, and advocacy, as well as the use of evidence-based practice procedures and the ability to identify and use evidence-based procedures. The CoA also recognizes that not all interventions that may occur during practicum meet the definition of “empirically supported.”

• When students are not being supervised on site by doctoral level psychologists, the program must provide on-going weekly opportunities for students to discuss their clinical work with a doctoral level psychologist appropriately credentialed for the jurisdiction in which the program is located.

• It is recognized that supervision on site can be provided by doctoral interns or post-doctoral fellows in psychology, under the supervision of a psychologist appropriately credentialed for the jurisdiction.

• The program should document how the program ensures the quality of the practicum sites, including regularly scheduled site reviews.

• The program should document the use of evaluation procedures for practicum experiences, methods for identifying strengths and weaknesses of practicum settings, and how a problem with a site is managed.

• The program should identify the administrative methods used to ensure that practicum placements meet these criteria and discuss how students are matched to these sites.

• The program should demonstrate how training and educational experiences are conducted in ways that integrate science and practice.

• The program’s curriculum plan should provide clear evidence that practicum is integrated with other elements of the program.

• The program should discuss how it regularly evaluates the forum for the discussion of the practicum experience.

• The program should include a description of how it uses feedback from the clinical supervisors to address the progress, development, and competencies of the practicum student.

• The program should identify how the minimum acceptable level of achievement is defined and assessed, and identify policies for remediation or dismissal from a practicum site when this level of achievement is not met.

• The program should identify how the required practicum experiences are sufficient to prepare the students for internship.

C-13 D. Telesupervision (formerly C-28; Commission on Accreditation, July 2010; revised November 2015, July 2017)

The CoA recognizes that accredited programs may utilize telesupervision in their program curriculum. At the same time, the CoA recognizes there are unique benefits to in-person supervision. Benefits to in-person supervision include, but are not limited to: opportunities for professional socialization and assessment of trainee competence, recognition and processing of subtle, nonverbal, and emotional or affective cues and interactions in supervision, all of which are essential aspects of professional development, ensuring quality, and protecting the public. Therefore, the CoA recognizes that there must be guidelines and limits on the use of telesupervision in accredited programs. The following applies only to the MINIMUM number of required hours of supervision. At the doctoral level, these are the minimal supervision requirements for each practicum site, as defined by the doctoral program. Supervision beyond the minimum number of required hours may utilize methods or modalities that are deemed appropriate by the accredited program. Nothing in this Implementing Regulation contravenes other requirements in the Standards of Accreditation (SoA). It only clarifies the utilization of telesupervision at the doctoral practicum level. Definitions:

Telesupervision is supervision of psychological services through a synchronous audio and video format where the supervisor is not in the same physical facility as the trainee. (See the definition of supervision as noted in the Glossary) In-person supervision is supervision of psychological services where the supervisor is physically in the same room as the trainee. (See the definition of supervision as noted in the Glossary)

Guidelines and Limits: • Telesupervision may not account for more than 50% of the total supervision at a given practicum

site, and may not be utilized until a student has completed his/her first intervention practicum experience. Furthermore, it is the doctoral program’s responsibility to ensure that the student has had sufficient experience and in-person supervision in intervention at the doctoral level and possesses a level of competence to justify this modality of supervision in his/her sequence of training.

Programs that utilize telesupervision are expected to address generally accepted best practices. Furthermore, as with all accredited programs, programs that utilize telesupervision must demonstrate how they meet all standards of the SoA. Programs utilizing ANY amount of telesupervision need to have a formal policy addressing their utilization of this supervision modality, including but not limited to:

• An explicit rationale for using telesupervision; • How telesupervision is consistent with their overall aims and training outcomes; • How and when telesupervision is utilized in clinical training; • How it is determined which trainees can participate in telesupervision; • How the program ensures that relationships between supervisors and trainees are established at the

onset of the supervisory experience; • How an off-site supervisor maintains full professional responsibility for clinical cases; • How non-scheduled consultation and crisis coverage are managed; • How privacy and confidentiality of the client and trainees are assured; and

• The technology and quality requirements and any education in the use of this technology that is required by either trainee or supervisor.

C-14 D. Direct Observation (Commission on Accreditation, November 2015; revised February 2017)

This Implementing Regulation is intended to clarify the expectations of the CoA with regard to “direct observation” as described in doctoral Standards of Accreditation (SOA) as follows:

Standard II.B.3.d “As part of a program’s ongoing commitment to ensure the quality of its graduates, each practicum evaluation must be based in part on direct observation of the practicum student and her/his developing skills (either live or electronically).”

Direct observation provides essential information regarding trainees’ development of competencies, as well as the quality of the services provided, that cannot be obtained through other methods. Direct observation allows supervisors to provide a more accurate assessment and evaluation of observable aspects of trainees’ competency development regarding one or more profession-wide and program-specific (if any) competencies associated with that training experience. The direct observation requirement described in this IR applies to all training experiences that fall under the program’s application of practicum training in IR C-12 D. At minimum, programs are required to conduct one direct observation per evaluation period as described below. In situations in which students complete an extra, elective, non-required practicum, or placement and this experience is considered in the evaluation of a required competency, the training experience must include a direct observation as a part of the evaluation of the experience. Definitions and Guidelines: Direct observation includes in-person observation (e.g., in-room or one-way mirror observation of direct service contact), live simultaneous audio-video streaming, or audio or video recording. A training site that does not permit live observation, audio or video recording by policy is not a sufficiently unique circumstance to circumvent this requirement. The supervisor who is evaluating the trainee’s performance must base part of that evaluation on direct observation. Supervisors conducting direct observation must be appropriately trained, credentialed, and prepared in their discipline and in the health service psychology activities being supervised, legally authorized for independent practice in their jurisdiction, and legally responsible for the direct service being provided. Supervisors who perform the direct observation must be competent in performing the supervised activity, as well as in providing supervision. Direct observation is required for each practicum evaluation completed. All accredited programs must verify on the evaluation form that direct observation occurs for each evaluation period as defined by the program. In a given evaluation period, a student may complete more than one practicum experience (e.g., separate rotations within a single-semester practicum; student completing two different practica during the same semester). If a separate evaluation is completed for each rotation or setting, each evaluation must include direct observation. If a single evaluation covers all rotations or settings, then a minimum of one direct observation is required. Per IR C-12 D, it is recognized that supervision on site can be provided by doctoral interns or postdoctoral residents in health service psychology under the supervision of a psychologist appropriately credentialed in the jurisdiction. In these situations, the direct observation requirement may only be met by having the appropriately credentialed supervisor(s), legally responsible for the direct service being provided, conduct

the observation and evaluation. This does not preclude doctoral interns or postdoctoral residents from contributing to the direct observation or evaluation process. It is not expected that all of the individual competencies (profession-wide or program-specific) would be directly observed during every practicum experience, but rather that the scope of the direct observation would be sufficient to contribute meaningfully to an evaluation of student performance in competencies relevant to that practicum placement.

C-15 D. Awarding the Doctoral Degree Prior to Completion of the Internship (formerly C-5; Commission on Accreditation, original date unknown; revised January 2001, November

2015) All accredited program requirements, including the internship, should be satisfactorily completed prior to awarding the doctoral degree in the student’s substantive area of health service psychology. In special instances in which students participate in graduate ceremonies prior to completing the internship, the program should ensure that university certification by transcript, diploma, or other means of the student’s having completed the degree requirements for the accredited program in professional psychology does not precede the actual completion of all such program requirements. Programs in health service psychology that certify the completion of all requirements of that program for the doctoral degree before completion of an internship violate accreditation guidelines. Whether or not a student completes a dissertation prior to an internship is a matter of individual and program discretion. Internships are designed and funded as training experiences at the doctoral level. The competency level of the training experience is consistent with that designation, and it would be inappropriate simply to rename the training as postdoctoral. If the trainee is a “respecialization” intern, the fact that the trainee has a doctoral degree in another field of the discipline does not change the doctoral level of experience required in the trainee's field of professional respecialization.

C-16 D. Affiliated Internship Training Programs (formerly C-10; Commission on Accreditation, March 1998; revised October 2007, November 2015)

An exclusively affiliated internship is an accredited internship that only admits interns who are students from a specific accredited doctoral program. A partially affiliated internship is an accredited internship in which a portion of the interns admitted are students from a specific accredited doctoral program. The procedures for evaluating and designating the programs are as follows:

1. The internship and the program with which it is affiliated are site visited and accredited separately and in the same manner as other programs and internships. However, as part of their self-study reports, the programs would designate that they are either (a) an affiliated internship or (b) a program that places students at an affiliated internship.

2. The CoA understands that affiliated internships (formerly known as captive internships) involve

close integration with the affiliated doctoral programs. However, affiliated internship programs are independently accredited and must be reviewed by the CoA as separate entities and meet all the accreditation requirements expected of a non-affiliated internship program. Affiliated internships must provide the CoA with information specific to the internship program during the course of review. As such, an internship self-study may not simply reference aspects of a doctoral program’s self-study to fulfill the internship requirements of the Standards of Accreditation (SoA). All relevant program materials must be submitted within the internship self-study, and all information (e.g. policies and procedures, outcome data, etc.) should be specific to the internship training program.

3. Any affiliated internship programs that make use of multiple independently administered entities

as training sites will be reviewed as a consortium and will be required to meet all aspects of Standard I.A.3 of the SoA for internship programs.

4. The internship clearly states its status as exclusively affiliated or partially affiliated in all

descriptive material and representations to the public. If approved, the affiliated internship will be listed in the American Psychologist listing for accredited internships. The listing for the internship agency will state that it is an exclusively affiliated or partially affiliated internship; the name of the accredited doctoral program also will be stated (e.g., X Internship [affiliated with Y University Training Program]).

C-17 D. Expected Internship Placements for Students in Accredited Doctoral Programs (formerly C-31(c); Commission on Accreditation, July 2012; revised November 2015)

The CoA conducts reviews in accordance with its Standards of Accreditation (SoA) and as required by the US Department of Education (USDE) and Higher Education Accreditation (CHEA) regulations (see relevant USDE and CHEA regulations below). Several pieces of the SoA are relevant to this review. Standard I.C.2 states that eligibility for accreditation by the CoA requires completion“…of an internship prior to receiving the doctoral degree.” Standard I.A.1.b requires that training for practice is sequential, cumulative and graded in complexity and designed to prepare students for further organized training. Standard II.B.4 states that “…the program must demonstrate that all students complete a one year full-time or two year part-time internship.” Therefore, it is clear that placement of students in an organized internship is both an important step in the doctoral training sequence and an important outcome of the graduate doctoral training. Accredited doctoral programs, or doctoral programs seeking accreditation, must provide to the CoA appropriate data to demonstrate outcomes consistent with the SoA and the program's stated educational aim(s). Because completion of an internship is a required component of training for all accredited doctoral programs, the CoA requires that the doctoral program provide evidence of students' successful completion of an internship program of appropriate quality. For all other internship placements (including APPIC member, CAPIC member, CDSPP compliant site, etc.), the doctoral program must demonstrate how it ensures the quality of the internship training experience. To that end, an accredited program that sends students to sites that are not accredited must provide information to the CoA regarding its process for monitoring the quality of internship training, including the quality of student achievement and development of competencies at these sites. Information regarding the nature of the training provided must be of sufficient detail to demonstrate the adequacy and quality of these training experiences. CoA is required to follow these US Department of Education (USDE) and Council of Higher Education Accreditation (CHEA) regulations in its reviews: USDE - §602. 16 Accreditation and preaccreditation standards.

a. The agency must demonstrate that it has standards "'for accreditation, and preaccreditation, if offered, that are sufficiently rigorous to ensure that the agency is a reliable authority regarding the quality of the education or training provided by the institutions or programs it accredits.

b. The agency meets this requirement if- (1) The agency's accreditation standards effectively address the quality of the institution or program in the following areas: (i) Success with respect to student achievement in relation to the institution’s mission, which may include different standards for different institutions or programs, as established by the institution, including, as appropriate, consideration of course completion, State licensing examination, and job placement rates. CHEA -12A 3. 12A. Advances Academic Quality. Advancing academic quality is at the core of voluntary accreditation. "Academic quality" refers to results associated with teaching, learning, research, and service within the framework of institutional mission. To be recognized, the accrediting organization provides evidence that it has: 3. standards or policies that include expectations of institutional or program quality, including student achievement, consistent with its mission.

C-18 D. Outcome Data for Doctoral Programs (formerly C-32; Commission on Accreditation, October 2012; April 2016)

This Implementing Regulation clarifies the type of data the CoA needs to make an accreditation decision on doctoral programs. The CoA requires all accredited programs to provide outcome data on the extent to which the program is effective in achieving its aim(s) and demonstrating student attainment of required discipline-specific knowledge, profession-wide competences, and program-specific competencies (if any). As stated in the Standards of Accreditation (SoA) and the accompanying Implementing Regulation (IR) for doctoral programs, discipline-specific knowledge serves as a cornerstone for the establishment of identity in and orientation to health service psychology. Programs are required to demonstrate discipline-specific knowledge of its students (Standard II.B.1.a D and IR C-7 D).

Discipline-specific knowledge serves as a cornerstone for the establishment of identity as a psychologist and orientation to health service psychology. Therefore, all students in accredited doctoral programs shall acquire a general knowledge base in the discipline of psychology, broadly construed. a. Discipline-specific knowledge represents the requisite core knowledge of psychology an

individual must have to attain the profession-wide competencies. Programs may elect to demonstrate discipline-specific knowledge of students by: i. Using student selection criteria that involve standardized assessments of a foundational

knowledge base (e.g., GRE subject tests). In this case, the program must describe how the curriculum builds upon this foundational knowledge to enable students to demonstrate graduate level discipline-specific knowledge.

ii. Providing students with broad exposure to discipline-specific knowledge. In this case, the program is not required to demonstrate that students have specific foundational knowledge at entry but must describe how the program's curriculum enables students to demonstrate graduate-level discipline-specific knowledge.

In addition to demonstrating that students obtain discipline-specific knowledge, programs must evaluate profession-wide and program-specific (if any) competencies. As stated in the SoA for doctoral programs relevant to student profession-wide and program-specific competencies (II.D.1):

1. Evaluation of students’ competencies a. The program must evaluate students’ competencies in both profession-defined and

program-defined areas. By the time of degree completion, each student must demonstrate achievement of both the profession-wide competencies and those required by the program. Thus, for each competency, the program must:

i. Specify how it evaluates student performance and the minimum level of achievement or performance required of the student to demonstrate competency. Programs must demonstrate how their evaluation methods and minimum levels of achievement are appropriate for the measurement of each competency. The level of achievement expected should reflect the current standards for the profession.

ii. Provide outcome data that clearly demonstrate that by the time of degree completion, all students have reached the appropriate level of

achievement in each profession-wide competency and in each program-defined competency. While the program has flexibility in deciding what outcome data to present, the data should reflect assessment that is consistent with best practices in student competency evaluation.

iii. Present formative and summative evaluations linked to exit criteria and data demonstrating achievement of competencies for each student in the program.

b. For program graduates, the program must provide distal evidence of students’ competencies and program effectiveness and must evaluate graduates’ career paths in health service psychology after they have left the program.

i. Two years after graduation, the program must provide data on how well the program prepared students in each profession-wide and program-specific competency. The program must also provide data on students’ job placement and licensure rates.

ii. At 5 years postgraduation, the program must provide data on graduates, including data on graduates’ licensure (as appropriate for their current job duties) and their scholarly/research contributions (as consistent with the program’s aims).

In addition, the United States Department of Education (USDE) requires recognized accrediting bodies (such as the CoA) to collect and monitor data-driven outcomes, especially as they relate to student achievement. In making an accreditation decision on a program, CoA must demonstrate that it reviews student achievement through review of the program’s outcome data.

All accredited programs are required to demonstrate an educational/training curriculum that is consistent with program aim(s) and is designed to foster student development of required profession-wide competencies and program-specific competencies (if any). Expected minimal levels of achievements must be specified for all profession-wide competencies and program-specific competencies (if any). It is each program’s responsibility to collect, present, and utilize aggregated proximal and distal outcome data that are directly tied to profession-wide competencies and program-specific competencies (if any). Definitions: Proximal data are defined as outcomes on students as they progress through and complete the program, which are tied to the required profession-wide competencies and program-specific competencies (if any).

• Proximal data at a minimum must include evaluations of students’ performance by those who are responsible for their training (e.g., by course instructors, thesis/dissertation committees, supervisors).

• Completion of an unevaluated activity (attendance at a class or seminar, completion of a manuscript, completion of practicum hours) is not considered sufficient proximal outcome data. Rather, the program must utilize evaluative data (e.g., course outcomes/grades, supervisor evaluation of practicum performance, dissertation defense outcome, acceptance of a peer-reviewed presentation or publication) that demonstrate the program’s success in promoting mastery of profession-wide competencies and program-specific competencies (if any).

• While student self-ratings, ratings of satisfaction with training, or ratings by others (e.g., peers) may be a part of proximal assessment, they are not considered sufficient outcome data in this context since they do not address the program’s success in promoting attainment of profession-wide competencies and program-specific competencies (if any).

Distal data are defined as outcomes on students after they have completed the program, which are tied to the profession-wide competencies and program-specific competencies (if any).

• Distal data typically include information obtained from alumni surveys addressing former students’ perceived assessments of the degree to which the program promoted mastery of profession-wide competencies and program-specific competencies (if any).

• Distal data reflecting completion of professional activities and accomplishments (e.g., licensure, employment, memberships, and affiliations), such as those found in the self-study tables, are important examples of distal outcomes but alone are not sufficient because they do not fully reflect achievement of all expected competencies.

• Although alumni surveys assessing former students’ overall satisfaction with the training program (including the degree to which the education and training is relevant) may be an important component of a program’s ongoing self-study process, they are not considered sufficient outcome data in this context since they do not address the program’s success in promoting expected competencies.

Distal data must be collected annually on alumni who are 2 years and 5 years post-graduation in that year. Although programs are expected to contact as many of these alumni as possible, it is recognized that not all graduates will be reachable. If response rates are particularly low, the program should explain low response rates and describe efforts to contact its graduates.

Level of Specificity Discipline-Specific Knowledge According to the Standards of Accreditation (Standard II.B.1.a D), accredited programs are required to demonstrate that their students attain requisite core knowledge of psychology. Consistent with IR C-7 D, accredited programs are required to identify minimum levels of achievement that are acceptable to demonstrate students’ discipline-specific knowledge at the advanced graduate level, to assess all required content areas within each category of discipline-specific knowledge for each student (e.g. history and systems of psychology; affective aspects of behavior; biological aspects of behavior; cognitive aspects of behavior; developmental aspects of behavior; social aspects of behavior; advanced integrative knowledge of basic discipline-specific content areas; research methods; quantitative methods; psychometrics), and to provide data to CoA that document that by the time of graduation, all students have attained the required minimum levels of achievement for each required area of discipline-specific knowledge. As described in IR C-7 D, programs must demonstrate that students have attained advanced graduate level discipline-specific knowledge in all content areas of each category prior to graduation. This demonstration may include but is not limited to: course grades in graduate-level courses, as described in IR C-7 D, scores on comprehensive exams in discipline-specific knowledge areas, or other evaluated learning experiences. The program must set a minimum level of achievement for demonstration of student attainment of advanced graduate level discipline-specific knowledge in each area. Because discipline-specific knowledge serves as the foundation to further training in health service psychology, data regarding discipline-specific knowledge need only be presented at the proximal level; distal data are not required for discipline-specific knowledge. Profession-Wide Competencies According to the Standards of Accreditation (Standard II.B.1.b D), accredited programs are required to provide a training/educational curriculum that fosters the development of nine profession-wide competencies (Research, Ethical and legal standards, individual and cultural diversity, professional values, attitudes, and behaviors, Communication and interpersonal skills, Assessment, Intervention, Supervision, and Consultation and interprofessional/interdisciplinary skills). Accredited programs are required to operationalize competencies in terms of multiple elements. At a minimum, those elements must reflect the content description of each PWC defined in IR C-8D, including the bulleted content, and must be consistent with the program aim(s). It is incumbent upon the program to demonstrate that there is a sufficient number

of elements articulated for each PWC so as to demonstrate adequate trainee attainment of competence. Programs must assess student performance at the level of the elements using multiple methods and within time frames appropriate for each PWC, give feedback to students at the level of elements, but report to CoA at the level of the superordinate competency. Program Specific Competencies Accredited programs may choose to include program-specific competencies as part of their educational curriculum. These should be consistent with the program’s aim(s) and the professional standards and practices of health service psychology. Further, programs must demonstrate education/training to facilitate development of these competencies, appropriate mechanisms to assess student performance on these competencies (including expected minimal levels of achievement for successful completion of the program), and its success in ensuring that students reach expected levels of performance. Similar to the expectations for profession-wide competencies, programs that choose to have program-specific competencies are expected to assess student performance at the level of the competency elements, and give feedback to students at the level of elements, but report to CoA at the level of the superordinate competency. Aggregation of Data

Aggregated data are compilations of proximal or distal data across students, which may be broken down by cohort, program year, or academic year. Aggregate data are used to demonstrate the effectiveness of the program as a whole in accomplishing its Aims and Competencies, rather than the accomplishment of an individual student over time. Overaggregation of data can obscure differences that are important for the program to recognize in evaluating its effectiveness over time. To the extent possible, data should be presented in table form using basic descriptive statistics (e.g., % meeting the minimum level of achievement, N). The program should choose statistics that allow for evaluation of whether all students are acquiring competencies in relation to its defined minimal levels of achievement for all program competencies (i.e., discipline-specific knowledge, profession wide competencies and any program-specific competencies). The program should provide meaningful data in such a way that the CoA can determine that by the time of program completion, all students have attained these minimal levels of achievement. If data presented indicate that in a particular year or cohort less than 100% of students have reached the minimum level of achievement for a content area, the program should describe how those students who did not meet the minimum level of achievement either did not continue to progress in the program or were able to remediate and later meet the minimum level of achievement.

Discipline-Specific Knowledge When a program is reviewing its outcome data to evaluate its effectiveness in promoting discipline-specific knowledge, it is expected that multiple data points from multiple sources may be used, and that basic descriptive statistics (e.g., means and standard deviations for course grades, comprehensive exam scores in discipline-specific knowledge areas), should be used. When presenting aggregated data to the CoA, it is expected that programs will present single data points for each discipline-specific knowledge area, demonstrating its overall outcomes of success in promoting student attainment of substantial knowledge at the graduate-level

• If data are aggregated over a number of years (i.e., not by cohort or year), the program needs to explain how aggregating the data in this alternate way facilitates the program’s self-improvement and demonstrates that all students meet the MLAs by the time of graduation.

Profession-Wide Competencies and Program Specific Competencies When a program is reviewing its outcome data to evaluate its effectiveness in promoting profession-wide competencies and program-specific competencies (if any), it is expected that multiple data points from multiple sources for multiple elements will be used, and that basic descriptive statistics (e.g., means and standard deviations for course grades, clinical competency examination scores, practicum evaluations ratings, alumni ratings of preparation for practice in competencies), will be used. When presenting aggregated data to the CoA, it is expected that programs will present single data points for each profession-wide competency and program-specific competency (if any), demonstrating its overall outcomes of success in promoting student attainment of competencies.

• Proximal data and distal data should be presented separately. For distal data, the presentation should clearly differentiate between data for those who are 2 years post-graduation and those 5 years post-graduation.

• If data are aggregated over a number of years (i.e., not by cohort or year), the program must explain how aggregating the data in this alternate way facilitates the program’s self-improvement.

C-19 D. Licensure Rate for Doctoral Programs (formerly C-31(d); Commission on Accreditation, July 2012; revised November 2015)

Evaluation of Graduates' Licensure Rates: CoA accreditation of doctoral and internship education and training programs is based on "preparation for entry-level practice in health service psychology" (see Standards of Accreditation Introduction section II. Guiding Principles of Accreditation - Section B.2.b Broad and General Preparation for Practice at the Entry Level). All doctoral programs, whether seeking accreditation or reaccreditation, are expected to achieve this objective of preparing students for entry level practice in professional psychology. One tangible index of preparation for entry level practice is a program's success in preparing its graduates to be licensed as psychologists. The CoA interprets the licensure rate of program graduates within the context of: (1) the requirement that all accredited doctoral programs prepare students for entry-level practice; (2) each program's own stated educational aim(s); and, (3) statements made by the program to the public. Because specific educational aims in the programs CoA accredits may differ, the CoA does not specify a threshold or minimum number when reviewing a program's licensure rate. Rather, the CoA uses its professional judgment to determine if the program's licensure rate, in combination with other factors such as the attrition of students from the program and their time to degree, demonstrates students' successful preparation for entry-level practice in professional psychology. This includes determining if program graduates' licensure rates are consistent with the expressed or implied promises the program makes to the public and to CoA with respect to achieving its educational aim(s). In general, the more emphasis a program places on producing graduates who will be practitioners, the higher expectations CoA will have for the proportion of students who matriculate into the program and eventually become licensed. In the process of periodic review, a program needs to discuss its licensure data in terms of its educational aims and provide information to address discrepancies between those aims and the actual licensure of students admitted to the program. All accredited doctoral programs are, however, expected to prepare students for entry-level practice and the program's achievement of this should be reflected in student success in achieving licensure after completion of the program. An accredited doctoral program is also required to provide data on licensure to the public consistent with Implementing Regulation C-26 D. NOTE: The CoA also has to conduct its reviews in accordance with the regulations of the US Secretary of Education and the Council of Higher Education Accreditation (CHEA) requiring: USDE - §602. 16 Accreditation and preaccreditation standards.

a. The agency must demonstrate that it has standards "'for accreditation, and preaccreditation, if offered, that are sufficiently rigorous to ensure that the agency is a reliable authority regarding the quality of the education or training provided by the institutions or programs it accredits.

b. The agency meets this requirement if- (1) The agency's accreditation standards effectively address the quality of the institution or program in the following areas:

(i) Success with respect to student achievement in relation to the institution’s mission, which may include different standards for different institutions or programs, as established by the institution, including, as appropriate, consideration of course completion, State licensing examination, and job placement rates.

CHEA -10. Promotes Academic Quality and Advances Student Achievement (parts A and B). Advancement of academic quality is at the core of voluntary accreditation. To be recognized, the accrediting organization provides evidence that it implements and enforces standards, policies or procedures which:

A. Articulate the accrediting organization’s expectations for academic quality and results associated with institutional or program performance, including student achievement, consistent with institution or program mission.

B. Require institutions or programs to establish and make public their expectations for achievement of academic quality and indicators of student success, to implement processes to determine whether students and graduates meet the stated expectations and to make public, in aggregate form, evidence of student success.

C-20 D. Selection and Admissions of Students into Accredited Doctoral Programs (formerly C-31(a); Commission on Accreditation, July 2012; revised November 2015)

Standard III.A.1 of the Doctoral section of the Standards of Accreditation (SoA) states:

The program has an identifiable body of students at different levels of matriculation who:

a. Are of sufficient number to ensure opportunities for meaningful peer interaction, support, and socialization.

b. Are reflective of a systematic, multiple-year plan, implemented and sustained over time, designed to attract students from a range of diverse backgrounds as outlined in the Glossary.

c. By prior achievement, students have demonstrated appropriate competency for the program’s aims as well as expectations for a doctoral program.

d. By interest and aptitude they are prepared t meet the program’s aim(s). e. They reflect through their intellectual and professional development and intended career paths

the program's aim(s) and philosophy. Preparation for practice in health service psychology is a major educational goal for all accredited programs. Thus, the CoA expects that accredited programs will admit students who are appropriately prepared to succeed in doctoral education and training. CoA expects programs will provide students with appropriate educational and training opportunities enabling admitted students to complete the program. The CoA also expects that students will demonstrate success in achieving the profession-wide and program specific competencies as assessed by the program. To this end, the CoA expects programs to clearly define their admissions standards and to specify how these standards reflect their educational aims. Further, the program needs to discuss how its admissions and selection standards are adequate and appropriate for its educational aims. In compliance with Standard II.D, the program must demonstrate its effectiveness in meeting its educational aim(s) for students in the program and any program graduates. This effectiveness must be demonstrated relative to the program's stated educational aim(s), and must be consistent with Standard III.C.1, in that "program faculty engage in and document actions and procedures that actively encourage timely completion of the program and maximize student success" [emphasis added]. The CoA's review of doctoral student selection policies and procedures necessarily requires the exercise of professional judgment, programs must demonstrate that:

1. They have and abide by written policies and procedures for student selection; 2. Those written policies and procedures are consistent with their educational

aims; 3. Those written policies and procedures are developed to ensure that students are well-prepared to

succeed and that program graduates are prepared for entry to practice; As part of CoA’s evaluation of a program’s student selection policies and procedures, the CoA will also consider the program’s outcome data on program graduates, including attrition, time to degree, graduate rate, and licensure data as indices of the program’s effectiveness in selecting students who are able to complete a doctoral program and enter into practice. CoA recognizes that doctoral programs’ student selection and admissions practices may be informed by their training aims or by institutional or program missions (e.g., that emphasize providing opportunities for enrollment of nontraditional graduate students, or that enroll students with very diverse prior educational experiences). However, CoA reviews programs based only on educational aims that include broad and

general preparation for entry-level practice, integration of science and practice, and the program’s philosophy and mission in relation to current professional standards and regional and national needs. Thus, selection and admissions practices must be consistent with effective training and outcomes in these areas.

C-21 D. Diversity Recruitment and Retention

(formerly C-22; Commission on Accreditation, November 2009; revised March 2013, November 2015, October 2018)

The Standards of Accreditation (SoA) state that five principles, one of which is a commitment to cultural and individual diversity, “guide accreditation decisions, such that programs whose policies and procedures violate them would not be accredited.” Furthermore, the Commission “is committed to a broad definition of cultural and individual differences and diversity that includes, but is not limited to, age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and socioeconomic status (SoA, p. 3).” Diversity is essential to science and quality education and training in Health Service Psychology. The goals of diversity recruitment and retention include, but are not limited to, creating and maintaining inclusive environments and improving access to quality education and training. An inclusive environment is one in which the program creates an atmosphere that is welcoming, respectful and affirming of students’, and faculty members’ multiple identities. In accordance with Standards I.B.2, III.A.1.b.i-ii and IV.B.5 of the Standards of Accreditation (SoA) for doctoral programs, an accredited doctoral program is responsible for making systematic, coherent, and long-term efforts to 1) attract (i.e., recruit) and 2) retain diverse students and faculty into the program. In addition, the program is responsible for assessing the effectiveness of both its recruitment and retention efforts and identifying areas of improvement. For both recruitment and retention of students and faculty, the program must provide program-level efforts and activities, in addition to any institutional, departmental, or other unit activities that are used. Programs are expected to seek and utilize generally accepted best practices in the field regarding recruitment and retention of diverse individuals. In planning for the recruitment and retention of diverse individuals, accredited programs should consider the following:

- A program may include institutional-level initiatives addressing diversity, but these, in and of themselves, are not considered sufficient.

- The lack of faculty openings, or having existing faculty with strong representation of diversity, does not exempt the program from the responsibility of having a systematic, multi-year plan in place.

- Similarly, having an existing student body with strong representation of diversity does not exempt the program from the responsibility of having a systematic, multi-year plan in place.

- The demographic information about faculty and students in the tables of the self-study and annual report is not sufficient to demonstrate a program’s compliance with Standards I.B.2, III.A.1.b.i-ii, and IV.B.5.

Recruitment The program is expected to document that it has developed and implemented a systematic plan to recruit both students and faculty from diverse backgrounds. Students An accredited doctoral program should document and report in its self-study:

• that it has developed a systematic, multi-year, and multiple effort plan, implemented and sustained over time, to attract students from a range of diverse identities;

• the concrete and specific program-level activities, approaches, and initiatives it implements to increase diversity among its students;

• the areas of diversity recruitment in which it has had success, as well as the areas of diversity recruitment it is working to improve, recognizing the broad definition of diversity, and;

• how it examines the effectiveness of its efforts to attract diverse students, and the steps it has taken to revise/enhance its strategies.

Faculty An accredited doctoral program should demonstrate and report in its self-study:

• that it has developed a systematic, multi-year, and multiple effort plan, implemented and sustained over time, to attract faculty from a range of diverse identities (i.e., when there are faculty openings);

• the concrete and specific program-level activities, approaches, and initiatives it implements to increase diversity among its faculty;

• the areas of diversity recruitment in which it has had success, as well as the areas of diversity recruitment it is working to improve, recognizing the broad definition of diversity, and;

• how it examines the effectiveness of its efforts to attract diverse faculty, and the steps it has taken to revise/enhance its strategies.

Retention The program is expected to document that it has developed and implemented a systematic plan to retain both students and faculty from diverse backgrounds. Doctoral students An accredited doctoral program is expected to describe in its self-study:

• the specific activities, approaches, and initiatives it implements to maintain diversity among its students and ensure a supportive and inclusive environment for all students;

• concrete program-level actions to retain diverse students; • how these efforts are broadly integrated across key aspects of the program; • how the program examines the effectiveness of its efforts to retain diverse students, and the steps

it has taken as needed to revise and/or enhance its retention strategies. Faculty An accredited doctoral program is expected to describe in its self-study:

• the specific activities, approaches, and initiatives it implements to maintain diversity among its faculty and ensure a supportive and inclusive work environment for its diverse faculty members.

• how the program examines the effectiveness of its efforts to maintain diversity among its faculty, and the steps it has taken to revise/enhance its strategies as needed.

C-22 D. Student Attrition Rates for Doctoral Programs (formerly C-31(b); Commission on Accreditation, July 2012; revised November 2015)

Attrition during Initial or Periodic Review In the initial or periodic review of a doctoral program, the CoA looks at a number of indicators of program success. Standard III.A.1 of the SoA states:

The program has an identifiable body of students at different levels of matriculation who:

a. Are of sufficient number to ensure opportunities for meaningful peer interaction, support, and

socialization. b. Are reflective of a systematic, multiple-year plan, implemented and sustained over time,

designed to attract students from a range of diverse backgrounds as outlined in the Glossary. c. By prior achievement, students have demonstrated appropriate competency for the program’s

aims as well as expectations for a doctoral program. d. By interest and aptitude they are prepared t meet the program’s aim(s). e. They reflect through their intellectual and professional development and intended career paths

the program's aim(s) and philosophy. Further, in Standard III.B.1, the SoA states:

Program faculty are accessible to students and provide them with a level of guidance and supervision. They serve as appropriate professional role models and engage in actions that promote the students’ acquisition of knowledge, skills, and competencies consistent with the program’s training aim(s).

A doctoral program is expected to prepare students for entry level to practice in health service psychology; an essential part of such preparation is the extent to which students successfully complete the doctoral program. During periodic review of the program, the CoA reviews self-study materials (including the narrative and tabular information) as well as program correspondence. The overall attrition rate seven years preceding the review is considered by the CoA to be an indicator of the effectiveness of the program’s student selection criteria, the appropriateness and availability of faculty for mentoring students in the program, and the program's success in training students for entry into practice. Attrition data evaluated during periodic review include the proportion of admitted students who do not complete their degrees. As such, the attrition rate at the time of periodic review may be very different from the attrition rate submitted annually to the CoA within the Annual Report Online, in which students who have already departed the program are no longer included as part of either the numerator or the denominator. In addition to overall attrition rate, the CoA may consider attrition rates within cohorts (i.e., year of entry) and other aspects of the attrition data. The CoA does not set specific percentages in determining acceptable cohort or overall attrition rates. The importance of contextual issues (i.e., evaluating the attrition rate in the context of the full record of information available about a program) makes it impossible to apply a "one-size-fits-all" metric in determining program quality. Thus, the review of the attrition data requires the professional judgment of the CoA to determine how the seven-year attrition rate (including the overall attrition rate and rates for specific cohorts) reported by a program is appropriate for the profession and for the program to maintain consistency with its public materials.

Annual Review Attrition Threshold The attrition threshold based on annual review data (i.e., ARO data) is used as a broad indicator of changes in a program based upon the total number of students in the program, and the total number of students who leave the program during a year for any reason. For purposes of the annual review, the CoA uses an empirical metric to identify a level of attrition that leads to additional review, by using the mean of the most recent three years of overall attrition rates, as derived from ARO data provided by all doctoral programs. The specific threshold represents an attrition rate at the 95th percentile for those programs. In other words, the CoA seeks additional information from programs which fall within the lowest 7% of all programs in overall attrition (i.e., those programs with the least favorable attrition rates) among all accredited doctoral programs. When a program's attrition rate prompts CoA to ask for additional information during annual reviews, that information is reviewed by the CoA to determine if additional review is necessary. It is important to note that whether or not a program's three-year data have triggered a request for additional information during the annual review does not determine whether or not attrition will be the subject of more comprehensive analyses during the periodic review.

C-23 D. Faculty Qualifications (November 2015)

Faculty qualifications. Individual faculty may fulfill multiple roles within a program (e.g., teaching, clinical and/or research supervision, administration). In terms of program policy, it is the program’s responsibility to specify clearly articulated procedures for ensuring appropriate faculty training, current expertise, and effectiveness for each role they fulfill in the program. If such procedures exist in an administrative unit higher than the program, then the program must demonstrate how it has sufficient input or oversight to ensure training consistent with accreditation standards. In terms of self-study content, it is the program’s responsibility to provide clear and specific evidence in the self-study that faculty are appropriately qualified for each role that they hold in the program. That evidence should include current and relevant expertise (e.g., board certification, formal or other post-doctoral training, systematic study, ongoing professional development, research productivity, clinical competence, professional credential, academic degree/area of study, respecialization).

C-24 D. Program Names, Labels, and Other Public Descriptors (formerly C-6(a); Commission on Accreditation, January 2002; revised January 2003, November 2015)

How the program describes itself: It is recognized that programs have many possible reasons why they choose the self-descriptors or labels that they do. Some are bound by state law, others by institutional regulation, and others simply seek to assign a label to their program to describe their focus to the public. Given that these self-descriptors do not necessarily coincide with recognized areas of accreditation, any program whose label does not reflect the specific area in which it received accreditation must portray its accredited status in a manner consistent with the SoA. Postdoctoral programs accredited in substantive or specialty areas may offer training in areas of emphasis. Areas of emphasis may be described in all public materials except the certificate of completion. Programs will state clearly that accreditation is specific to the substantive or specialty area only.

Preferred: • “Doctoral program in clinical psychology” • “Internship in clinical psychology” • “Internship in health service psychology”

Examples with accurate accreditation status:

• “Doctoral program in medical psychology, accredited as a program in clinical psychology” • “Internship in pediatric psychology, accredited as a doctoral internship in health service

psychology” • “Postdoctoral residency with an emphasis in geropsychology, accredited as a postdoctoral

residency in clinical psychology”

C-25 D. Accreditation Status and CoA Contact Information (formerly C-6(b); Commission on Accreditation; November 2010; revised March 2015, November 2015)

Standard V.A.1.b of the Standards of Accreditation (SoA) for doctoral programs states that the program must include in its public materials:

“The program must disclose its status with regard to accreditation, including the specific academic program covered by that status, and the name, address, and telephone number of the Commission on Accreditation. The program should make available, as appropriate through its sponsor institution, such reports or other materials as pertain to the program’s accreditation status.”

Programs that are accredited by agencies recognized by the U.S. Department of Education (e.g., CoA) are required to provide the contact information for the accrediting body when the accreditation status is cited. The intent of this Implementing Regulation is to clarify how this information should be presented in order to ensure consistency across programs as well as provide useful information to the public. Accreditation status: • For doctoral programs the only official accredited statuses are: “Accredited on contingency,”

“Accredited,” “Accredited on probation,” and “Accredited inactive,” • Programs may indicate their appropriate status (see above) by referring to “APA” accredited or

accredited “by the Commission on Accreditation of the American Psychological Association,” For example, “APA-accredited,” APA-accredited on contingency,” “accredited by the Commission on Accreditation of the American Psychological Association, “accredited on contingency by the Commission on Accreditation of the American Psychological Association,” etc.

• Programs should not use the term “APA-approved,” since at APA this term is used to denote approved sponsors of continuing education rather than accreditation of academic/training programs.

• If there are multiple programs in the same department, institution, or agency, it should be clearly indicated in public materials which programs are APA-accredited. Multiple accredited programs should refer to their accredited status individually and in accordance with IR C-24 D.

CoA contact information: • In ALL public documents, including the program’s website (if applicable), where the program’s

accreditation status is cited as above, the name and contact information for the CoA must be provided. • Information must include the address and direct telephone number for the APA Office of Program

Consultation and Accreditation. Other information (i.e., website, e-mail address) may also be included. • Programs should clarify that this contact information should be used for questions related to the

program’s accreditation status. In doing so, the program should also ensure that its own contact information is clearly indicated in its materials so that the public knows how to contact the program directly with any other questions.

• Programs are encouraged to use the following format to provide this information: *Questions related to the program’s accredited status should be directed to the Commission on Accreditation:

Office of Program Consultation and Accreditation American Psychological Association 750 1st Street, NE, Washington, DC 20002 Phone: (202) 336-5979 / E-mail: apaaccred@apa.org Web: www.apa.org/ed/accreditation

_____________________________________________________________________________________________*For the purposes of this Implementing Regulation, only students that have had their doctoral degrees conferred on their transcripts are considered “graduates”. “Time to completion” is the amount of time between the date of entry into the program and the date of program completion on the official transcript.

C-26 D. Disclosure of Education/Training Outcomes and Information Allowing for Informed Decision-Making to Prospective Doctoral Students

(formerly C-20; Commission on Accreditation, May 2006; revised November 2006, July 2007, July 2010, March 2012, April 2013, March 2014, May 2014, November 2015)

Standard V of the Standards of Accreditation (SoA) requires that doctoral graduate programs provide potential students, current students, and the public with accurate information on the program and on program expectations. This information is meant to describe the program accurately and completely, using the most up-to-date data on education and training outcomes, and be presented in a manner that allows applicants to make informed decisions about entering the program. The CoA requires accredited programs to update the data tables annually and post the information in its public materials (e.g. website) by October 1 each year. Failure to update the information is as much of a concern as failure to provide the necessary information in the required format. After October 1, the Commission will review programs’ compliance with the below requirements and that the data provided are consistent with the program’s data from the Annual Report Online (ARO). _____________________________________________________________________________________ Presentation of Required Information To ensure that the required information for each program is available to the public in a consistent fashion, the following provisions are required:

• The information must all be located in a single place and be titled “Student Admissions, Outcomes, and Other Data”;

• If the program has a website, the information must be located no more than one-click away from the main/home doctoral landing page; and (see update to this provision below)

• The link from the main/home doctoral landing page to the required information must also be titled “Student Admissions, Outcomes, and Other Data”;

• The data must be presented in tables consistent with those listed at the end of this regulation. Programs may choose to provide other data to supplement the requirements of this regulation, but these tables must be provided. If the program chooses to provide supplemental information, it should be provided below the corresponding required tables.

• Table cells should not be left blank; instead, please enter a “0” if not applicable except where indicated in table

_____________________________________________________________________________________ Because the information required should include those education and training outcomes that will allow applicants to make informed and comparative decisions, the Commission requires that all doctoral programs minimally provide the following to prospective students in its public materials, including its website, if it has one: 1) time to program completion; 2) program costs (tuition and fees) and fellowships and other funding available; 3) internship acceptance rates; 4) student attrition rates; and 5) licensure outcomes. These are defined as follows: 1. Time to Completion Time to completion must be presented in two ways:

_____________________________________________________________________________________________*For the purposes of this Implementing Regulation, only students that have had their doctoral degrees conferred on their transcripts are considered “graduates”. “Time to completion” is the amount of time between the date of entry into the program and the date of program completion on the official transcript.

• First, programs must provide the mean and the median number of years that students have taken to complete the program from the time of first matriculation. These data should be provided for all graduates* in each of the past ten (10) years.

• Second, the program should provide the percentage of students completing the program in fewer than five years, five years, six years, seven years, and more than seven years.

In a text box below the table, programs must also note any admissions policies that allow students to enter with credit for prior graduate work and the expected implications for time to completion. 2. Program Costs Programs are expected to make available the total costs per student for the current first year cohort. This information should include full-time student tuition, tuition per credit hour for part-time students, and any fees or costs required of students beyond tuition costs. For example, if a program requires students to travel to attend a mandatory component of the program, the estimated costs of this travel should be included as well. Programs may also provide information regarding current adjustments to tuition including, but not limited to: financial aid, grants, loans, tuition remission, assistantships, and fellowships. Even if program cost information is provided elsewhere on another university or other site, it must be provided in the doctoral program’s materials as well. NOTE: Please enter discrete dollar values in the Program Costs table and not percentages. For instance, if the program covers students’ full costs within a category, please enter “$0” in that cell. 3. Internships Programs are expected to provide data on students’ success in obtaining internships. The program is required to report for each of the past ten (10) years:

• The total number of students who sought or applied for internships • The number and percent of total who obtained internships • The number and percent of total who obtained APA/CPA-accredited internships • The number and percent of total who obtained APPIC member internships that were not APA/CPA-

accredited (if applicable) • The number and percent of total who obtained other membership organization internships (e.g.,

CAPIC) that were not APA/CPA-accredited (if applicable) • The number and percent of total who obtained internships conforming to CDSPP guidelines (school

psychology programs only) that were not APA/CPA-accredited (if applicable) • The number and percent of total who obtained other internships that were not APA/CPA-accredited

(if applicable) • The number and percent of total who obtained paid internships • The number and percent of total who obtained half-time internships (if applicable)

NOTES: In calculating the above percentages, the program must base these on the total number of students who sought or who applied for internship in each year, including those that withdrew from the application process. To ensure readability and understanding for prospective students, Internship Placement-Table 1 and Internship Placement-Table 2 must be presented separately.

*Please refer to footnote on first page of this Implementing Regulation for definition of graduates.

4. Attrition Programs must report the number and percentage of students who have failed to complete the program once enrolled. These data should be calculated for each entering cohort by dividing the number of students in that cohort who have left the program for any reason by the total number of students initially enrolled in that same cohort. These data should be provided by cohort for all students who have left the program in the last ten (10) years or for all students who have left since the program became initially accredited, whichever time period is shorter. 5. Licensure Reporting of program licensure data is an expectation of the US Secretary of Education’s National Advisory Committee on Institutional Quality and Integrity for program accreditors, including the APA Commission on Accreditation. As such, programs must report the number and percentage of program graduates* who have become licensed psychologists within the preceding decade. In calculating the licensure percentage:

• The denominator is the total number of program graduates between 2 and 10 years ago

• The numerator is the number of these graduates (between 2 and 10 years ago) who became licensed psychologists in the past 10 years

• The licensure percentage, then, is calculated by dividing the number of graduates (between 2

and 10 years ago) who became licensed psychologists in the past 10 years by the number of graduates during the 8 year span from 2 to 10 years ago. For example, the figures reported by a program for 2017 would be number of graduates from the program between 2007 and 2017 who have achieved licensure in the past 10 years divided by the total number of students graduating from the program between 2007 and 2017

Programs may clarify their licensure rate for the public in light of their training model and program aims.

Total number of students with doctoral degree conferred on transcriptMean number of years to complete the programMedian number of years to complete the program

Time to Degree Ranges N % N % N % N % N % N % N % N % N % N % N %Students in less than 5 years b gStudents in 5 years c hStudents in 6 years d iStudents in 7 years e jStudents in more than 7 years f kNote: (b+c+d+e+f)=a each year; (g+h+i+j+k)=100 each year

2015-2016 Total

Time to Completion for all students entering the program

Outcome2011-2012 2012-2013 2013-2014 2014-2015 2016-20172008-20092007-2008

Year in which Degrees were Conferred

2009-2010 2010-2011

Also, please describe or provide a link to program admissions policies that allow students to enter with credit for prior graduate work, and the expected implications for time to completion. Please indicate NA if not applicable:

Program Costs

Description2017-2018 1st-year Cohort Cost

Tuition for full-time students (in-state)Tuition for full-time students (out-of-state)Tuition per credit hour for part-time students (if applicable enter amount; if not applicable enter "NA")University/institution fees or costsAdditional estimated fees or costs to students (e.g. books, travel, etc.)

C-27 D. Notification of Changes to Accredited Programs (formerly C-19; Commission on Accreditation, February 2005; revised October 2006, November 2015)

In accordance with Standard V.B.2 of the Standards of Accreditation (SoA) and Section 8.7 D of the Accreditation Operating Procedures (AOP), all accredited programs whether under a single administrative entity or in a consortium, must inform the accrediting body in a timely manner of changes that could alter the program's quality. The Commission on Accreditation (CoA) must be informed in advance of major program changes such as changes in degree offered, policies/procedures, administrative structure, faculty resources, supervision resources, area of emphases, or tracks/rotations. In the case of doctoral programs, this includes changes in the areas of emphasis. Programs must submit to the Office of Program Consultation and Accreditation a detailed written description of the proposed change(s) and the potential impact upon the relevant accreditation standards. The CoA will review the program change(s) and may request additional information or a new self-study. In the case of a substantive change (such as a change in consortium membership), the Commission may also determine that a site visit is needed to assess whether the revised program is consistent with the SoA. Upon completion of this review, the Commission will note the proposed change and include the information in the next scheduled review or inform the program of any needed immediate additional actions. The only exception to the policy of informing the Commission in advance is the occurrence of an unavoidable event beyond the reasonable control and anticipation of the program (e.g., educational/training site unexpectedly withdrawing from a consortium because of financial crisis; resources affected by a natural disaster). In such circumstances, it is incumbent upon the program to immediately inform the CoA in writing of the change and to include in its notification a proposed plan for maintaining program consistency with the SoA. The CoA will then proceed as above. Consultation on program changes is available from the Office of Program Consultation and Accreditation.

C-28 D. “Intent to Apply” (Commission on Accreditation, July 2015; revised October 2016)

“All programs can seek review of “intent to apply” status and “accredited, on

contingency” prior to seeking full accreditation. The application for acknowledgement of “intent” includes documentation related to key standards of accreditation. Review for this status is a document review only. The review is conducted to verify that the essential

elements are in place to begin a program and as such is not an accredited status and does not provide the public with a judgment regarding the quality of the program.

Rather if a program is approved as “intent” for accreditation, it serves as a notice to the public that the program will be seeking accreditation in the near future.” 6.1 D AOP

Overview/Logistics:

A program may seek “intent to apply” declaration at any time, including prior to or after admitting students. The “intent to apply” declaration indicates that once students are in place, the program intends to apply for an APA accredited status (either contingent or full accreditation). A program may be listed as “intent to apply” for a maximum of three years. The “intent to apply” declaration is effective as of the date of the Commission’s decision to acknowledge the declaration. If the program exceeds its three year period it will need to inform its publics and students that it is no longer designated as an “intent to apply” program. Declaration of “intent to apply” is not a requirement for an application for “accredited, on contingency” or “full accreditation.” Programs are advised to consider the time constraints associated with public notice of “intent to apply” and application for accreditation. For programs seeking the” intent to apply” declaration, the application process is primarily intended to provide the program an opportunity to systematically describe the infrastructure upon which it will be building a program consistent with the Standards of Accreditation (SoA). The Commission on Accreditation will provide formative feedback to the program in response to their application for “intent to apply.” Although the intent application includes completion and review of only certain sections of Standards I-V of the SoA, the program clearly intends to seek an accreditation status and be in compliance with all aspects of the SoA.

Process to Apply: To apply for this declaration, programs are asked to submit documentation in accordance with the self-study instructions with the provisions listed below. It is recognized that a program will have elements in place and others in development and both of which will be reviewed by the CoA for prospective alignment with the SoA. The program will address:

• Standard I, describing the type of program and degree, the administrative structure of the program, program context, structure, and resources, and its policies and procedures.

• Standard II, describing its aims, its curriculum plan with course descriptions, its plans to measure profession-wide competencies and program-specific competencies, if any, its training elements including plans for practicum and internship, its plans to measure proximal and distal outcomes, and its plan to review outcome measures to evaluate and improve the program.

• Standard III, describing selection processes and criteria for admission to the program, its plans to provide a supportive learning environment, and its plans to facilitate student success in the program, providing evaluation, feedback, and remediation, if necessary. A plan for the size of each cohort of students should be included for each year up to full implementation of the curriculum.

• Standard IV, describing the designated leader of the program who is in place, plans for recruitment of qualified faculty of the program, plans for how faculty will contribute to the development of curriculum and training experiences, evaluation of program effectiveness, and its plan for faculty sufficiency as the program develops. A plan for numbers of core faculty in place at each year of the program as it develops to full curriculum implementation should be included.

• Standard V in general disclosure and communication with prospective and current students, and will provide all materials currently available to its publics. The materials must include: o An accurate description of the “intent to apply” declaration; o A timeline for its intention to apply for “accredited, on contingency,” or “full accreditation” of

the program; o The date that the declaration expires; and o The contact information for the APA CoA.

The program is advised to consider its timeline in light of the requirements for application for accreditation status.

C-29 D. Initial Doctoral Program Accreditation (Commission on Accreditation, July 2015; October 2018)

There are two processes by which an unaccredited doctoral program may apply for initial accreditation: 1) apply first for "accredited, on contingency" status and later for full accreditation or 2) apply directly for full accreditation. Programs that seek accredited on contingency status prior to full accreditation are strongly advised to read this entire Implementing Regulation in detail before initiating their application. Programs that are applying for full accreditation may focus primarily on the Fully Accredited section below.

Accredited, On Contingency

Doctoral programs seeking “accredited, on contingency” must be reviewed on all aspects of the SoA, which involves submission of a self-study and a site visit. "Accredited, on contingency" is granted to a doctoral program when the program demonstrates initial evidence of educational quality consistent with the SoA and the capacity to meet all accreditation standards in the designated time frame." (AOP 6.1D)

Initial Application for “Accredited, On Contingency”

Timeline:

Programs that are Accredited, On Contingency are required to be awarded Full Accreditation within a specific timeline that is described below. The Commission cannot grant extensions of this required timeline, and programs that fail to achieve Full Accreditation within the prescribed time frame will lose their accredited status entirely. Therefore, it is critically important that applicant programs refrain from applying for Accredited, On Contingency status until they are certain that they 1) meet all eligibility criteria for on contingency status and 2) will meet all of the requirements to apply for Full Accreditation within the prescribed timeline below. Programs are strongly advised to consult with the Office of Program Consultation and Accreditation prior to applying for “accredited, on contingency” status, in order to ensure that they understand all requirements for this two-part sequence of applications for full accreditation.

As an accrediting body recognized by the U.S. Department of Education, the Commission on Accreditation (CoA) cannot permit a program to hold “accredited, on contingency” status for more than five years [Section 602.16(a)(2) of the Criteria for Recognition by the U.S. Secretary of Education]. By the end of this five-year window, programs that are accredited on contingency must either earn full accreditation or withdraw from accreditation. Therefore, doctoral programs seeking "accreditation, on contingency" status are again advised to carefully consider the complete timeline and eligibility criteria required to achieve both "accredited, on contingency" status and ultimately full accreditation status, as described below.

There are multiple steps in the review of applicant programs for Full Accreditation, as described in the Accreditation Operating Procedures of the Commission on Accreditation, some of which are tied to the Commission’s annual calendar of meetings. In order to ensure that applicant programs have the opportunity to obtain full accreditation within 5 years of being placed on contingency status, it is mandatory that the program be ready to apply for full accreditation within 3 years of obtaining the “accredited, on contingency” status. If the application for full accreditation is not submitted within 3 years of the “accredited, on contingency” status being awarded; or if the program is denied full accreditation; or if full accreditation is not granted by CoA within 5 years of the program entering “accredited, on contingency” status, the program will be deemed to have withdrawn from accreditation.

A program that is accredited on contingency is an APA-accredited program, and students whose doctoral graduation date falls during the period that this accreditation status is in place will be deemed to have graduated from an accredited program. Because of the time-limited nature of the "accredited, on

contingency" status, students in the program and the public must be kept informed of any change in the program’s timeline that could negatively impact full accreditation. In addition, the program must publish the date of expiration of the "accredited, on contingency" status in its public materials by stating, “The program is accredited, on contingency through the following date: _____.”

Process to Apply:

Eligibility A doctoral program is eligible to submit a self-study to attain “accredited, on contingency” status after it has enrolled a minimum of two student cohorts, one of which must be engaged in practicum training. These two cohorts must be enrolled in two different academic years, rather than in two different semesters or quarters within the same academic year. At a minimum, aggregated proximal evaluation data (described in IR C-18D) for one practicum term must be provided by the time of the site visit. Should the program not have aggregated proximal evaluations for at least one practicum term by the time of the site visit, the program will be ineligible for “accredited, on contingency” status. If these proximal data are presented solely at the time of the site visit, rather than in the original self-study, the program is required to provide a copy of the outcome data to both the site visitors and the CoA. In the event that the program has already collected proximal and distal data for discipline-specific knowledge (DSK), profession-wide competencies, and program-specific competencies, those data must be submitted with the self-study, in accordance with the instruction in Implementing Regulation (IR) C-18 D. See below for more information on the self-study submission. The Self-Study The program applying for “accredited, on contingency” status must submit a self-study that is complete in all ways, with the exception of the proximal and distal outcome data required under Standard II.D.1 and IR C-18 D. For any outcome data that are not yet available, the self-study must include plans for how the program will collect and evaluate future proximal and distal outcomes required to demonstrate minimum levels of achievement in DSK, profession-wide and program-specific competencies (if any). In summary, the program is to submit all outcome data that are available at the time of the self-study, proximal practicum data (at the site visit or in the self-study), and plans and methods for complete future proximal and distal outcome data collection, as described in IR C-18 D (including samples of data collection tools as well as the methods by which data will be collected). With the exception of the provision of complete outcome data (Standard II), each standard will be addressed with respect to the program’s plans, policies, and procedures to meet the requirements of the SoA. At this stage, the program must submit syllabi for any required courses or evaluated learning experiences, including those that have not yet been offered. In the case of required courses or evaluated learning experiences that have not yet been taught, the instructor may be listed as ‘to be determined’; however, the program must provide a plan to demonstrate how it will ensure that a qualified instructor will be identified to teach the given course, consistent with IR C-23 D (Faculty Qualifications). The program must describe faculty sufficiency to effectively administer the program as it exists at time of submission. In addition, the program must describe the plans for ensuring faculty sufficiency as the program grows to include students at all levels of matriculation and to meet the full implementation of the curriculum plan. Within three years of being granted “accredited, on contingency” status, the program is required to provide an application for full accreditation, as described below, and is required to have at least one program graduate.

Fully Accredited

Accredited (or “fully accredited”) designates a program that, in the professional judgement of the CoA, is consistent, substantively and procedurally, with the SoA.

Initial Application for Full Accreditation

Timeline:

Contingent to full To be eligible for full accreditation, the program must provide a new self-study, including proximal and distal outcome data, and have a second site visit, so that the Commission has complete materials on which to base an accreditation decision (See AOP Section 6.1 D). Programs accredited on contingency must meet all of the deadlines and criteria described below to apply for full accreditation. As described more fully above, a program that has not 1. applied for full accreditation within three years of being accredited on contingency and 2. been granted full accreditation within five years of being accredited on contingency will be deemed to have withdrawn from accreditation. Full (without previous “contingency” status) A doctoral program may apply for review of its initial application for full accreditation when it meets all basic eligibility requirements below and the self-study is complete in all respects.

Process to Apply:

Eligibility Programs applying for initial full accreditation, with or without previous “accreditation, on contingency” status, are required to have an identifiable body of students at all levels of matriculation, including at least one program graduate. The Self-Study The program is required to submit a complete self-study that demonstrates compliance with all aspects of the SoA, including both proximal and distal outcome data (see IR C-18 D). Per the SoA, programs must provide distal evidence of students’ competencies as well as data demonstrating program effectiveness in preparing students on the profession-wide competencies and any applicable program-specific competencies (Standard II.D.1.b). Note that in order to meet this requirement, a program must have at least one program graduate when it applies for full accreditation. Per IR C-18D, accredited programs are required to collect distal data from program graduates when they are 2 years and 5 years post-graduation. Programs are permitted to collect distal data at additional time points if they choose. The use of more immediate distal data for the application for full accreditation does not relieve the program of the responsibility to collect 2-year and 5-year distal data from program graduates once alumni have reached the 2- and 5-year marks. All programs are required to describe the process by which they will collect the required 2- and 5-year data and to provide all evaluation tools by which they will accomplish this data collection. It is permissible for programs to wait until the site visit to provide distal outcome data. If the distal data do not appear in the self-study, the program is responsible for providing these outcome data to both the site visitors and the CoA. If distal outcome data are not provided by the time of the site visit, the program will not be eligible for full accreditation.

C-30 D. Partnership/Consortium (Commission on Accreditation, October 2016)

A Doctoral Program may consist of, or be located under, a single administrative entity (e.g. institution, agency, school, department) which controls its program resources, or a partnership/consortium, where each administrative entity contributes to partnership/consortium program resources. A partnership/consortium is comprised of 2 or more independently administered entities, which have agreed to share resources and centralized decision-making essential to the establishment, implementation, and maintenance of a training program. The CoA seeks to understand the stability of a partnership/consortium’s shared resources through this Implementing Regulation which specifically details the components that must be in place and described via a partnership/consortial agreement when two or more independent entities meet the above criteria to provide doctoral training. The written agreement must articulate these components (a-g):

a) The nature and characteristics of the participating entities; b) The rationale for the partnership/consortium; c) Each partner’s commitment to the training/education program and its aim(s); d) Each partner’s obligations regarding contributions, financial support, and access to resources. e) Each partner’s agreement to adhere to central control and coordination of the training program by

the partnership/consortium’s administrative structure; f) Each partner’s commitment to uniform administration and implementation of the program’s

training principles, policies, and procedures addressing trainee admission, training resource access, potential performance expectations, and evaluations; and

g) Approval by each entity’s administrative authority (with authority to sign contracts for the entity) to honor this agreement including signature and date.

Consistent with IR C-27 D, any change in components a-g above or in the leadership of the programs in the partnership/consortium, must be communicated to the CoA. An individual partner (member entity) of an accredited partnership/consortium may not publicize itself as independently accredited unless it also has independently applied for and received accreditation.

Section C: Internship Programs

IR Name Old # New # SoA location

Completion of an Accredited Internship Program: Issue of Half-Time, Two-Year Internship Programs C-8 C-1 I I.A.2 Affiliated Internship Training Programs C-10 C-2 I I.B.1.a Definition of "Developed Practice Areas" for Internship Programs and the Process by which Areas May be Identified as Such C-14 C-3 I I.B.1.b Review of Applications for the Recognition of Developed Practice Areas C-14(a) C-4 I I.B.1.b

Appeal of Decisions for Areas Seeking to be added to the Scope of Accreditation as Developed Areas C-14(b) C-5 I I.B.1.b Unfunded Internship and Stipend Equity C-9 C-6 I I.B.4.a Record of Student Complaints in CoA Periodic Review C-3 C-7 I I.C.4 Profession-Wide Competencies C-8 I II.A Positive Student Identification Consistent with Higher Education Opportunity Act C-25 C-9 I II.C Interns and Use of the Title "Doctor" C-4 C-10 I II.C.1 Consistency in Internship Experiences Within a Program C-17 C-11 I II.C.2 Internship Didactics C-29 C-12 I II.C.2 Jurisdiction of Licensure for Supervisors C-15 C-13 I II.C.3 Required Supervision in Internship Programs C-15(b) C-14 I II.C.3 Telesupervision C-28 C-15 I II.C.3 Outcome Data for Internship Programs C-30 C-16 I II.D Direct Observation C-17 I II.D.1.a Selection of Interns C-7 C-18 I III.A Diversity Recruitment and Retention C-22 C-19 I III.A.2.a Diversity Education and Training C-23 C-20 I III.A.2.b Position Titles of Psychology Interns C-6 C-21 I V.A Program Names, Labels, and other Public Descriptors C-6(a) C-22 I V.A Accreditation Status and CoA Contact Information C-6(b) C-23 I V.A.1.d Notification of Changes to Accredited Programs C-19 C-24 I V.B.2 “Intent to Apply” C-25 I N/A “Accredited, on Contingency” C-26 I N/A Trainee Admissions, Support, and Outcome Data C-27 I N/A Consortium C-28 I N/A

C-1 I. Completion of an Accredited Internship Training Program: Issue of Half-Time, Two-Year Internship Programs

(formerly C-8; Commission on Accreditation, 1987; revised 1998, November 2015) Accredited internship training sites may host interns on a full-time or a half-time basis. In either case, doctoral training programs in psychology need to ensure that the students’ overall internship experience is appropriate in terms of breadth, depth and focus. Internship agencies that accept half-time students also need to ensure the same, whether or not the student plans to be at the same agency for both half years. Thus, if a student plans to divide the total internship experience among two or more agencies, it is important that the sponsoring doctoral program, the intern, and the participating internship agencies have a mutual understanding of the students’ overall plan. Students engaged in half-time internship training will complete their programs within 24 months. In an accredited setting that accepts interns for half-time placement, both years should be completed at that setting for the intern to claim completion of an accredited internship. Internship training agencies must also make clear to the public that practicum students and others who use the setting for training are not completing an accredited internship.

C-2 I. Affiliated Internship Training Programs (formerly C-10; Commission on Accreditation, March 1998; revised October 2007, November 2015)

An exclusively affiliated internship is an accredited internship that only accepts interns who are students from a specific accredited doctoral program. A partially affiliated internship is an accredited internship in which a portion of the interns accepted are students from a specific accredited doctoral program. The procedures for evaluating and designating the programs are as follows:

5. The internship and the doctoral program with which it is affiliated are site visited and accredited separately and in the same manner as other programs and internships. However, as part of their self-study reports, the internship program would designate that it is an affiliated internship.

6. The CoA understands that affiliated internships involve close integration with the affiliated doctoral

programs. However, affiliated internship programs are independently accredited and must be reviewed by the CoA as separate entities and meet all the accreditation requirements expected of a non-affiliated internship program. Affiliated internships must provide the CoA with information specific to the internship program during the course of review. As such, an internship self-study may not simply reference aspects of a doctoral program’s self-study to fulfill the internship requirements of the Standards of Accreditation. All relevant program materials must be submitted within the internship self-study, and all information (e.g. policies and procedures, outcome data, etc.) should be specific to the internship training program.

7. Any affiliated internship programs that make use of multiple independently administered entities

as training sites will be reviewed as a consortium and will be required to meet all aspects of Standard I.A.3 of the Standards of Accreditation for internship programs.

8. The internship clearly states its status as exclusively affiliated or partially affiliated in all

descriptive material and representations to the public. If approved, the affiliated internship will be listed in the American Psychologist listing for accredited internships. The listing for the internship agency will state that it is an exclusively affiliated or partially affiliated internship; the name of the accredited doctoral program also will be stated (e.g., X Internship [affiliated with Y University Training Program]).

C-3 I. Definition of “Developed Practice Areas” for Internship Programs and the Process by which Areas May be Identified as Such

(formerly C-14; Commission on Accreditation, October 2006; pursuant to changes in the scope of accreditation approved by the APA Council of Representatives in August 2006; revised November 2015)

Scope of Accreditation for Internship Programs: The CoA reviews applications from internship training programs in practice areas including clinical psychology, counseling psychology, school psychology, and other developed practice areas or in health service psychology. Definition Developed practice areas of psychology have all of the following characteristics:

• National recognition of the practice area by a national organization(s) whose purpose includes recognizing or representing and developing the practice area, by relevant divisions of the APA, or by involvement in similar umbrella organizations;

• An accumulated body of knowledge in the professional literature that provides a scientific basis for the practice area including empirical support for the effectiveness of the services provided;

• Representation by or in a national training council that is recognized, functional, and broadly accepted;

• Development and wide dissemination by the training council of doctoral educational and training guidelines consistent with the Accreditation SoA;

• Existence of the practice area in current education and training programs; • Geographically dispersed psychology practitioners who identify with the practice area and provide

such services.

Process Steps in the identification process are:

5. Application by the training council will be initially reviewed by the CoA based upon the criteria defined above to determine the eligibility of the area for public comment on its inclusion;

6. If in this initial review, the area meets the criteria for eligibility, the CoA will invite subsequent public comment as well as inviting letters of support or concern from relevant organizations;

7. Final decision by the CoA. 8. In the case of a decision to not include the area in the scope of accreditation, the training council

may file an appeal using an appeal process parallel to the current procedures for the appeal of program-level decisions. Specific procedures for that appeal will be developed.

(See Implementing Regulation B-2 for more information about changes in the scope of accreditation)

C-4 I. Review of Applications for the Recognition of Developed Practice Areas (formerly C-14(a); Commission on Accreditation, October 2007; revised October 2008, November 2015)

A program cannot be reviewed for accreditation in a developed practice area until that area has been added to the scope of accreditation. An area applying for recognition must first demonstrate training in that area at the doctoral level before programs will be recognized in that area at the internship level. Application Areas seeking to become included in the scope of accreditation must provide all information requested in the application, which is available from the Office of Program Consultation and Accreditation. Applications not following the required format will be returned without review. Staff members of the Office of Program Consultation and Accreditation will confirm receipt of the application and ensure that all required information has been provided. Staff members may request the submission of any missing information, and the application will not be reviewed by the CoA until all required materials have been provided. Areas may submit their applications at any time. However, in order to be reviewed during a specific CoA meeting, applications must be received at least 2 months prior to that meeting. A list of CoA meeting dates is available at http://www.apa.org/ed/accreditation/calendar.aspx. Applications received after that deadline will be reviewed during the next available meeting. Review Upon receipt of the area’s completed application materials, the Executive Committee of the CoA will be charged with the review of the application. The Executive Committee maintains the right to seek additional consultation and expertise in the area as necessary. Based upon its review of the record, the Executive Committee will develop a recommendation for action by the full CoA. If the full CoA believes the area meets the criteria outlined in Implementing Regulation C-3 I, then the CoA will invite public comment on inclusion of the area in the scope of accreditation as a Developed Practice Area. After review of any public comments, the CoA will make its final decision on inclusion of the area as a Developed Practice Area. However, if the area wishes to be specified by name as part of the scope of accreditation, then the application and CoA recommendation will be forwarded to the APA Council of Representatives for review.

C-5 I. Appeal of Decisions for Areas Seeking to be added to the Scope of Accreditation as Developed Practice Areas

(formerly C-14(b); Board of Educational Affairs, November 2007) A decision by the CoA not to recommend an area for inclusion in the scope of accreditation as a Developed Practice Area may be appealed to the APA Board of Educational Affairs using the process outlined for appeals of program review decisions (see Implementing Regulations D5-1 and D5-2). The Chief Executive Officer of the group or training council petitioning for recognition of the area, or the responsible administrative officer of the group may challenge a CoA decision not to recognize a proposed Developed Practice Area. Such an appeal must be received within 30 days of receipt of written notice of the CoA decision. The appeal must specify the grounds on which the appeal is made, which must be either a procedural violation or substantive error by the CoA in its review of the area’s consistency with the provisions of Implementing Regulation C-3 I. The appeal should be addressed to the President of the APA. A nonrefundable appeal fee will be charged to the appellant group or training council, such fee to be submitted with the letter of appeal. Appointment of Appeal Panel Within 30 days of receipt of the area’s letter of appeal, the APA Board of Educational Affairs will provide the group or training council with a list of six potential appeal panel candidates, no one of whom will have had affiliation with the proposed Developed Practice Area filing the appeal or with the accreditation process related to the non-recognition of the area. The Office of Program Consultation and Accreditation will determine the willingness of the potential panel members to serve, and notify the group or training council to that effect. Within 15 days, the group or training council may select three panel members from this list to serve as its appeal panel. If the group or training council does not notify the Office of Program Consultation and Accreditation of its selection within 15 days, the Board of Educational Affairs will designate three members to serve on the appeal panel. Scope and Conduct of Appeal An appeal is not a de novo hearing, but a challenge of the decision of the CoA based on the evidence before the CoA at the time of its decision. The CoA’s decision should not be reversed by the appeal panel without sufficient evidence that the CoA’s decision was plainly wrong or without evidence to support it. Accordingly, the appeal panel should not substitute its judgment for that of the CoA merely because it would have reached a different decision had it heard the matter originally. The procedural and substantive issues addressed by the appeal panel will be limited to those stated in the area’s appeal letter. If an issue requires a legal interpretation of the CoA’s procedures or otherwise raises a legal issue, the issue may be resolved by APA legal counsel instead of the appeal panel. Only the facts or materials that were before the CoA at the time of its decision may be considered by the panel. The panel will be provided with only those documents reviewed by the CoA in making its decision, the letter that notified the group or training council of the CoA’s decision, the letter of appeal, written briefs submitted by the group or training council, and reply briefs submitted by the CoA. The letter of appeal and written briefs shall not refer to facts or materials that were not before the CoA at the time the decision was made.

The appeal panel will convene a hearing at APA during one of three pre-scheduled appeal panel hearing dates. In addition to the three members of the appeal panel, the appeal hearing will be attended by one or more representatives of the group or training council representing the proposed Developed Practice Area, one or more representatives of the CoA, and staff of the Office of Program Consultation and Accreditation. APA’s legal counsel will also attend the hearing. In addition to advising APA, counsel has the responsibility to assure compliance with the above procedures and may resolve legal or procedural issues or can advise the panel regarding those issues. Decision and Report of Appeal Panel The CoA’s decision should be affirmed unless (a) there was a procedural error and adherence to the proper procedures that would dictate a different decision; or (b) based on the record before it, the CoA’s decision was plainly wrong or without evidence to support it. The appeal panel has the options of: (a) upholding the CoA decision; or (b) returning the matter to the CoA for reconsideration of its decision in light of the panel’s ruling regarding procedural violations or substantive errors. The report of the appeal panel will state its decision and the basis of that decision based on the record before the panel. The report of the panel will be addressed to the President of the APA and sent within 30 days of the hearing. Copies will be provided to the Chief Executive Officer or to the responsible administrative officer of the group or training council whose appeal was heard, the Chair of the CoA, the Chair of the Board of Educational Affairs, and the Office of Program Consultation and Accreditation.

C-6 I. Intern Funding (formerly C-9; Commission on Accreditation, October 1981; revised March 1992, November 2001, July

2011, November 2015) This Implementing Regulation clarifies the CoA’s interpretation of Standard I.B.4.a of the Standards for Accreditation (SoA) for internship programs regarding: unfunded internships, the sufficiency of intern stipends, and the equity of stipends. Unfunded Internships The Commission on Accreditation (CoA) strongly discourages the use of unfunded internship positions. The CoA understands, however, the rare or unusual circumstance in which the award of an additional unfunded internship would serve to alleviate unavoidable hardship for the potential unfunded intern candidate (e.g., remaining geographically close to an ailing family member) Examples of less clearly defensible rationales would be elective geographic preference or the specific theoretical persuasion of a desired internship program or supervisor. The CoA is in full support of internship positions being equitably funded; however, it will consider exceptional program and individual circumstances in which a program can offer quality training despite a lack of funding. In such cases, the “burden of evidence” lies with the program to demonstrate that the lack of funding does not adversely affect morale or quality of training. In circumstances in which the case for an unfunded internship would seem to be compelling, the responsibility for documenting and the accountability for articulating the rationale for the placement rest with the doctoral and internship programs, jointly. The APA Office of Program Consultation and Accreditation staff are always available for consultation, but the decision to accept unfunded interns rests with the program alone. The awarding of such positions should be documented fully in both the doctoral and internship programs’ annual reports to the CoA, and the program should anticipate that site visitors may make focused inquiry into the case circumstances resulting in the ad hoc creation of an unfunded internship position. Under virtually all “exceptional” circumstances, it would be the CoA’s expectancy that single or individual cases would be the source of such unfunded internships, but events can occur (e.g., closure of a nearby internship) that might constitute the kind of extraordinary circumstance necessitating the creation of more than one unfunded position in a given training year. However, in the view of the CoA, the routine or regular granting of one or more unfunded internship positions would not adhere to the spirit of the present CoA policy. Programs also are enjoined to avoid the explicit or implicit communication to applicants or potential applicants that unfunded internship placements might be negotiable during recruitment at any point during the recruitment cycle. Again, maneuvers by a program and student to create the appearance of a special need after the recruiting season has ended will not be seen as consonant with the spirit of the policy. Sufficiency of Funding The payment of a stipend is a concrete acknowledgement that an intern in the agency is valued and emphasizes that there is a significant training component in addition to experiential learning. While recognizing that internship stipends will not rise to the level of salaries for permanent staff psychologists, it should also be clear that compensation needs to be sufficient so as to avoid imposing an undue hardship upon the intern in terms of basic living needs.

Internship training should be funded so as to: (1) lend tangible value to the intern‘s service contribution; (2) communicate a valid and dignified standing with professional/trainee community; and (3) be set at a level that is representative and fair in relationship to both the geographic location and clinical setting of the training site. Stipends should be reasonable in comparison to other accredited internship programs in the local area. Wherever possible, basic support for health/medical insurance should be in place to protect the welfare of interns and their families. Internship programs should communicate to CoA any intentions to substantially decrease interns’ stipends, in accordance with Implementing Regulation C-24 I. Stipend Equity The CoA continues to encourage uniform stipends across positions within internship programs, including consortia or otherwise. Consistent with the SoA, the CoA recognizes that, unless there are exceptional circumstances, the resources of a consortium are expected to be pooled, including compensation for interns. In certain exceptional cases, the CoA recognizes that resource inequities might exist. In these cases, the CoA expects programs to make these inequities clear in their public materials and encourages the programs to identify how resources might be pooled across consortium participants in such a way that comparable intern compensation can be achieved

C-7 I. Record of Student Complaints in CoA Periodic Review (formerly C-3; Commission on Accreditation, October 1998; revised November 2015)

Standard I.C.4 of the Standards for Accreditation for internship programs addresses the need for accredited programs to recognize the rights of interns to be treated with courtesy and respect, to inform them of the principles outlining ethical conduct of psychologists, and to ensure that interns are aware of avenues of recourse should problems with regard to these principles arise. In accordance with Standard I.C.4 of the internship Standards of Accreditation, a program is responsible for keeping information and records of all formal complaints and grievances, of which it is aware, filed against the program and/or against individuals associated with the program since its last accreditation site visit. These records will be reviewed by the Commission on Accreditation (CoA) as part of its periodic review of programs. The CoA expects a program to keep all materials pertaining to each of the complaints/grievances filed against it during the aforementioned time period. The site visitors shall review the full record of program materials on any or all of the filed complaints/grievances.

C-8 I. Profession-Wide Competencies (Commission on Accreditation, October 2015; revised July 2017)

Introduction The Commission on Accreditation (CoA) requires that all trainees who complete accredited training programs, regardless of substantive practice area, degree type, or level of training, develop certain competencies as part of their preparation for practice in health service psychology (HSP). The CoA evaluates a program’s adherence to this standard in the context of the SoA sections that articulate profession-wide competencies at the doctoral (Section II.B.1.b), internship (Section II.A.2), and post-doctoral (Section II.B.1) levels. This Implementing Regulation refers specifically to aspects of a program’s curriculum or training relevant to acquisition and demonstration of the profession-wide competencies required in all accredited programs. The CoA acknowledges that programs may use a variety of methods to ensure trainee competence, consistent with their program aim(s), degree type, and level of training. However, all programs must adhere to the following training requirements:

• Consistency with the professional value of individual and cultural diversity (SoA Introduction, Section II.B). Although Individual and Cultural Diversity is a profession-wide competency, the CoA expects that appropriate training and attention to diversity will also be incorporated into each of the other profession-wide competencies, consistent with SoA Introduction, Section II.B.2.a.

• Consistency with the existing and evolving body of general knowledge and methods in the science and practice of psychology (SoA Introduction, Section II.B.2.d). The CoA expects that all profession-wide competencies will be grounded, to the greatest extent possible, in the existing empirical literature and in a scientific orientation toward psychological knowledge and methods.

• Level-appropriate training. The CoA expects that training in profession-wide competencies at the

doctoral and internship levels will provide broad and general preparation for entry level independent practice and licensure (SoA Introduction, Section II.B.2.b) Training at the postdoctoral level will provide advanced preparation for practice (SoA Introduction, Section II.B.2.c). For postdoctoral programs that are accredited in a specialty area rather than a developed practice area of HSP, the program will provide advanced preparation for practice within the specialty.

• Level-appropriate expectations. The CoA expects that programs will require trainee demonstrations of profession-wide competencies that differ according to the level of training provided (i.e., doctoral, internship, post-doctoral). In general, trainees are expected to demonstrate each profession-wide competency with increasing levels of independence and complexity as they progress across levels of training.

• Evaluation of trainee competence. The CoA expects that evaluation of trainees’ competence in each required profession-wide competency area will be an integral part of the curriculum, with evaluation methods and minimum levels of performance that are consistent with the SoA (e.g., for clinical competencies, evaluations are based at least in part on direct observation; evaluations are consistent with best practices in student competency evaluation).

I. Research This competency is required at the doctoral and internship levels. Demonstration of the integration of science and practice, but not the demonstration of research competency per se, is required at the post-doctoral level The CoA recognizes science as the foundation of HSP. Individuals who successfully complete programs accredited in HSP must demonstrate knowledge, skills, and competence sufficient to produce new knowledge, to critically evaluate and use existing knowledge to solve problems, and to disseminate research. This area of competence requires substantial knowledge of scientific methods, procedures, and practices. Trainees are expected to: Doctoral students:

• Demonstrate the substantially independent ability to formulate research or other scholarly activities (e.g., critical literature reviews, dissertation, efficacy studies, clinical case studies, theoretical papers, program evaluation projects, program development projects) that are of sufficient quality and rigor to have the potential to contribute to the scientific, psychological, or professional knowledge base.

• Conduct research or other scholarly activities. • Critically evaluate and disseminate research or other scholarly activity via professional publication

and presentation at the local (including the host institution), regional, or national level. Interns:

• Demonstrates the substantially independent ability to critically evaluate and disseminate research or other scholarly activities (e.g., case conference, presentation, publications) at the local (including the host institution), regional, or national level.

II. Ethical and legal standards This competency is required at the doctoral, internship, and post-doctoral levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees at all levels are expected to demonstrate competency in each of the following areas:

• Be knowledgeable of and act in accordance with each of the following:

o the current version of the APA Ethical Principles of Psychologists and Code of Conduct;

o relevant laws, regulations, rules, and policies governing health service psychology at the organizational, local, state, regional, and federal levels; and

o relevant professional standards and guidelines.

• Recognize ethical dilemmas as they arise, and apply ethical decision-making processes in order to

resolve the dilemmas.

• Conduct self in an ethical manner in all professional activities.

III. Individual and cultural diversity This competency is required at the doctoral, internship, and post-doctoral levels. Effectiveness in health service psychology requires that trainees develop the ability to conduct all professional activities with sensitivity to human diversity, including the ability to deliver high quality services to an increasingly diverse population. Therefore, trainees must demonstrate knowledge, awareness, sensitivity, and skills when working with diverse individuals and communities who embody a variety of cultural and personal background and characteristics. The Commission on Accreditation defines cultural and individual differences and diversity as including, but not limited to, age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and socioeconomic status. The CoA recognizes that development of competence in working with individuals of every variation of cultural or individual difference is not reasonable or feasible. Trainees at all levels are expected to demonstrate:

• an understanding of how their own personal/cultural history, attitudes, and biases may affect how they understand and interact with people different from themselves;

• knowledge of the current theoretical and empirical knowledge base as it relates to addressing

diversity in all professional activities including research, training, supervision/consultation, and service;

• the ability to integrate awareness and knowledge of individual and cultural differences in the

conduct of professional roles (e.g., research, services, and other professional activities). This includes the ability apply a framework for working effectively with areas of individual and cultural diversity not previously encountered over the course of their careers. Also included is the ability to work effectively with individuals whose group membership, demographic characteristics, or worldviews create conflict with their own.

Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence as they progress across levels of training. Trainees are expected to: Doctoral students:

• Demonstrate the requisite knowledge base, ability to articulate an approach to working effectively with diverse individuals and groups, and apply this approach effectively in their professional work.

Interns:

• Demonstrate the ability to independently apply their knowledge and approach in working effectively with the range of diverse individuals and groups encountered during internship.

Post-doctoral residents:

• Demonstrate the ability to independently apply their knowledge and demonstrate effectiveness

in working with the range of diverse individuals and groups encountered during residency, tailored to the learning needs and opportunities consistent with the program’s aim(s).

IV. Professional values and attitudes This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Doctoral students and Interns are expected to:

• behave in ways that reflect the values and attitudes of psychology, including integrity, deportment, professional identity, accountability, lifelong learning, and concern for the welfare of others.

• engage in self-reflection regarding one’s personal and professional functioning; engage in activities

to maintain and improve performance, well-being, and professional effectiveness.

• actively seek and demonstrate openness and responsiveness to feedback and supervision.

• respond professionally in increasingly complex situations with a greater degree of independence as they progress across levels of training.

V. Communication and interpersonal skills This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. The CoA views communication and interpersonal skills as foundational to education, training, and practice in health service psychology. These skills are essential for any service delivery/activity/interaction, and are evident across the program’s expected competencies. Doctoral students and interns are expected to:

• develop and maintain effective relationships with a wide range of individuals, including colleagues, communities, organizations, supervisors, supervisees, and those receiving professional services.

• produce and comprehend oral, nonverbal, and written communications that are informative and well-integrated; demonstrate a thorough grasp of professional language and concepts.

• demonstrate effective interpersonal skills and the ability to manage difficult communication well. VI. Assessment This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees demonstrate competence in conducting evidence-based assessment consistent with the scope of Health Service Psychology. Doctoral students and Interns are expected to:

• Demonstrate current knowledge of diagnostic classification systems, functional and dysfunctional behaviors, including consideration of client strengths and psychopathology.

• Demonstrate understanding of human behavior within its context (e.g., family, social, societal and

cultural).

• Demonstrate the ability to apply the knowledge of functional and dysfunctional behaviors including context to the assessment and/or diagnostic process.

• Select and apply assessment methods that draw from the best available empirical literature and that

reflect the science of measurement and psychometrics; collect relevant data using multiple sources and methods appropriate to the identified goals and questions of the assessment as well as relevant diversity characteristics of the service recipient.

• Interpret assessment results, following current research and professional standards and guidelines,

to inform case conceptualization, classification, and recommendations, while guarding against decision-making biases, distinguishing the aspects of assessment that are subjective from those that are objective.

• Communicate orally and in written documents the findings and implications of the assessment in

an accurate and effective manner sensitive to a range of audiences. VII. Intervention This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees demonstrate competence in evidence-based interventions consistent with the scope of Health Service Psychology. Intervention is being defined broadly to include but not be limited to psychotherapy. Interventions may be derived from a variety of theoretical orientations or approaches. The level of intervention includes those directed at an individual, a family, a group, a community, a population or other systems. Doctoral students and Interns are expected to demonstrate the ability to:

• establish and maintain effective relationships with the recipients of psychological services. • develop evidence-based intervention plans specific to the service delivery goals. • implement interventions informed by the current scientific literature, assessment findings, diversity

characteristics, and contextual variables. • demonstrate the ability to apply the relevant research literature to clinical decision making. • modify and adapt evidence-based approaches effectively when a clear evidence-base is lacking, • evaluate intervention effectiveness, and adapt intervention goals and methods consistent with

ongoing evaluation.

VIII. Supervision This competency is required at the doctoral and internship level. The CoA views supervision as grounded in science and integral to the activities of health service psychology. Supervision involves the mentoring and monitoring of trainees and others in the development

of competence and skill in professional practice and the effective evaluation of those skills. Supervisors act as role models and maintain responsibility for the activities they oversee. Trainees are expected to: Doctoral students:

• Demonstrate knowledge of supervision models and practices.

Interns:

• Apply this knowledge in direct or simulated practice with psychology trainees, or other health professionals. Examples of direct or simulated practice examples of supervision include, but are not limited to, role-played supervision with others, and peer supervision with other trainees.

IX. Consultation and interprofessional/interdisciplinary skills This competency is required at the doctoral and internship level. The CoA views consultation and interprofessional/interdisciplinary interaction as integral to the activities of health service psychology. Consultation and interprofessional/interdisciplinary skills are reflected in the intentional collaboration of professionals in health service psychology with other individuals or groups to address a problem, seek or share knowledge, or promote effectiveness in professional activities. Trainees are expected to: Doctoral students and Interns:

• Demonstrate knowledge and respect for the roles and perspectives of other professions. Doctoral students:

• Demonstrates knowledge of consultation models and practices. Interns:

• Apply this knowledge in direct or simulated consultation with individuals and their families, other health care professionals, interprofessional groups, or systems related to health and behavior.

Direct or simulated practice examples of consultation and interprofessional/interdisciplinary skills include but are not limited to:

• role-played consultation with others. • peer consultation, provision of consultation to other trainees.

C-9 I. Positive Identification of Interns Consistent with Higher Education Opportunity Act (formerly C-25; Commission on Accreditation, November 2009; revised November 2015)

Consistent with the 2008 Higher Education Opportunity Act, all accrediting agencies recognized by the U.S. Department of Education are required by federal law to engage in a review of the methods used by its accredited programs for positive identification of interns who are enrolled in any form of distance/online/electronically mediated education. As such, the APA Commission on Accreditation (CoA) requires that if a trainee in an APA-accredited program is engaged in any form of distance, online, or electronically mediated education for any part of their educational sequence (doctoral, internship, residency), the program must provide CoA with information in its self-study regarding the methods it and its host institution use to identify that trainee. In particular, the program must provide CoA with information about how it ensures that an intern who registers or receives credit for a course/seminar/didactic that uses any form of distance, online, or electronically mediated education is the same intern who participates in and completes that course. Whatever methodology is used must clearly protect intern privacy. Finally, interns must be provided with information at the time of registration or enrollment of any projected additional intern charges associated with verification of intern identity.

C-10 I. Interns and the Use of the Title “Doctor” (formerly C-4; Commission on Accreditation, date unknown; revised November 2015)

The use of the title “doctor” orally and/or in writing in the absence of an earned doctorate is a violation of the “Ethical Principles of Psychologists.” All training directors of accredited internship programs should remind their faculties/staffs and their interns of the ethical principle involved in this issue, and that a violation of the same is inconsistent with the APA guidelines.

C-11 I. Consistency in Internship Experiences Within a Program (formerly C-17; Commission on Accreditation, January 2003; revised November 2015)

The Commission recognizes that internship programs may provide training tracks or rotations that constitute different training experiences for interns. In these cases, programs must demonstrate how each track/rotation promotes the program’s overall aim(s), profession wide competencies, and program-specific competencies (if applicable) and is consistent with the Standards of Accreditation for Health Service Psychology.

C-12 I. Internship Didactics (formerly C-29; Commission on Accreditation, July 2010; revised November 2015)

The purpose of this IR is to define the types of information required from internship programs about their didactic activities. Didactic activities are defined as planned sessions of instruction that are included within the internship training curriculum. When didactic activities are used to meet or support training related to any of the program’s aim(s) or required profession wide competencies, it is the program’s responsibility to include adequate information on those didactics within the self-study to convey their nature and content. A title alone would not be sufficient; descriptions shall include an abstract/brief description of the content, learning objectives, or other additional information necessary (e.g., bibliography, readings) to demonstrate the material covered.

C-13 I. Jurisdiction of Licensure for Supervisors (formerly C-15; Commission on Accreditation, November 2001; revised November 2003, November

2015) Standard II.C.3.c of the Standards of Accreditation for Health Service Psychology for internship programs states that supervisors, “are appropriately trained and licensed, are involved in ongoing supervisory relationships with an intern and have primary professional responsibility for the cases on which supervision is provided.” In interpreting this provision for internship programs, the CoA looks to determine appropriate credentialing of the supervisor on the basis of jurisdiction governing the practice or service that is being supervised, provided the credential is generic in legally qualifying the holder for the independent practice of health service psychology. For example: When the services for which supervision is being provided are conducted in a context where a state

or provincial credential is required for practice, then the appropriate credential would be that provided by the state or province.

When services for which supervision is being provided are being conducted in a federal jurisdiction

(e.g., the VA, Bureau of Prisons), then the credentialing rules pertaining to practice in a federal setting would apply.

For those interns providing services in multiple jurisdictions (such as a Bureau of Prisons internship

that has an external community rotation), the jurisdiction governing the intern service that is being supervised would determine the appropriate supervisor credential.

When the services for which supervision is being provided are conducted in a context where a state

or province requires a credential in a specific substantive area (e.g., school psychology certification), the generic credential in health service psychology and the specific substantive area credential are both required.

C-14 I. Required Supervision in Internship Training Programs (formerly C-15(b); Commission on Accreditation, January 2007; revised November 2009, November

2015) Standard II.C.3.b-c of the Standards of Accreditation (SoA) for internship programs states that:

b. Interns receive at least 4 hours of supervision per week. c. One or more doctoral level psychologists, who are appropriately training and licensed,

are involved in ongoing supervisory relationships with an intern and have primary professional responsibility for the cases on which supervision is provided. The supervisor(s) must conduct a total of at least 2 hours per week of individual supervision with the intern during the course of the year.

The purpose of this Implementing Regulation is to clarify the supervision required for interns. Supervision is characterized as an interactive educational experience between the intern and the supervisor. This relationship: a) is evaluative and hierarchical, b) extends over time, and c) has the simultaneous purposes of enhancing the professional functioning of the more junior person(s); monitoring the quality of professional services offered to the clients that she, he, or they see; and serving as a gatekeeper for those who are to enter the particular profession (Bernard & Goodyear, 2009). Two weekly hours of individual supervision must be conducted by a doctoral-level licensed psychologist who is involved in an ongoing supervisory relationship with the intern and has primary professional clinical responsibility for the cases on which he/she provides supervision. An intern may have different primary supervisors engaged in providing individual supervision during the course of the training year. Supervisory hours beyond the two hours of individual supervision must be supervised by professionals who are appropriately credentialed for their role/contribution to the program. These 2 additional hours of supervision should be consistent with the definition of supervision provided above. These interactive experiences can be in a group or individual format and must be provided by appropriately credentialed health care providers. The primary doctoral-level licensed psychologist supervisor maintains overall responsibility for all supervision, including oversight and integration of supervision provided by other mental health professionals with psychological research and practice.

C-15 I. Telesupervision (formerly C-28; Commission on Accreditation, July 2010; revised November 2015, July 2017)

The CoA recognizes that accredited programs may utilize telesupervision in their program curriculum. At the same time, the CoA recognizes there are unique benefits to in-person supervision. Benefits to in-person supervision include, but are not limited to: opportunities for professional socialization and assessment of trainee competence, recognition and processing of subtle, nonverbal, and emotional or affective cues and interactions in supervision, all of which are essential aspects of professional development, ensuring quality, and protecting the public. Therefore, the CoA recognizes that there must be guidelines and limits on the use of telesupervision in accredited programs. Nothing in this Implementing Regulation contravenes other requirements in the Standards of Accreditation for Health Service Psychology (SoA). It only clarifies the utilization of telesupervision at the internship level. Definitions:

Telesupervision is supervision of psychological services through a synchronous audio and video format where the supervisor is not in the same physical location as the trainee. (See the definition of supervision as noted in the Glossary) In-person supervision is supervision of psychological services where the supervisor is physically in the same room as the trainee. (See the definition of supervision as noted in the Glossary)

Guidelines and Limits:

• Telesupervision may not account for more than one hour (50%) of the minimum required (as defined in the SoA) two weekly hours of individual supervision, and two hours (50%) of the minimum required (as defined in the SoA) four total weekly hours of supervision.

• Supervision beyond the minimum number of required hours may utilize methods or modalities that are deemed appropriate by the accredited program.

Programs that utilize telesupervision are expected to address generally accepted best practices. Furthermore, as with all accredited programs, programs that utilize telesupervision must demonstrate how they meet all standards of the SoA. As part of accomplishing this, programs utilizing ANY amount of telesupervision need to have a formal policy addressing their utilization of this supervision modality, including as a minimum:

• An explicit rationale for using telesupervision; • How telesupervision is consistent with their overall aims and training outcomes; • How and when telesupervision is utilized in clinical training; • How it is determined which trainees can participate in telesupervision; • How the program ensures that relationships between supervisors and trainees are established at the

onset of the supervisory experience; • How an off-site supervisor maintains full professional responsibility for clinical cases; • How non-scheduled consultation and crisis coverage are managed; • How privacy and confidentiality of the client and trainees are assured; and • The technology and quality requirements and any education in the use of this technology that is

required by either trainee or supervisor.

C-16 I. Outcome Data for Internships Programs (formerly C-30; Commission on Accreditation, July 2011; revised April 2016)

This Implementing Regulation clarifies the type of data the CoA needs to make an accreditation decision on internship programs. The CoA requires all accredited programs to provide outcome data on the extent to which the program is effective in achieving its aim(s), required profession wide competences and program-specifc competencies (if any). As stated in the Standards of Accreditation (SoA) for internships (II.D.1):

2. Evaluation of Interns’ Competencies

a. Current Interns. As part of its ongoing commitment to ensuring the quality of its graduates, the program must evaluate interns in both profession-defined and program-defined competencies. By the end of the internship, each intern must demonstrate achievement of both the profession-wide competencies and any additional competencies required by the program. For each competency, the program must:

i. specify how it evaluates intern performance; ii. identify the minimum level of achievement or performance required of the

intern to demonstrate competency; iii. provide outcome data that clearly demonstrate all interns successfully

completing the program have attained the minimal level of achievement of both the profession-wide and any program-specific competencies;

iv. base each intern evaluation in part on direct observation (either live or electronic) of the intern;

v. While the program has flexibility in deciding what outcome data to present, the data should reflect assessment that is consistent with professionally accepted practices in intern competencies evaluation.

b. Internship Program Alumni. The program must evaluate the functioning of alumni in terms of their career paths in health service psychology. Each program must provide data on how well the program prepared interns in each of the profession-wide and any program-specific competencies. The program must also provide data on interns’ job placement and licensure status.

Also, the United States Department of Education (USDE) requires recognized accrediting bodies (such as the CoA) to collect and monitor data-driven outcomes, especially as they relate to student achievement. In making an accreditation decision on a program, CoA must demonstrate that it reviews intern achievement through review of the program’s outcome data.

All accredited programs are required to demonstrate an educational/training curriculum that is consistent with program aim(s) and is designed to foster intern development of required profession-wide competencies and program specific competencies (if any). Expected minimal levels of achievements must be specified for all profession-wide competencies and program specific competencies (if any). It is each program’s responsibility to collect, present, and utilize aggregated proximal and distal outcome data that are directly tied to profession wide competencies and program-specific competencies (if any).

Definitions: Proximal data are defined as outcomes on interns as they progress through and complete the program, which are tied to the required profession-wide competencies and program-specific competencies (if any).

• Proximal data at a minimum must include the evaluations of interns by others responsible for their training (e.g., by supervisors/trainers), including mid-point and end-of-year evaluations. This is most easily accomplished when the evaluation mechanisms parallel the profession wide competencies and program-specific competencies (if any). It is expected that these data would at least include the semi-annual feedback provided to interns as required by Standard III.B of the SoA.

• While intern self-ratings, ratings of satisfaction with training, or ratings by others (e.g., peers) may be a part of proximal assessment, they are not considered sufficient outcome data in this context since they do not address the program’s success in promoting attainment of profession wide competencies and program-specific competencies (if any).

Distal data are defined as outcomes on interns after they have completed the program, which are tied to the profession-wide competencies and program specific competencies (if any).

• Distal data typically include information obtained from alumni surveys addressing former interns’ perceived assessments of the degree to which the program promoted mastery of profession wide competencies and program specific competencies (if any).

• Distal data reflecting completion of professional activities and accomplishments (e.g., licensure, employment, memberships, and affiliations), such as those found in the self-study tables, are important examples of distal outcomes but alone are not sufficient because they do not fully reflect achievement of all expected competencies.

• Although alumni surveys assessing former interns’ overall satisfaction with the training program (including the degree to which the education and training is relevant) may be an important component of a program’s ongoing self-study process, they are not considered sufficient outcome data in this context since they do not address the program’s success in promoting expected competencies.

• Although CoA does not specify the interval at which distal data should be collected, the interval should be appropriate to allow the program to evaluate its success in promoting expected competencies to determine if changes need to be made, consistent with Standard II.

Profession Wide Competencies--Level of Specificity: According to the Standards of Accreditation (Standard II.A), accredited programs are required to provide a training/educational curriculum that fosters the development of nine profession-wide competencies. Accredited programs are required to operationalize competencies in terms of multiple elements. At a minimum, those elements must reflect the content description of each PWC defined in IR C-8I, including the bulleted content, and must be consistent with the program aim(s). It is incumbent upon the program to demonstrate that there is a sufficient number of elements articulated for each PWC so as to demonstrate adequate trainee attainment of competence. Programs must assess intern performance at the level of the elements, give feedback to interns at the level of elements, but then report to CoA at the level of the PWC. Aggregated data are compilations of proximal data and compilations of distal data across interns, which may be broken down by cohort or years. Aggregate data demonstrate the effectiveness of the program as a whole, rather than the accomplishment of an individual intern over time.

• To the extent possible, data should be presented in table form using basic descriptive statistics (e.g., sample sizes, means, percentages). The program should choose statistics that allow for evaluation of whether all trainees are acquiring competencies in relation to its defined minimal levels of

achievement for required profession wide competencies and program-specific competencies (if any).

• If data are aggregated over a number of years (i.e., not broken down by cohort or year), the program needs to demonstrate how aggregating the data in this way facilitates the program’s self-improvement.

Program Specific Competencies—Level of Specificity: Accredited programs may choose to include program-specific competencies as part of their educational curriculum. These should be consistent with the program’s aim(s) and professional standards and practices of Health Service Psychology. Further, programs must demonstrate education/training to facilitate development of these competencies, appropriate mechanisms to assess intern performance on these competencies (including expected minimal levels of achievement for successful completion of the program), and its success in ensuring that interns reach expected levels of performance. Similar to the expectations for Profession Wide Competencies, programs that choose to have program-specific competencies are expected have multiple elements for each of those competencies, assess intern performance at the level of the elements, give feedback to interns at the level of elements, but then report to CoA at the level of the superordinate competency. Aggregated data must be presented in a manner that demonstrates the success of the program as a whole while allowing for an assessment of how well interns are performing in relation to defined minimal levels of achievement.

C-17 I. Direct Observation (Commission on Accreditation, July 2015; revised February 2017)

This Implementing Regulation is intended to clarify the expectations of CoA with regard to “direct observation” as described in internship Standards of Accreditation (SOA) as follows: Standard II.D.1.a.iv.

“base each intern evaluation in part on direct observation (either live or electronic) of the intern;” Standard II.D.1.a.v.

“While the program has flexibility in deciding what outcome data to present, the data should reflect assessment that is consistent with professionally accepted practices in intern competencies evaluation.”

Definitions and Guidelines: Direct observation provides essential information regarding trainees’ development of competencies, as well as the quality of the services provided, that cannot be obtained through other methods. This allows supervisors to provide a more accurate assessment and evaluation of observable aspects of trainees’ competency development regarding one or more profession-wide and program-specific competencies (if any) associated with that training experience. Direct observation includes in-person observation (e.g., in room or one-way mirror observation of client contact during an intake or test feedback session), live synchronous audio-video streaming, or audio or video recording. A training site that does not permit live observation, audio or video recording by policy is not a sufficiently unique circumstance to circumvent this requirement. To these ends, all accredited programs must verify on the evaluation form that direct observation is conducted by the immediate supervisor responsible for the activity or experience being evaluated. As indicated in the SoA (Standard III.B.1), at a minimum an internship must provide written feedback on a semiannual basis. Each of these written evaluations must be based in part on an instance of direct observation. When an intern completes multiple rotations within a training year, each is considered a unique and separate training experience and requires direct observation as part of the intern evaluation process for that rotation.

C-18 I. Selection of Interns (formerly C-7; Commission on Accreditation, October 1983; revised 1998, November 2015)

As stated in Internship Standard III.A of the Standards of Accreditation:

1. Identifiable Body of Interns. The program has an identifiable body of interns who are

qualified to begin doctoral internship training. a. They are currently enrolled in a doctoral program accredited by an accrediting

body recognized by the U.S. Secretary of Education or by the Canadian Psychological Association. If the internship accepts an intern from an unaccredited program, the program must discuss how the intern is appropriate for the internship program.

b. Interns have interests, aptitudes, and prior academic and practicum experiences that are appropriate for the internship’s training aim(s) and competencies.

c. Adequate and appropriate supervised practicum training for the internship program must include face-to-face delivery of health service psychological services.

Only those students accepted to an internship training program under the preceding conditions would be properly referred to as “interns.” It is entirely possible, however, that an internship agency might afford instructional opportunity for a psychologist or graduate student in developmental psychology, social psychology, neuropsychology, or some other academic/science area of psychology, in the course of which they may be introduced, under proper supervision, to psychological assessment and intervention/techniques. However, such experience would not properly be considered internship training, and certification of having completed an accredited internship would not be appropriate. In instances in which the program accepts interns from programs other than those in health service psychology, the CoA may raise questions similar to the following of the accredited internship training agency:

• How many of such persons are involved in any way with the accredited internship training program?

• What requirement, if any, do they impose for the time of internship training staff or other resources of the internship training program?

• How are those persons referred to while participating with the program? Is it clear to everyone what their role is, and what their purpose is in association with the program?

• Is there any certification of their participation, and if so, what is its nature?

C-19 I. Diversity Recruitment and Retention (formerly C-22; Commission on Accreditation, November 2009; revised March 2013; October 2018)

The Standards of Accreditation (SoA) state that that five principles, one of which is a commitment to cultural and individual diversity, “guide accreditation decisions, such that programs whose policies and procedures violate them would not be accredited.” Furthermore, the Commission “is committed to a broad definition of cultural and individual differences and diversity that includes, but is not limited to, age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and socioeconomic status (SoA, p. 3).” Diversity is essential to science and quality education in Health Service Psychology. The goals of diversity recruitment and retention include, but are not limited to, creating and maintaining inclusive environments and improving access to quality education and training. An inclusive environment is one in which the program creates an atmosphere that is welcoming, respectful and affirming of interns’ and faculty/staff members’ multiple identities. In accordance with Standards I.B.3, III.A.2.a and IV.B. of the Standards of Accreditation (SoA) for internship programs, an accredited internship program is responsible for making systematic, coherent, and long-term efforts to 1) attract (i.e., recruit) diverse interns and faculty/staff as well as, 2) retain diverse faculty/staff into the program. In addition, the program is responsible for assessing the effectiveness of both its recruitment and retention efforts and identifying areas of improvement. For both recruitment and retention, the program must provide program-level efforts and activities, in addition to any institutional, departmental, or other unit activities that are used. Programs are expected to seek and utilize generally accepted best practices in the field regarding recruitment and retention of diverse individuals. In planning for the recruitment and retention of diverse individuals, accredited programs should consider the following:

- A program may include institutional-level initiatives addressing diversity, but these, in and of themselves, are not considered sufficient.

- The lack of faculty/staff openings or having existing faculty/staff with strong representation of diversity, does not exempt the program from the responsibility of having a systematic, multi-year plan in place.

- Similarly, having an existing intern group with strong representation of diversity does not exempt the program from the responsibility of having a systematic, multi-year plan in place.

- The demographic information about faculty/staff and interns in the tables of the self-study and annual report is not sufficient to demonstrate a program’s compliance with Standards I.B.3., III.A.2.a and IV.B.

Recruitment The program is expected to document that it has developed and implemented a systematic plan to recruit both interns and faculty/staff from diverse backgrounds. Interns An accredited internship program should document and report in its self-study:

• that it has developed a systematic, multi-year, and multiple effort plan, implemented and sustained over time, to attract interns from a range of diverse identities;

• the concrete and specific program-level activities, approaches, and initiatives it implements to increase diversity among its interns;

• the areas of diversity recruitment in which it has had success, as well as the areas of diversity recruitment it is working to improve, recognizing the broad definition of diversity, and;

• how it examines the effectiveness of its efforts to attract diverse interns, and the steps it has taken to revise/enhance its strategies.

Faculty/Staff An accredited internship program should demonstrate and report in its self-study:

• that it has developed a systematic, multi-year, and multiple effort plan, implemented and sustained over time, to attract faculty/staff from a range of diverse identities (i.e., when there are faculty/staff openings);

• the concrete and specific program-level activities, approaches, and initiatives it implements to increase diversity among its faculty/staff;

• the areas of diversity recruitment in which it has had success, as well as the areas of diversity recruitment it is working to improve, recognizing the broad definition of diversity, and;

• how it examines the effectiveness of its efforts to attract diverse faculty/staff, and the steps it has taken to revise/enhance its strategies.

Retention The program is expected to document that it has developed and implemented a systematic plan to retain faculty/staff from diverse backgrounds. Faculty/Staff An accredited internship program is expected to describe in its self-study:

• the specific activities, approaches, and initiatives it implements to maintain diversity among its faculty/staff and ensure a supportive and inclusive work environment for its diverse faculty/staff members.

• how the program examines the effectiveness of its efforts to maintain diversity among its faculty/staff, and the steps it has taken to revise/enhance its strategies as needed.

C-20 I. Diversity Education and Training (formerly C-23; Commission on Accreditation, November 2009; revised March 2013, November 2015)

In accordance with Standard II.A.2.c for internship programs of the Standards of Accreditation (SoA), a program has and implements a thoughtful and coherent plan to provide interns with relevant knowledge and experiences about the role of cultural and individual diversity in psychological phenomena and professional practice. Although the Commission asks for demographic information about faculty/staff and interns in the tables of the self-study and annual report, the information requested is limited to the data collected in federal reports, which is not sufficient in demonstrating a program’s compliance with Standard II.A.2.c. Consistent with Standard I.B.3, as described in the internship SoA, cultural and individual diversity includes but is not limited to age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and social economic status.

An accredited program is expected to articulate and implement a specific plan for integrating diversity into its didactic and experiential training. This training should be based on the multicultural conceptual and theoretical frameworks of worldview, identity, and acculturation, rooted in the diverse social, cultural, and political contexts of society, and integrated into the science and practice of psychology. Programs are expected to train interns to respect diversity and be competent in addressing diversity in all professional activities including research, training, supervision/consultation, and service.

The program should demonstrate that it examines the effectiveness of its education and training efforts in this area. Steps to revise/enhance its strategies as needed should be documented.

C-21 I. Position Titles of Psychology Interns (formerly C-6; Commission on Accreditation, original date unknown; revised 1998; November 2015)

According to Standard V.A of the Standard of Accreditation (SoA), an internship program will have an “identifiable body of interns who have a training status at the site that is officially recognized in the form of a title or designation such as "psychology intern" (consistent with the licensing laws of the jurisdiction in which the internship is located.” The CoA recognizes that this may encompass a number of titles to which interns at training sites are referred. However, consistent with Standard V of the SoA, all accredited internship programs should be clear and consistent in their public materials about the training they offer, regardless of their agency’s local terminology in reference to interns/trainees. The internship program’s public materials should make clear that the fact that it is an accredited internship training program.

C-22 I. Program Names, Labels, and Other Public Descriptors (formerly C-6(a); Commission on Accreditation, January 2002; revised January 2003, November 2015)

What the internship program is called: Because accreditation is available to both doctoral internships and postdoctoral residencies, programs must portray themselves in a manner that does not misrepresent their level of training. Thus, in general, doctoral internship programs should not describe themselves as “residencies,” and postdoctoral residency programs should not describe themselves as “internships.” It is recognized, however, that agencies and institutions providing training at either or both of these levels may have local or state regulations about, or restrictions on, the terms used to portray programs that prepare individuals for practice. In the event that it is not possible to use the term “internship” for doctoral internship training programs, and “residency” for postdoctoral residency training programs, the program in question should include in all public documents (e.g., brochures, materials, web sites, certificates of completion) a statement about the program’s accredited status. Programs are to avoid reference to themselves as “pre-doctoral” internships in all public materials, including certificate of completion. “Internship” or “doctoral internship” are acceptable.

Preferred: “Internship in Clinical Psychology” “Internship in Health Service Psychology” “Doctoral internship in Counseling Psychology”

Example with accurate accreditation status: “Residency in Clinical Psychology, accredited as a doctoral internship in health service

psychology” How the program describes itself: It is recognized that programs have many possible reasons why they choose the self-descriptors or labels that they do. Some are bound by state law, others by institutional regulation, and others simply seek to assign a label to their program to describe their focus to the public. Given that these self-descriptors do not necessarily coincide with recognized areas of accreditation, any program whose label does not reflect the specific area in which it received accreditation must portray its accredited status in a manner consistent with the SoA.

Preferred: • “Doctoral program in clinical psychology” • “Internship in clinical psychology” • “Internship in health service psychology”

Examples with accurate accreditation status:

• “Doctoral program in medical psychology, accredited as a program in clinical psychology” • “Internship in pediatric psychology, accredited as a doctoral internship in health service

psychology” Certificate of completion of internships: The certificate of completion for doctoral internships should reflect the program's substantive area of professional psychology, or indicate that the program is an internship in "health service psychology.”

Examples: “X has successfully completed a doctoral internship in clinical psychology” “Y has successfully completed a doctoral internship in health service psychology”

C-23 I. Accreditation Status and CoA Contact Information (formerly C-6(b); Commission on Accreditation; November 2010; revised March 2015, November 2015)

Standard V.A.1.d of the Standards of Accreditation for Health Service Psychology (SoA) for internship programs states that the program must include in its public materials:

“d. The program provides its status with regard to accreditation, including the specific training program covered by that status, and the name, address, and telephone number of the Commission on Accreditation. The program should make available, as appropriate through its sponsor institution, such reports or other materials that pertain to the program’s accreditation status.”

Programs that are accredited by agencies recognized by the U.S. Department of Education (e.g., CoA) are required to provide the contact information for the accrediting body when the accreditation status is cited in public materials. The intent of this Implementing Regulation is to clarify how this information should be presented in order to ensure consistency across programs as well as provide useful information to the public. Accreditation status: • The only official accredited statuses are: “Accredited on contingency,” “Accredited,” “Accredited on

probation,” and “Accredited inactive,” • Programs may indicate their appropriate status (see above) by referring to “APA” accredited or

accredited “by the Commission on Accreditation of the American Psychological Association.” For example, “APA-accredited,” “APA-accredited on contingency,” “accredited by the Commission on Accreditation of the American Psychological Association, “accredited on contingency by the Commission on Accreditation of the American Psychological Association,” etc.

• Programs should not use the term “APA-approved,” since at APA this term is used to denote approved sponsors of continuing education rather than accreditation of academic/training programs.

• If there are multiple programs in the same department, institution, or agency, it should be clearly indicated in public materials which programs are APA-accredited. Multiple accredited programs should refer to their accredited status individually and in accordance with IR C-22 I.

CoA contact information: • In ALL public documents, including the program’s website (if applicable), where the program’s

accreditation status is cited as above, the name and contact information for the CoA must be provided. • Information must include the address and direct telephone number for the APA Office of Program

Consultation and Accreditation. Other information (i.e., website, e-mail address) may also be included. • Programs are to clarify that this contact information should be used for questions related to the

program’s accreditation status. In doing so, the program should also ensure that its own contact information is clearly indicated in its materials so that the public knows how to contact the program directly with any other questions.

• Programs are encouraged to use the following format to provide this information: *Questions related to the program’s accredited status should be directed to the Commission on Accreditation:

Office of Program Consultation and Accreditation American Psychological Association 750 1st Street, NE, Washington, DC 20002 Phone: (202) 336-5979 / E-mail: apaaccred@apa.org Web: www.apa.org/ed/accreditation

C-24 I. Notification of Changes to Accredited Programs (formerly C-19; Commission on Accreditation, February 2005; revised October 2006, November 2015)

In accordance with Standard V.B.2 of the Standards of Accreditation (SoA) and Section 8.7 I of the Accreditation Operating Procedures (AOP), all accredited programs, whether under a single administrative entity or in a consortium, must inform the accrediting body in a timely manner of changes that could alter the program's quality. The Commission on Accreditation (CoA) must be informed in advance of major program changes such as changes in policies/procedures, administrative structure, staff resources, supervision resources, area of emphases, or tracks/rotations. This includes new, additional, or eliminated rotation experiences or training sites. For example, consortium programs must inform the CoA of any substantial changes in structure, design or training sites. It also includes requests for changes in accreditation status (e.g., request to transition from inactive back to active status prior to recruitment). Programs must submit to the Office of Program Consultation and Accreditation a detailed written description of the proposed change(s) and the potential impact upon the relevant accreditation standards. The CoA will review the program change(s) and may request additional information or a new self-study. In the case of a substantive change (such as a change in consortium membership or major change in training focus), the Commission may also determine that a site visit is needed to assess whether the revised program is consistent with the SoA. Upon completion of this review, the Commission will note the proposed change and include the information in the next scheduled review or inform the program of any needed immediate additional actions. The only exception to the policy of informing the Commission in advance is the occurrence of an unavoidable event beyond the reasonable control and anticipation of the program (e.g., educational/training site unexpectedly withdrawing from a consortium because of financial crisis; resources affected by a natural disaster). In such circumstances, it is incumbent upon the program to immediately inform the CoA in writing of the change and to include in its notification a proposed plan for maintaining program consistency with the SoA. The CoA will then proceed as above. Consultation on program changes is available from the Office of Program Consultation and Accreditation.

C-25 I. “Intent to Apply” (Commission on Accreditation, October 2015; revised October 2016)

All programs can seek public notification of “intent to apply” prior to seeking accreditation. The application for intent to apply includes documentation related to key standards of the SoA. This review is a document review only and does not include a site visit. The review is conducted to verify that the essential elements are adequately described. “Intent to apply” is a declaration and is not an accredited status. This declaration does not constitute a judgment by the CoA regarding the quality of the program. Rather, this serves as public notice of the program’s intent to seek accreditation in the near future.

Overview/Logistics:

A program may seek “intent to apply” declaration at any time, including prior to or after admitting interns. The “intent to apply” declaration indicates that once interns are in place, the program intends to apply for an APA accredited status (either “on contingency” or full accreditation). A program may be listed as “intent to apply” for a maximum of two years. The “intent to apply” declaration is effective as of the date of the Commission’s decision to acknowledge the declaration. If the program exceeds its two year period it will need to inform its publics and interns that it is no longer designated as an “intent to apply” program. Declaration of “intent to apply” is not a requirement for an application for “accredited, on contingency” or “full accreditation.” For programs seeking the” intent to apply” declaration, the application process is intended to provide the program an opportunity to systematically describe the infrastructure upon which it will be building a program consistent with the Standards of Accreditation (SoA). The Commission on Accreditation will provide feedback to the program in response to their application for “intent to apply.” Although the application includes completion and review of only certain sections of Standards I-V of the SoA, the program clearly intends to seek an accreditation status and be in compliance with all aspects of the SoA.

Process to Apply: To apply for this declaration, programs are asked to submit documentation in accordance with the self-study instructions with the provisions listed below. It is recognized that a program will have elements in place and others in development, both of which will be reviewed by the CoA for prospective alignment with the SoA. The program must address the following:

• Standard I, describing the type of program, institutional and program setting and resources, program policies and procedures, and program climate.

• Standard II, describing its aim(s), required profession-wide competencies, its program-specific competencies (if any), its specialty competencies (for residency programs, if applicable), its learning elements to develop competencies, its plans to measure proximal and distal outcomes, and its plan to review outcome measures to evaluate and improve the program.

• Standard III, describing its plan for intern selection processes and criteria, including a plan for recruitment of interns who are diverse, and its plan for providing evaluation, feedback, and remediation, if necessary, to trainees.

• Standard IV, describing the designated director of the program who is in place, plans for providing a sufficient number of appropriately qualified supervisors to accomplish the program’s aim(s), and plans for the recruitment and retention of supervisors/staff who are from diverse backgrounds.

• Standard V in the areas of general disclosure and communication with prospective and current trainees, and its plan for communicating with the doctoral program (in the case of internship

programs). Additionally, the program will provide all materials currently available to its publics. These materials must include: o The program’s timeline to apply for “accredited, on contingency,” or “full accreditation;” o The date that the declaration expires; and o The contact information for the APA CoA.

The program is advised to consider its timeline in light of the requirements for application for accreditation status.

C-26 I. “Accredited, on Contingency” (Commission on Accreditation, October 2015)

"Accredited, on contingency" is an accredited status that reflects a program’s adherence to the Standards of Accreditation (SoA). Programs seeking “accredited, on contingency” will be reviewed for adherence with all aspects of the SoA. Programs will be granted this status if the internship program sufficiently meets all standards with the exception of outcome data on interns while they are in the program and after program completion. Process to Apply: Programs may apply for "accredited, on contingency" status prior to the arrival of interns on site provided that interns will be on site by the time of the site visit. Programs applying for “accredited, on contingency” status are not required to provide outcome data at the time of application, though they must submit any proximal and distal data collected to date. If data are presented at the time of the site visit, the program must send a copy of these data to the CoA. The internship program will submit a self-study detailing all SoA components except a complete set of outcome data. However, the self-study must include plans for how the program will evaluate proximal and distal outcomes required to demonstrate minimum levels of achievement in profession-wide competencies and program-specific competencies (if any). Components of the self-study submission for “accredited, on contingency”: With the exception of the provision of complete outcome data (Standard II), each standard will be addressed with respect to the program’s plans and policies to meet the requirements of the SoA. The program must submit its evaluation plans and forms to evaluate intern outcomes and, when possible, provide existing outcome data. Term of accredited, on contingency status: The maximum amount of time an internship program can be “accredited, on contingency” is two years for a program lasting one year, or four years for a program that is half-time for two years in duration. The program is advised to consider its timeline in light of requirements to apply for full accreditation status. To apply for full accreditation, programs must provide aggregated proximal and distal data. Interns in the program as well as the public must be kept informed of any change in the program’s timeline that could negatively impact accreditation. Such notice must include current information in all the program’s public documents (e.g., website, brochure, APPIC Directory listing). Additionally, the program’s public documents must refer all interested parties to the CoA website, on which is maintained a current listing of accredited program statuses. The program must publish the date that the outcome data are due to move from “accredited, on contingency” to full accreditation, as well as the consequences of not submitting data at that time in its public materials. In the event that a program does not provide required proximal and distal data at the end of two years (four years for 2-year, half-time programs), the program will be considered to have voluntarily withdrawn from accreditation. Consistent with 8.2(b)I of the AOP, “failure to do so (provide outcome data) will lead to the program’s being deemed to have withdrawn from accreditation, following completion of the program by the interns currently on-site at the program.” That is, if the program is deemed to have voluntarily withdrawn from accreditation, interns in the program at the time will have completed an accredited program. Programs that submit proximal and distal data will be eligible for an additional three years as a “fully accredited” program.

C-27 I Trainee Admissions, Support, and Outcome Data (Commission on Accreditation, April 2016)

Standard V.A. of the Standards of Accreditation for Internship Programs requires that programs provide potential and current trainees and the public with accurate information on the program and on program expectations. This information is meant to describe the program accurately and completely, using the most up-to-date data about important admissions, support, and outcome variables, and must be presented in a manner that allows applicants to make informed decisions about entering the program.

The CoA requires each accredited program to provide information in its public materials regarding program admissions expectations, program support provided to interns, and initial post-training placement in a standardized way. This information is required to be posted in the program’s public material(s) (e.g., website, brochure), and must be updated annually. This information will be reviewed by the CoA as part of periodic program review.

Presentation of Required Information

To ensure that the required information for each program is available to the public in a consistent fashion, programs are required to update information annually, no later than September 1.

• The information must all be located in a single place and be titled “Internship Admissions, Support, and Initial Placement Data”;

• If the program has a website, the information must be located no more than one click away from the main/home program landing page (e.g., within the program’s online brochure);

• If the program has more than one source of public materials (e.g., website and brochure), the information must be included in the primary recruiting document used to educate potential applicants about the program. For instance, if a brief brochure is provided and then applicants are directed to a website, then the information would be located on the website. Alternatively, if a program has a website “introductory page” and then applicants are instructed to download an extensive brochure, the information can be contained in the brochure;

• Table cells must not be left blank; instead, please enter “NA” if not applicable; • The data must be presented in tables consistent with those listed at the end of this regulation.

Programs may choose to provide other data to supplement the requirements of this regulation, but these tables must be provided. If the program chooses to provide supplemental information, it should be provided below the corresponding required tables.

• While consortium programs are expected to pool resources and thus provide equivalent financial and other benefit support across sites, the CoA recognizes that there are instances in which this is not possible. In those instances, consortium sites must replicate the table titled “Financial and Other Benefit Support for Upcoming Training Year” for each site as necessary to ensure accurate representation of support available.

INTERNSHIP PROGRAM TABLES

Date Program Tables are updated:____________

Internship Program Admissions Briefly describe in narrative form important information to assist potential applicants in assessing their likely fit with your program. This description must be consistent with the program’s policies on intern selection and practicum and academic preparation requirements: Does the program require that applicants have received a minimum number of hours of the following at time of application? If Yes, indicate how many: Total Direct Contact Intervention Hours N Y Amount Total Direct Contact Assessment Hours N Y Amount Describe any other required minimum criteria used to screen applicants:

Financial and Other Benefit Support for Upcoming Training Year* Annual Stipend/Salary for Full-time Interns Annual Stipend/Salary for Half-time Interns Program provides access to medical insurance for intern? Yes No

If access to medical insurance is provided Trainee contribution to cost required? Yes No Coverage of family member(s) available? Yes No

Coverage of legally married partner available? Yes No Coverage of domestic partner available? Yes No Hours of Annual Paid Personal Time Off (PTO and/or Vacation) Hours of Annual Paid Sick Leave In the event of medical conditions and/or family needs that require extended leave, does the program allow reasonable unpaid leave to interns/residents in excess of personal time off and sick leave?

Yes No

Other Benefits (please describe)

* Note. Programs are not required by the Commission on Accreditation to provide all benefits listed in this table.

Initial Post-Internship Positions (Provide an Aggregated Tally for the Preceding 3 Cohorts)

2012-2015 Total # of interns who were in the 3 cohorts Total # of interns who did not seek employment because they returned to their doctoral program/are completing doctoral degree

PD EP Community mental health center Federally qualified health center Independent primary care facility/clinic University counseling center Veterans Affairs medical center Military health center Academic health center Other medical center or hospital Psychiatric hospital Academic university/department Community college or other teaching setting Independent research institution Correctional facility School district/system Independent practice setting Not currently employed Changed to another field Other Unknown

Note: “PD” = Post-doctoral residency position; “EP” = Employed Position. Each individual represented in this table should be counted only one time. For former trainees working in more than one setting, select the setting that represents their primary position.

IR C-28 I. Consortium (Commission on Accreditation, October 2016; February 2019)

I. Development of Consortium An internship training program may consist of, or be located under, a single administrative entity (e.g. institution, agency, school, department) that controls all program resources, or within a consortium, where more than one administrative entity contributes to the consortium program resources. A consortium is, therefore, comprised of 2 or more independently administered entities, that have agreed to share resources and have developed centralized decision-making for the establishment, implementation, and maintenance of a training program. The CoA seeks to understand the stability of a consortium's shared resources through this Implementing Regulation which specifically details the components that must be in place and described via a consortial agreement when two or more independent entities meet the above criteria to provide internship training. The written consortial agreement must include and articulate these components (a-h):

a) The nature and characteristics of the participating entities; b) The rationale for the consortial partnership; c) Each partner's commitment to the training/education program and its aim(s); d) Each partner’s obligations regarding contributions, financial support (see IR C-6 I), and access to

resources. e) Each partner's agreement to adhere to central control and coordination of the training program by

the consortium's administrative structure; f) Each partner's commitment to uniform administration and implementation of the program's training

principles, policies, and procedures addressing trainee admission, training resource access, potential performance expectations, and evaluations;

g) Each partner’s commitment to ensure continuation of training for interns in the consortium, particularly if at least one partner leaves the consortium; and

h) Approval by each entity's administrative authority (with authority to sign contracts for the entity) to honor this agreement including signature and date.

Consistent with IR C-24 I, any change in components a-g above and/or in the leadership of the programs in the consortium, must be communicated to the CoA. An individual consortial partner (member entity) of an accredited consortium is not and may not publicize itself as independently accredited unless it also has independently applied for and received accreditation as an independently accredited program. II. General Information for a Currently Accredited Consortium Undergoing Dissolution or the Development of New Consortium When One or More Member Entities is Currently Accredited

Given the differences in consortium programs, transition processes are complex. An accredited program that is seeking to form or dissolve a consortium is strongly advised to consult with the Office of Program Consultation and Accreditation early in the planning process. Further, consistent with IR C-24 I, the CoA must be informed in advance of such major program changes as well as the intended timeframe of the planned transition.

Per Accreditation Operating Procedures and IR D.4-6, the CoA’s responsibility for accreditation extends to programs and not individuals completing programs; therefore, the accreditation status of a program on the final day of the internship year is the status that is to be designated on internship completion certificates.

Consistent with Standard V of the Standards of Accreditation, programs must be accurately and completely described in documents that are available to current interns, applicants, and the public. It is especially important for all accredited programs (independent or consortium) to communicate clearly in public materials to current and prospective interns. Such communication should include the current accreditation status of the program, the accreditation status for which the member entity is applying, and the specific training experiences of each program. In addition, communication of the program’s decision to dissolve or develop a consortium during the training year and how these changes may impact accreditation status is to be included. The general procedures and guidelines for reviewing applications are outlined in the Accreditation Operating Procedures. An expedited review process for any transition application for member entities in a consortium cannot be guaranteed. III. Specific Information Related to the Dissolution of an Accredited Consortium Member entities that comprise an existing accredited consortium may wish to separate and become independently accredited. Transitioning from being a member entity of an accredited consortium to becoming an independently accredited internship program requires each independent member entity to apply for accreditation as a separate program. When the member entity decides to separate from the consortium it may choose to: Option 1) separate entirely from the consortium and then apply for contingent or full accreditation status or: Option 2) remain in the accredited consortium while concurrently applying as an individually accredited program. In making the decision regarding which option to choose, the member entity must consider the impact of such changes on the current and/or incoming intern cohort(s). If the member entity chooses Option 1, the following applies:

a) The member entity may decide to apply through two mechanisms available to all applicants: 1) “accredited, on contingency” status or 2) full accreditation. During the time that the program has separated from the consortium and has not yet been independently accredited, the program is not accredited. A member entity may also seek to declare intent to apply, consistent with IR C-25 I.

b) If the member entity applies for “accredited, on contingency” status then it must meet all requirements in IR C-26 I.

c) If the member entity applies for full accreditation it is expected to provide proximal and distal outcome data, consistent with IR C-16 I. These data must be specific to the independent site applying for accreditation. In certain cases, when consortium program data is easily attributed to the program that is seeking independent accreditation, data that has been collected during the consortium time period may be used as part of that included in an application for full accreditation. Programs seeking to do this should consult with the Office of Program Consultation and Accreditation.

If the member entity chooses option 2, the following will occur:

a) The member entity must simultaneously meet all Standards of Accreditation as the consortium member entity AND the requirements for one of the other applicant options: “accredited, on contingency”, as outlined in IR C-26 I or full accreditation.

b) Consistent with IR C-24 I, the consortium must communicate to the CoA how it will be able to meet the Standards of Accreditation without the components that the withdrawing member entity was contributing to the consortium.

c) The consortium agreement must be maintained during the transition period.

In the situation of a two-member consortium, if one-member entity withdraws from the consortium, then neither program is accredited as a consortium or as an individual program at the time of separation unless independent accreditation has already been attained by the separating entity(ies). Each member entity may decide to apply for accreditation as outlined in IR C-26 I for “accredited, on contingency” or for full accreditation. If they have separated, neither of the member entities may advertise themselves as independently accredited programs until the CoA has reviewed and approved the accreditation for each independent program. IV. Specific Information Related to Development of a New Consortium if One or More Member Entity(ies) is Currently Independently Accredited The following parameters do not apply to programs that are already accredited as a consortium and wish to add member entities into the existing accredited consortium. The addition of a consortium member to an accredited consortium should be communicated to CoA as a substantive change, consistent with IR C-24 I. In the event that two or more independent programs (at least one of which is already independently accredited) wish to enter into a newly developed consortial agreement, they may decide to apply through two mechanisms available to all applicants: 1) accredited on contingency status or 2) full accreditation. Two or more independent programs may also seek to declare the intent to apply as a consortium consistent with IR C-25 I. The accreditation status of one independently accredited program does not transfer to any other unaccredited member entity(ies) when programs join together; the unaccredited program is not and may not advertise as an accredited program or member of an accredited consortium until the consortium has applied for and received accreditation. If the independently accredited program chooses to maintain independent accreditation while concurrently applying for an accreditation status as a consortium program, then the independent program must meet all Standards of Accreditation as an independently accredited program AND as a consortium member entity. At a minimum, the basic integrity of the independent program and the training aim(s), profession-wide and any program-specific competencies must be maintained during the transition period. Since more than one already independently accredited program may be transitioning to a consortium, it follows that each program may have additional or more refined aims and program-specific competencies that must be clarified. In addition, the program must clarify the resources (i.e., supervision, space, clerical support) available to the consortium entity and to the independently accredited program. These resources may overlap as long as both programs remain in compliance with the Standards of Accreditation, but there must be sufficient resources to maintain the programs. During the transition period, the independently accredited program and the accredited consortium may not advertise that the independent program is a member entity until the accreditation status for the consortium program has been approved by the CoA.

Section C: Postdoctoral Programs

IR Name Old # New # SoA location Accreditation Process for Postdoctoral Residencies C-11(a) C-2 P I Postdoctoral Residency Program Transitions C-11(c) C-3 P I Focus areas within Major Area of Training Postdoctoral Residency Programs C-11(d) C-4 P I.A.1

Postdoctoral Residency Specialty Practice Areas C-11(b) C-5 P I.A.1.b Diversity Recruitment and Retention C-22 C-6 P I.C.1.a Record of Resident Complaints in CoA Periodic Review C-3 C-7 P I.C.5 Statement on Number of Supervisors in Postdoctoral Residencies C-13 C-8 P II.C.4 Profession-Wide Competencies C-9 P II.B.1 Diversity Education and Training C-23 C-10 P II.B.1.b Postdoctoral Residency Didactics C-29 C-11 P II.C.1 Positive Resident Identification Consistent with Higher Education Opportunity Act C-25 C-12 P II.C.2 Jurisdiction of Licensure for Supervisors in Postdoctoral Residencies C-15(a) C-13 P II.C.4 Required Supervision in Internship and Postdoctoral Programs C-15(b) C-14 P II.C.4 Telesupervision C-28 C-15 P II.C.4

Outcome Data for Internships and Postdoctoral Residency Programs C-30 C-16 P II.D.1 Direct Observation C-17 P II.D.1.b.ii Program Names, Labels, and other Public Descriptors C-6(a) C-18 P V.A Accreditation Status and CoA Contact Information C-6(b) C-19 P V.A.1.b Notification of Changes to Accredited Programs C-19 C-20 P V.B.2 “Intent to Apply” C-21 P N/A “Accredited, on Contingency” C-22 P N/A Trainee Admissions, Support, and Outcome Data C-23 P N/A Consortium C-24 P N/A

C-2 P. Accreditation Process for Postdoctoral Residencies (formerly C-11(a); Commission on Accreditation, January 2000; revised January 2003; October 2004;

July 2010; November 2015)

Principles:

1. Postdoctoral residencies may be accredited as programs preparing individuals for practice at an advanced level in a major area of training or in a specialty practice area.

2. Accreditable specialty practice areas include only those recognized by broad professional endorsement, as defined in Implementing Regulation C-5 P.

3. All postdoctoral residency review processes will include a preliminary review according to the Standards of Accreditation for Health Service Psychology (SoA).

4. Certificates of completion provided to residents that provide information about practice areas for which the program prepares residents must reflect the practice areas (major area of training or specialty) in which it was reviewed for accreditation.

5. The cost of the site visit to a postdoctoral residency program is linked to the number of visitors to the program.

6. The cost of the application and annual fee for all postdoctoral residency programs is the same for all programs as outlined below under Formats 1, 2, and 3.

7. In its self-study, the postdoctoral residency program is asked to indicate the Format (1, 2 or 3) it will follow.

Format #1 – Major Area of Training Programs Postdoctoral Residency Training in a Major Area of Training: Programs using this self-study format are those in Clinical, Counseling, School Psychology and other developed practice areas that provide greater depth of training than that which occurs during the internship training year. These programs articulate aim(s), Level 1 competencies required of all postdoctoral fellowship programs, and any Level 2 competencies integral to achieving the program’s aim(s), that apply to all postdoctoral residents. In the program, residents may have a greater exposure to focused emphases within the major area of training. These focused emphases can occur through rotations. Examples of such areas include, but are not limited to, substance abuse, PTSD, etc. Training programs in traditional practice areas that receive approval for a site visit will be visited by two site visitors who represent the major area of training. Certificates of completion from programs using this format describe only the major area of training. Format #2 – Specialty Practice Programs Postdoctoral Residency Training in Specialty Practice Areas: Programs using this self-study format have as a major goal the training of residents in a recognized specialty. Specialties are limited to those meeting the definition contained within CoA’s Implementing Regulation C-5 P. Programs applying for accreditation as a specialty indicate how they adhere to the SoA and to the education and training guidelines of the designated specialty. Aim(s), and “Level 3 Competencies” within the training program must be consistent with those of the designated specialty area (see SoA Standard II.A.3). Training programs in specialty practice areas that receive approval for a site visit will be visited by two site visitors, one of whom has

expertise in the specialty practice area. Certificates of completion for programs using this format describe only the specialty practice area of training. Format #3 – Multiple Practice Programs Postdoctoral Residency Training in Multiple Practice Areas: Programs using this format include combinations of two or more major area(s) of training and/or specialty practice programs organized within the same training agency or institution that conform to the definition provided in IR C-5 P. Training agencies and institutions applying with multiple practice programs indicate how they adhere to the SoA and to the postdoctoral training guidelines of the designated specialty practice areas. For example, multiple practice area postdoctoral residency programs that provide training in Clinical Psychology and in Clinical Neuropsychology, where the field follows a two-year training sequence, would need to adhere to the specialty area guidelines in that specialty. The multiple practice program will define its aim(s), how it provides training in the required Level 1 competencies, and define appropriate Level 2 (if any) and Level 3 competencies for Clinical Neuropsychology. The number of site visitors to a multiple practice program will be determined by the number of major area(s) of training and/or specialty practice residencies within the program. Certificates of completion for programs using this format indicate the major area of training or specialty practice training program completed by each resident. Multiple practice postdoctoral programs under Format #3 pay only a single application and annual fee in the same fashion as programs representing Formats #1 or #2. When there is a discrepancy across programs in the year at which the next site visit is due (e.g., a Clinical Psychology residency is accredited for 10 years and a Clinical Health Psychology residency is accredited for 3 years), the programs may request a single reaccreditation site visit in 3 years or independent visits in 3 and again in 10 years. Applicant and accredited multiple practice postdoctoral residency programs are encouraged to consult with the Office of Program Consultation and Accreditation for the purpose of maximizing the clarity and comprehensiveness of the self-study that is submitted to the Commission on Accreditation.

C-3 P. Postdoctoral Residency Program Transitions (formerly C-11(c); Commission on Accreditation, July 2010; revised November 2015)

Consistent with Implementing Regulation C-1 P, an agency or institution with an existing postdoctoral residency training program in a major area of training (Clinical, Counseling, School Psychology, or other developed practice area) may wish to develop and seek accreditation in one or more specialty practice areas. For example, an institution or consortium with an accredited postdoctoral program in Clinical Psychology may develop an associated postdoctoral program in Clinical Health Psychology and seek accreditation as a multiple practice program, consistent with Format #3 described in IR C-1 P. Alternatively, an existing accredited major area of training program with multiple emphasis areas may wish to develop all emphasis areas into separately accredited specialty programs. For example, an agency or institution with an accredited residency program in Clinical Psychology may develop specialty practice programs in Clinical Child Psychology and Clinical Neuropsychology, with the intention of eventually discontinuing the Clinical Psychology program. In taking this action, the program will want to ensure that the transition from a single program to multiple specialty programs does not jeopardize accreditation of the existing program. Accredited postdoctoral programs planning to add new specialty practice postdoctoral programs, or to transition from a traditional practice program into one or more specialty practice program should consider the following factors in making the transition:

• Programs considering a transition must consult with the Office of Program Consultation and Accreditation early in the planning process. Further, consistent with IR C-20 P, the Commission on Accreditation must be informed in advance of such major program changes as well as the intended timeframe of the planned transition.

• For a program to maintain accreditation as a major area of training program while developing

specialty training with the intent of seeking accreditation in one or more specialty practice programs as a multiple practice program, the existing traditional practice program must continue to maintain compliance with the SoA. At a minimum, the basic integrity of the major area of training program’s training aim(s) must be maintained. Since all or part of the program would be transitioning to a specialty program, it follows that the program may have additional or more refined aims and competencies.

• Transitioning from an accredited program in a major area of training program to a specialty program or multiple practice programs is a complex process. The CoA makes accreditation decisions individually for each program within multiple practice programs. It is therefore possible for an existing accredited program to be reaccredited and a newly developed applicant program to fail to receive accreditation. In order to avoid jeopardizing existing accreditation, host agencies or institutions are advised to continue administering their existing accredited program throughout the new program accreditation process.

• Specialty practice programs seeking accreditation within an agency or institution should clearly distinguish themselves from the major area of training programs already accredited within the same agency or institution. Consequently, as part of their own self-study, applicant specialty programs are advised to address Standards II, III, and IV for the existing program as well as Standards II, III, and IV for each of the specialty programs seeking accreditation. During the transition, postdoctoral residents can be considered part of the existing accredited program and the applicant specialty program.

• Consistent with Standard V, programs should be accurately and completely described in documents

that are available to current residents, applicants, and the public. Training experiences within an existing, accredited program must be clearly distinguished from training experiences that are not part of the accredited program. It is especially important for programs to clearly communicate to current and prospective residents the accreditation status of the program.

C-4 P. Focus areas within Major Area of Training Postdoctoral Residency Programs (formerly C-11(d); Commission on Accreditation, April 2011; revised November 2015)

The CoA recognizes that postdoctoral residency programs accredited in a major area of training may offer one or more focus areas within a program. However, such programs may lack key elements required of a single accredited major area of training program, and instead may resemble multiple specialty programs. For example, a traditional practice program in Clinical Psychology with focus areas in neuropsychology and health psychology may lack key features that distinguish it from two separate specialty practice programs in Clinical Neuropsychology and Clinical Health Psychology. This Implementing Regulation is intended to clarify the key features that differentiate a single major area of training program with focus areas, from multiple practice programs, each of which should be individually accredited. Key elements that define a program regardless of focus area(s):

• A set of competencies [Standard II.B] emanating from a program aims [Standard II.A]; * • Some shared educational and training experiences across all residents in the program (e.g.,

didactics, seminars) [Standard II.C]; • Shared minimal levels of achievement across all residents in the program [Standard II.D]; • A designated director responsible for overall program oversight and management [Standard

IV.A.1]; • Its rationale for the duration of training within the program is consistent with its unitary training

model (if the length is greater than one year) [Standard I.A.2]; • Demonstration through proximal and distal outcome data that the program meets the program

aim(s) and competencies [Standard II.D]. A major area of training program (e.g., Clinical Psychology) that offers concentrated training (e.g., focus area) must demonstrate during the program review process that it is indeed a single program, is sufficiently broad, accurately reflects the major area of training [IR C-5 P] and provides public information consistent with the above [Standard V and IR C-18 P]. This applies both to programs that offer concentrated training in an area where specialty accreditation by the CoA is recognized (e.g., Clinical Neuropsychology or Clinical Health Psychology) as well as unrecognized areas. Relevant IRs:

• C-1 P. Statement on Postdoctoral Residency Accreditation • C-2 P. Accreditation Process for Postdoctoral Residencies • C-3 P. Postdoctoral Residency Program Transitions • C-5 P. Postdoctoral Residency Specialty Practice Areas • C-18 P. Program Names, Labels, and Other Public Descriptors

* CoA acknowledges that a program may choose (but is not required) to have some competencies that are specific to tracks, rotations, or areas of emphasis within the program.

C-5 P. Postdoctoral Residency Specialty Practice Areas (formerly C-11(b); Commission on Accreditation, July 2001; revised February 2005; April 2010; July

2013; November 2015) The Standards of Accreditation for Health Service Psychology (SoA) include provisions for accreditation of postdoctoral residency training programs providing education and training in preparation for entering professional practice at an advanced level of competency in one of the substantive major areas of training (clinical, counseling school psychology, or other developed practice area) or in a specialty practice area. In defining the meaning of “specialty practice areas” for the purposes of the accreditation of postdoctoral residency training programs only, the Commission on Accreditation employs the criteria that follow. The substantive specialty practice area is one that has been endorsed as follows:

a. Specialty practice areas in health service psychology. If accreditation is sought in a recognized specialty practice area, the specialty practice area must meet at least two of the following requirements:

i. The specialty is recognized by the Commission on the Recognition of Specialties and Proficiencies in Professional Psychology (CRSPPP) of the American Psychological Association or by the American Board of Professional Psychology (ABPP).

ii. The specialty is recognized by and holds membership on the Council of Specialties (CoS).

iii. The specialty has provided the Commission on Accreditation with specialty-specific postdoctoral educational and training guidelines endorsed by the Council of Specialties.

The following areas currently meet the provisions above:

• Behavioral and Cognitive Psychology • Clinical Child Psychology • Clinical Health Psychology • Clinical Neuropsychology • Family Psychology • Forensic Psychology • Geropsychology • Rehabilitation Psychology

The above list will be updated as new areas meeting the criteria are added. NOTE: The CoA reviews and makes accreditation decisions about programs that have identified specialty practice areas based on the program’s compliance with the SoA.

C-6 P. Diversity Recruitment and Retention (formerly C-22; Commission on Accreditation, November 2009; revised March 2013; November 2015,

October 2018) The Standards of Accreditation (SoA) state that that five principles, one of which is a commitment to cultural and individual diversity, “guide accreditation decisions, such that programs whose policies and procedures violate them would not be accredited.” Furthermore, the Commission “is committed to a broad definition of cultural and individual differences and diversity that includes, but is not limited to, age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and socioeconomic status (SoA, p. 3).” Diversity is essential to science and quality education in Health Service Psychology. The goals of diversity recruitment and retention include, but are not limited to, creating and maintaining inclusive environments and improving access to quality education and training. An inclusive environment is one in which the program creates an atmosphere that is welcoming, respectful and affirming of residents’ and faculty/staff members’ multiple identities. In accordance with Standards I.B.3, III.A.3 and IV.B.2 of the Standards of Accreditation (SoA) for postdoctoral programs, an accredited postdoctoral program is responsible for making systematic, coherent, and long-term efforts to 1) attract (i.e., recruit) as well as, 2) retain diverse residents and faculty/staff into the program. In addition, the program is responsible for assessing the effectiveness of both its recruitment and retention efforts and identifying areas of improvement. For both recruitment and retention, the program must provide program-level efforts and activities, in addition to any institutional, departmental, or other unit activities that are used. Programs are expected to seek and utilize generally accepted best practices in the field regarding recruitment and retention of diverse individuals. In planning for the recruitment and retention of diverse individuals, accredited programs should consider the following:

- A program may include institutional-level initiatives addressing diversity, but these, in and of themselves, are not considered sufficient.

- The lack of faculty/staff openings or having existing faculty/staff with strong representation of diversity, does not exempt the program from the responsibility of having a systematic, multi-year plan in place.

- Similarly, having an existing resident group with strong representation of diversity does not exempt the program from the responsibility of having a systematic, multi-year plan in place.

- The demographic information about faculty/staff and residents in the tables of the self-study and annual report is not sufficient to demonstrate a program’s compliance with Standards I.B.3., III.A.3 and IV.B.2.

Recruitment The program is expected to document that it has developed and implemented a systematic plan to recruit both residents and faculty/staff from diverse backgrounds. Residents An accredited postdoctoral program should document and report in its self-study:

• that it has developed a systematic, multi-year, and multiple effort plan, implemented and sustained over time, to attract residents from a range of diverse identities;

• the concrete and specific program-level activities, approaches, and initiatives it implements to increase diversity among its residents;

• the areas of diversity recruitment in which it has had success, as well as the areas of diversity recruitment it is working to improve, recognizing the broad definition of diversity, and;

• how it examines the effectiveness of its efforts to attract diverse residents, and the steps it has taken to revise/enhance its strategies.

Faculty/Staff An accredited postdoctoral program should demonstrate and report in its self-study:

• that it has developed a systematic, multi-year, and multiple effort plan, implemented and sustained over time, to attract faculty/staff from a range of diverse identities (i.e., when there are faculty/staff openings);

• the concrete and specific program-level activities, approaches, and initiatives it implements to increase diversity among its faculty/staff;

• the areas of diversity recruitment in which it has had success, as well as the areas of diversity recruitment it is working to improve, recognizing the broad definition of diversity, and;

• how it examines the effectiveness of its efforts to attract diverse faculty/staff, and the steps it has taken to revise/enhance its strategies.

Retention The program is expected to document that it has developed and implemented a systematic plan to retain residents and faculty/staff from diverse backgrounds. Residents An accredited postdoctoral program is expected to describe in its self-study:

• the specific activities, approaches, and initiatives it implements to maintain diversity among its residents and ensure a supportive and inclusive environment for all residents;

• concrete program-level actions to retain diverse residents; • how these efforts are broadly integrated across key aspects of the program; • how the program examines the effectiveness of its efforts to retain diverse residents, and the steps

it has taken as needed to revise and/or enhance its retention strategies. Faculty/Staff An accredited postdoctoral program is expected to describe in its self-study:

• the specific activities, approaches, and initiatives it implements to maintain diversity among its faculty/staff and ensure a supportive and inclusive work environment for its diverse faculty/staff members.

• how the program examines the effectiveness of its efforts to maintain diversity among its faculty/staff, and the steps it has taken to revise/enhance its strategies as needed.

C-7 P. Record of Resident Complaints in CoA Periodic Review (formerly C-3; Commission on Accreditation, October 1998; revised November 2015)

Standard I.D.2 of the SoA addresses the need for accredited programs to recognize the rights of residents to be treated with courtesy and respect, to inform them of the principles outlining ethical conduct of psychologists, and to ensure that they are aware of avenues of recourse should problems with regard to these principles arise. In accordance with Standard I.D.2 of the Standards of Accreditation for postdoctoral residency programs, a program is responsible for keeping information and records of all formal complaints and grievances, of which it is aware, filed against the program and/or against individuals associated with the program since its last accreditation site visit. These records will be reviewed by the Commission on Accreditation (CoA) as part of its periodic review of programs. The CoA expects a program to keep all materials pertaining to each of the complaints/grievances filed against it during the aforementioned time period. The site visitors shall review the full record of program materials on any or all of the filed complaints/grievances.

C-8 P. Statement on Number of Postdoctoral Residents (formerly C-13; Commission on Accreditation, October 2000; revised November 2015)

Standard I.D.2 of the Standards of Accreditation for postdoctoral residency programs states that, “The program encourages peer interaction, and residents are provided with opportunities for appropriate peer interaction, support, and learning.” The Commission on Accreditation recognizes that the nature of the postdoctoral residency leads to a different socialization process and definition of “peers” than would be the case in an internship or doctoral program. For this reason, the Commission believes that some postdoctoral residency programs may be able to achieve meaningful interaction, support, and socialization without having more than one resident. Regardless of the number of residents in any given program, however, it is incumbent upon each program to demonstrate how it encourages peer interaction and provides its residents with opportunities for appropriate interaction, support, and learning.

C-9 P. Profession-Wide Competencies (Commission on Accreditation, October 2015; revised July 2017)

The Commission on Accreditation (CoA) requires that all trainees who complete accredited training programs, regardless of substantive practice area, degree type, or level of training, develop certain competencies as part of their preparation for practice in health service psychology (HSP). The CoA evaluates a program’s adherence to this standard in the context of the SoA sections that articulate profession-wide competencies at the doctoral (Section II.B.1.b), internship (Section II.A.2), and post-doctoral (Section II.B.1) levels. This Implementing Regulation refers specifically to aspects of a program’s curriculum or training relevant to acquisition and demonstration of the profession-wide competencies required in all accredited programs. The CoA acknowledges that programs may use a variety of methods to ensure trainee competence, consistent with their program aim(s), degree type, and level of training. However, all programs must adhere to the following training requirements:

• Consistency with the professional value of individual and cultural diversity (SoA Introduction, Section II.B). Although Individual and Cultural Diversity is a profession-wide competency, the CoA expects that appropriate training and attention to diversity will also be incorporated into each of the other profession-wide competencies, consistent with SoA Introduction, Section II.B.2.a.

• Consistency with the existing and evolving body of general knowledge and methods in the science and practice of psychology (SoA Introduction, Section II.B.2.d). The CoA expects that all profession-wide competencies will be grounded, to the greatest extent possible, in the existing empirical literature and in a scientific orientation toward psychological knowledge and methods.

• Level-appropriate training. The CoA expects that training in profession-wide competencies at the

doctoral and internship levels will provide broad and general preparation for entry level independent practice and licensure (SoA Introduction, Section II.B.2.b) Training at the postdoctoral level will provide advanced preparation for practice (SoA Introduction, Section II.B.2.c). For postdoctoral programs that are accredited in a specialty area rather than a developed practice area of HSP, the program will provide advanced preparation for practice within the specialty.

• Level-appropriate expectations. The CoA expects that programs will require trainee demonstrations of profession-wide competencies that differ according to the level of training provided (i.e., doctoral, internship, post-doctoral). In general, trainees are expected to demonstrate each profession-wide competency with increasing levels of independence and complexity as they progress across levels of training.

• Evaluation of trainee competence. The CoA expects that evaluation of trainees’ competence in each required profession-wide competency area will be an integral part of the curriculum, with evaluation methods and minimum levels of performance that are consistent with the SoA (e.g., for clinical competencies, evaluations are based at least in part on direct observation; evaluations are consistent with best practices in student competency evaluation).

I. Research This competency is required at the doctoral and internship levels. Demonstration of the integration of science and practice, but not the demonstration of research competency per se, is required at the post-doctoral level

The CoA recognizes science as the foundation of HSP. Individuals who successfully complete programs accredited in HSP must demonstrate knowledge, skills, and competence sufficient to produce new knowledge, to critically evaluate and use existing knowledge to solve problems, and to disseminate research. This area of competence requires substantial knowledge of scientific methods, procedures, and practices. Trainees are expected to: Doctoral students:

• Demonstrate the substantially independent ability to formulate research or other scholarly activities (e.g., critical literature reviews, dissertation, efficacy studies, clinical case studies, theoretical papers, program evaluation projects, program development projects) that are of sufficient quality and rigor to have the potential to contribute to the scientific, psychological, or professional knowledge base.

• Conduct research or other scholarly activities. • Critically evaluate and disseminate research or other scholarly activity via professional publication

and presentation at the local (including the host institution), regional, or national level. Interns:

• Demonstrates the substantially independent ability to critically evaluate and disseminate research or other scholarly activities (e.g., case conference, presentation, publications) at the local (including the host institution), regional, or national level.

II. Ethical and legal standards This competency is required at the doctoral, internship, and post-doctoral levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees at all levels are expected to demonstrate competency in each of the following areas:

• Be knowledgeable of and act in accordance with each of the following:

o the current version of the APA Ethical Principles of Psychologists and Code of Conduct;

o relevant laws, regulations, rules, and policies governing health service psychology at the organizational, local, state, regional, and federal levels; and

o relevant professional standards and guidelines.

• Recognize ethical dilemmas as they arise, and apply ethical decision-making processes in order to

resolve the dilemmas.

• Conduct self in an ethical manner in all professional activities. III. Individual and cultural diversity This competency is required at the doctoral, internship, and post-doctoral levels. Effectiveness in health service psychology requires that trainees develop the ability to conduct all professional activities with sensitivity to human diversity, including the ability to deliver high quality services to an increasingly diverse population. Therefore, trainees must demonstrate knowledge, awareness,

sensitivity, and skills when working with diverse individuals and communities who embody a variety of cultural and personal background and characteristics. The Commission on Accreditation defines cultural and individual differences and diversity as including, but not limited to, age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and socioeconomic status. The CoA recognizes that development of competence in working with individuals of every variation of cultural or individual difference is not reasonable or feasible. Trainees at all levels are expected to demonstrate:

• an understanding of how their own personal/cultural history, attitudes, and biases may affect how they understand and interact with people different from themselves;

• knowledge of the current theoretical and empirical knowledge base as it relates to addressing

diversity in all professional activities including research, training, supervision/consultation, and service;

• the ability to integrate awareness and knowledge of individual and cultural differences in the

conduct of professional roles (e.g., research, services, and other professional activities). This includes the ability apply a framework for working effectively with areas of individual and cultural diversity not previously encountered over the course of their careers. Also included is the ability to work effectively with individuals whose group membership, demographic characteristics, or worldviews create conflict with their own.

Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence as they progress across levels of training. Trainees are expected to: Doctoral students:

• Demonstrate the requisite knowledge base, ability to articulate an approach to working effectively with diverse individuals and groups, and apply this approach effectively in their professional work.

Interns:

• Demonstrate the ability to independently apply their knowledge and approach in working effectively with the range of diverse individuals and groups encountered during internship.

Post-doctoral residents:

• Demonstrate the ability to independently apply their knowledge and demonstrate effectiveness

in working with the range of diverse individuals and groups encountered during residency, tailored to the learning needs and opportunities consistent with the program’s aim(s).

IV. Professional values and attitudes This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Doctoral students and Interns are expected to:

• behave in ways that reflect the values and attitudes of psychology, including integrity, deportment, professional identity, accountability, lifelong learning, and concern for the welfare of others.

• engage in self-reflection regarding one’s personal and professional functioning; engage in activities

to maintain and improve performance, well-being, and professional effectiveness.

• actively seek and demonstrate openness and responsiveness to feedback and supervision.

• respond professionally in increasingly complex situations with a greater degree of independence as they progress across levels of training.

V. Communication and interpersonal skills This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. The CoA views communication and interpersonal skills as foundational to education, training, and practice in health service psychology. These skills are essential for any service delivery/activity/interaction, and are evident across the program’s expected competencies. Doctoral students and interns are expected to:

• develop and maintain effective relationships with a wide range of individuals, including colleagues, communities, organizations, supervisors, supervisees, and those receiving professional services.

• produce and comprehend oral, nonverbal, and written communications that are informative and well-integrated; demonstrate a thorough grasp of professional language and concepts.

• demonstrate effective interpersonal skills and the ability to manage difficult communication well. VI. Assessment This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees demonstrate competence in conducting evidence-based assessment consistent with the scope of Health Service Psychology. Doctoral students and Interns are expected to:

• Demonstrate current knowledge of diagnostic classification systems, functional and dysfunctional behaviors, including consideration of client strengths and psychopathology.

• Demonstrate understanding of human behavior within its context (e.g., family, social, societal and cultural).

• Demonstrate the ability to apply the knowledge of functional and dysfunctional behaviors including

context to the assessment and/or diagnostic process.

• Select and apply assessment methods that draw from the best available empirical literature and that reflect the science of measurement and psychometrics; collect relevant data using multiple sources and methods appropriate to the identified goals and questions of the assessment as well as relevant diversity characteristics of the service recipient.

• Interpret assessment results, following current research and professional standards and guidelines,

to inform case conceptualization, classification, and recommendations, while guarding against decision-making biases, distinguishing the aspects of assessment that are subjective from those that are objective.

• Communicate orally and in written documents the findings and implications of the assessment in

an accurate and effective manner sensitive to a range of audiences. VII. Intervention This competency is required at the doctoral and internship levels. Trainees are expected to respond professionally in increasingly complex situations with a greater degree of independence across levels of training. Trainees demonstrate competence in evidence-based interventions consistent with the scope of Health Service Psychology. Intervention is being defined broadly to include but not be limited to psychotherapy. Interventions may be derived from a variety of theoretical orientations or approaches. The level of intervention includes those directed at an individual, a family, a group, a community, a population or other systems. Doctoral students and Interns are expected to demonstrate the ability to:

• establish and maintain effective relationships with the recipients of psychological services. • develop evidence-based intervention plans specific to the service delivery goals. • implement interventions informed by the current scientific literature, assessment findings, diversity

characteristics, and contextual variables. • demonstrate the ability to apply the relevant research literature to clinical decision making. • modify and adapt evidence-based approaches effectively when a clear evidence-base is lacking, • evaluate intervention effectiveness, and adapt intervention goals and methods consistent with

ongoing evaluation.

VIII. Supervision This competency is required at the doctoral and internship level. The CoA views supervision as grounded in science and integral to the activities of health service psychology. Supervision involves the mentoring and monitoring of trainees and others in the development of competence and skill in professional practice and the effective evaluation of those skills. Supervisors act as role models and maintain responsibility for the activities they oversee. Trainees are expected to: Doctoral students:

• Demonstrate knowledge of supervision models and practices.

Interns:

• Apply this knowledge in direct or simulated practice with psychology trainees, or other health professionals. Examples of direct or simulated practice examples of supervision include, but are not limited to, role-played supervision with others, and peer supervision with other trainees.

IX. Consultation and interprofessional/interdisciplinary skills This competency is required at the doctoral and internship level. The CoA views consultation and interprofessional/interdisciplinary interaction as integral to the activities of health service psychology. Consultation and interprofessional/interdisciplinary skills are reflected in the intentional collaboration of professionals in health service psychology with other individuals or groups to address a problem, seek or share knowledge, or promote effectiveness in professional activities. Trainees are expected to: Doctoral students and Interns:

• Demonstrate knowledge and respect for the roles and perspectives of other professions. Doctoral students:

• Demonstrates knowledge of consultation models and practices. Interns:

• Apply this knowledge in direct or simulated consultation with individuals and their families, other health care professionals, interprofessional groups, or systems related to health and behavior.

Direct or simulated practice examples of consultation and interprofessional/interdisciplinary skills include but are not limited to:

• role-played consultation with others. • peer consultation, provision of consultation to other trainees.

C-10 P. Diversity Education and Training (formerly C-23; Commission on Accreditation, November 2009; revised March 2013; November 2015)

In accordance with Standard II.B.1.b of the Standards of Accreditation (SoA) for postdoctoral programs, a program has and implements a thoughtful and coherent plan to provide residents with relevant knowledge and experiences about the role of cultural and individual diversity in psychological phenomena and professional practice. Although the Commission asks for demographic information about faculty/staff and residents in the tables of the self-study and annual report, the information requested is limited to the data collected in federal reports, which is not sufficient in demonstrating a program’s compliance with Standard II.B.1.b. Consistent with Standard I.B.3, as described in the postdoctoral program SoA, cultural and individual diversity includes but is not limited to age, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, and social economic status.

An accredited program is expected to articulate and implement a specific plan for integrating diversity into its didactic and experiential training. This training should be based on the multicultural conceptual and theoretical frameworks of worldview, identity, and acculturation, rooted in the diverse social, cultural, and political contexts of society, and integrated into the science and practice of psychology. Programs are expected to train residents to respect diversity and be competent in addressing diversity in all professional activities including research, training, and service.

The program should demonstrate that it examines the effectiveness of its education and training efforts in this area. Steps to revise/enhance its strategies as needed should be documented.

C-11 P. Residency Didactics (formerly C-29; Commission on Accreditation, July 2010; revised November 2015)

The purpose of this IR is to clarify the type of information required from postdoctoral residency programs about their didactic activities. Didactic activities are defined as planned sessions of instruction that are included within the postdoctoral residency training curriculum. When didactic activities are used to meet or partially meet any of the program’s aim(s) or required curriculum areas, it is the program’s responsibility to include adequate information on those didactics within the self-study to convey their nature and content. A title alone would not be sufficient; descriptions may include an abstract/description of the content, learning objectives, or any other additional material necessary (e.g., bibliography, readings) to demonstrate the material covered.

C-12 P. Positive Identification of Students Consistent with Higher Education Opportunity Act (formerly C-25; Commission on Accreditation, November 2009; revised November 2015)

Consistent with the 2008 Higher Education Opportunity Act, all accrediting agencies recognized by the U.S. Department of Education are required by federal law to engage in a review of the methods used by its accredited programs for positive identification of residents who are enrolled in any form of distance/online/electronically mediated education. As such, the APA Commission on Accreditation (CoA) requires that if a student in an APA-accredited program is engaged in any form of distance, online, or electronically mediated education for any part of their educational sequence (doctoral, internship, residency), the program must provide CoA with information in its self-study regarding the methods it and its host institution use to identify that resident. In particular, the program must provide CoA with information about how it ensures that a student who registers or receives credit for a course that uses any form of distance, online, or electronically mediated education is the same student who participates in and completes that course. Whatever methodology is used must clearly protect resident privacy. Finally, residents must be provided with information at the time of registration or enrollment of any projected additional student charges associated with verification of resident identity.

C-13 P. Jurisdiction of Licensure for Supervisors in Postdoctoral Residencies (formerly C-15(a); Commission on Accreditation, January 2002; revised November 2003; November

2015) Standard II.C.4.c of the Standards of Accreditation (SoA) for postdoctoral residency programs states that: “A postdoctoral resident must have an appropriately trained and licensed doctoral-level psychologist serving as primary supervisor in order to ensure continuity of the training plan.” Standard IV.A.1.a states that: “The program has a designated director who is a psychologist, appropriately trained and credentialed (i.e., licensed, registered, or certified) to practice psychology in the jurisdiction in which the program is located, who is primarily responsible for directing the training program, and who has administrative authority commensurate with those responsibilities.” Standard IV.B.1.d states that formally designated supervising psychologists: “Are appropriately trained and credentialed (i.e. licensed, registered, or certified) to practice psychology in the jurisdiction in which the program is located.” In interpreting this provision for postdoctoral residency programs, the CoA looks to determine appropriate licensure of the supervisor on the basis of jurisdiction governing the practice or service that is being supervised. For example:

1) When the services on which supervision is being provided are conducted in a context where a state or provincial credential is required for practice, then the appropriate credential would be that provided by the state or province.

2) When services on which supervision is being provided are being conducted in a federal jurisdiction

(e.g., the VA, Bureau of Prisons), then the credentialing rules pertaining to practice in a federal setting would apply.

3) For those residents providing services in multiple jurisdictions (such as a Bureau of Prisons

internship that has an external community rotation), the jurisdiction governing the resident’s service that is being supervised would determine the appropriate supervisor credential.

C-14 P. Required Supervision in Postdoctoral Training Programs (formerly C-15(b); Commission on Accreditation, January 2007; November 2009; November 2015)

Standard II.C.4 of the Standards of Accreditation (SoA) for postdoctoral residency programs states that:

a. At least two hours per week of individual supervision focused on resident professional activities must be conducted by an appropriately trained and licensed doctoral-level psychologist.

b. Supervisors must maintain an ongoing supervisory relationship with the resident and have primary professional clinical responsibility for the cases for which they provide supervision.

c. A postdoctoral resident must have an appropriately trained and licensed doctoral-level psychologist serving as primary supervisor in order to ensure continuity of the training plan.

d. The primary supervisor must maintain overall responsibility for all supervision, including oversight and integration of supervision provided by other health professionals.

The purpose of this Implementing Regulation is to clarify the supervision required for postdoctoral residents. Supervision is characterized as an interactive educational experience between the resident and the supervisor. This relationship: a) is evaluative and hierarchical, b) extends over time, and c) has the simultaneous purposes of enhancing the professional functioning of the more junior person(s); monitoring the quality of professional services offered to the clients that she, he, or they see; and serving as a gatekeeper for those who are to enter the particular profession (Bernard and Goodyear, 2009). Two weekly hours of individual supervision must be conducted by a doctoral-level licensed psychologist who is involved in an ongoing supervisory relationship with the resident and has primary professional clinical responsibility for the cases on which he/she provides supervision. A postdoctoral resident must have a minimum of two doctoral level licensed psychologist supervisors, at least one of whom serves as the resident’s primary supervisor. Supervisory hours beyond the two hours of individual supervision must be supervised by professionals who are appropriately credentialed for their role/contribution to the program. The primary doctoral-level licensed psychologist supervisor maintains overall responsibility for all supervision, including oversight and integration of supervision provided by other mental health professionals with psychological research and practice.

C-15 P. Telesupervision (formerly C-28; Commission on Accreditation, July 2010; revised November 2015, July 2017)

The CoA recognizes that accredited programs may utilize telesupervision in their program curriculum. At the same time, the CoA recognizes there are unique benefits to in-person supervision. Benefits to in-person supervision include, but are not limited to, creating opportunities for especially flexible professional socialization and assessment of trainee competence as well as for enhancing recognition and processing of subtle, nonverbal, and emotional or affective cues and interactions in supervision, all of which are essential aspects of professional development, ensuring quality, and protecting the public. Therefore, the CoA recognizes that there must be guidelines and limits on the use of telesupervision in accredited programs. The following applies only to the MINIMUM number of required hours of supervision. Supervision beyond the minimum number of required hours may utilize methods or modalities that are deemed appropriate by the accredited program. Nothing in this Implementing Regulation contravenes other requirements in the Standards of Accreditation for Health Service Psychology (SoA). It only clarifies the utilization of telesupervision at the postdoctoral level. Definitions:

Telesupervision is supervision of psychological services through a synchronous audio and video format where the supervisor is not in the same physical facility as the trainee. (See the definition of supervision as noted in the Glossary) In-person supervision is supervision of psychological services where the supervisor is physically in the same room as the trainee. (See the definition of supervision as noted in the Glossary)

Guidelines and Limits: Telesupervision may not account for more than one hour (50%) of the minimum required (as defined in the SoA) two weekly hours of face-to-face supervision. Programs utilizing ANY amount of telesupervision need to have a formal policy addressing their utilization of this supervision modality, including but not limited to:

• An explicit rationale for using telesupervision; • How telesupervision is consistent with their overall aims and training outcomes; • How and when telesupervision is utilized in clinical training; • How it is determined which trainees can participate in telesupervision; • How the program ensures that relationships between supervisors and trainees are established at the

onset of the supervisory experience; • How an off-site supervisor maintains full professional responsibility for clinical cases; • How non-scheduled consultation and crisis coverage are managed; • How privacy and confidentiality of the client and trainees are assured; • The technology and quality requirements • Contingency plan for technology failures or unanticipated lack of availability; and • Education in the use of this technology that is required by either trainee or supervisor.

C-16 P. Outcome Data for Postdoctoral Residency Programs (formerly C-30; Commission on Accreditation, July 2011; revised April 2016)

This Implementing Regulation clarifies the type of data the CoA needs to make an accreditation decision on postdoctoral residency programs. The CoA requires all accredited programs to provide outcome data on the extent to which the program is effective in achieving its aim(s), required profession-wide competences; program specific competencies (if any); and specialty area competencies (as appropriate). As stated in the Standards of Accreditation (SoA) for postdoctoral residency programs (II.D.1):

a) An evaluation is made of the resident’s progress toward satisfactory attainment of the program’s expected competencies, as reflected in the completion of the program’s stated minimum levels of achievement and other program requirements. b) Data on residents’ competencies must include competency-based assessments of residents as they progress through, and at completion of, the program (proximal data), as well as information regarding their attainment of competencies after they complete the program (distal data).

i. Proximal data will, at the least include evaluations of residency by knowledgeable others (i.e., supervisors or trainers). The evaluation process and assessment forms must parallel the program’s expected competencies. These evaluations include the feedback provided to residents as required in Standard I.C.1(d). ii. At each evaluation interval, the evaluation must be based in part on direct observation of the competencies evaluated. iii. Distal data reflect the program’s effectiveness in achieving its aims, as reflected by resident attainment of program-defined competencies. iv. Distal data typically include information obtained from alumni surveys assessing former residents’ perceptions of the degree to which the program achieved its aims by preparing them in the competencies identified as important by the program. The data may also include graduates’ professional activities and accomplishments (e.g., licensure, employment, memberships, and affiliations).

Also, the United States Department of Education (USDE) requires recognized accrediting bodies (such as the CoA) to collect and monitor data-driven outcomes, especially as they relate to student achievement. In making an accreditation decision on a program, CoA must demonstrate that it reviews resident achievement through review of the program’s outcome data.

All accredited programs are required to demonstrate an educational/training curriculum that is consistent with program aim(s) and is designed to foster resident development of required profession-wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate). Expected minimal levels of achievements must be specified for all profession-wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate). It is each program’s responsibility to collect, present, and utilize aggregated proximal and distal outcome data that are directly tied to profession wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate).

Definitions: Proximal data are defined as outcomes on residents as they progress through and complete the program, which are tied to the required profession-wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate).

• Proximal data at a minimum must include the evaluations of residents by others responsible for their training (e.g., by supervisors/trainers), including mid-point and end-of-year evaluations. This is most easily accomplished when the evaluation mechanisms parallel the profession wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate). It is expected that these data would at least include the semi-annual feedback provided to residents as required by Standard I.C.2 of the SoA.

• While resident self-ratings, ratings of satisfaction with training, or ratings by others (e.g., peers) may be a part of proximal assessment, they are not considered sufficient outcome data in this context since they do not address the program’s success in promoting attainment of profession wide competencies ; program specific competencies (if any); and specialty area competencies (as appropriate).

Distal data are defined as outcomes on residents after they have completed the program, which are tied to the profession-wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate).

• Distal data typically include information obtained from alumni surveys addressing former residents’ perceived assessments of the degree to which the program promoted mastery of profession wide competencies, program specific competencies (if any), and specialty area competencies (as appropriate).

• Distal data reflecting completion of professional activities and accomplishments (e.g., licensure, employment, memberships, and affiliations), such as those found in the self-study tables, are important examples of distal outcomes but alone are not sufficient because they do not fully reflect achievement of all expected competencies.

• Although alumni surveys assessing former residents’ overall satisfaction with the training program (including the degree to which the education and training is relevant) may be an important component of a program’s ongoing self-study process, they are not considered sufficient outcome data in this context since they do not address the program’s success in promoting expected competencies.

• Although CoA does not specify the interval at which distal data should be collected, the interval should be appropriate to allow the program to evaluate its success in promoting expected competencies to determine if changes need to be made, consistent with Standard II.

Level of Specificity: Profession-Wide Competencies According to the Standards of Accreditation (cite appropriate section), accredited programs are required to provide a training/educational curriculum that fosters the development of three advanced competencies, two of which are profession-wide competencies. Accredited programs are required to operationalize competencies in terms of multiple elements. At a minimum, those elements must reflect the content description all advanced competencies (including the two profession-wide competencies as defined in IR C-8P, including the bulleted content), and must be consistent with the program aim(s). It is incumbent upon the program to demonstrate that there is a sufficient number of elements articulated for each PWC so as to demonstrate adequate trainee attainment of competence. Programs must assess resident performance at the level of the elements, give feedback to residents at the level of elements, but then report to CoA at the level of the superordinate competency.

Aggregated data are compilations of proximal data and compilations of distal data across residents, which may be broken down by cohort or years. Aggregate data demonstrate the effectiveness of the program as a whole, rather than the accomplishment of an individual resident over time.

• To the extent possible, data should be presented in table form using basic descriptive statistics (e.g., sample sizes, means, percentages). The program should choose statistics that allow for evaluation of whether all trainees are acquiring competencies in relation to its defined minimal levels of achievement for required profession-wide competencies; program specific competencies (if any); and specialty area competencies (as appropriate).

• If data are aggregated over a number of years (i.e., not broken down by cohort or years), the program needs to demonstrate how aggregating the data in this way facilitates the program’s self-improvement.

Program Specific Competencies Accredited programs may choose to include program specific competencies as part of their educational curriculum. These should be consistent with the program’s aim(s) and professional standards and practices of Health Service Psychology. Further, programs must demonstrate education/training to facilitate development of these competencies, appropriate mechanisms to assess resident performance on these competencies (including expected minimal levels of achievement for successful completion of the program), and its success in ensuring that residents reach expected levels of performance. Similar to the expectations for profession wide competencies, programs that choose to have program-specific competencies are expected have multiple elements for each of those competencies, assess resident performance at the level of the elements, give feedback to residents at the level of elements, but then report to CoA at the level of the superordinate competency. Aggregated data must be presented in a manner that demonstrates the success of the program as a whole while allowing for an assessment of how well residents are performing in relation to defined minimal levels of achievement. Specialty Competencies Programs accredited in a recognized specialty practice area must include competencies specific to the specialty area as part of their educational curriculum. These must be consistent with the program’s aim(s) and with the education and training guidelines of the recognized speciality. Further, programs must demonstrate education/training to facilitate development of these competencies, appropriate mechanisms to assess resident performance on these competencies (including expected minimal levels of achievement for successful completion of the program), and its success in ensuring that residents reach expected levels of performance. Similar to the expectations for profession-wide competencies and program specific competencies, programs that have specialty competencies are expected have multiple elements for each of those competencies, assess resident performance at the level of the elements, give feedback to residents at the level of elements, but then report to CoA at the level of the superordinate competency. Aggregated data must be presented in a manner that demonstrates the success of the program as a whole while allowing for an assessment of how well residents are performing in relation to defined minimal levels of achievement.

C-17 P. Direct Observation (Commission on Accreditation, July 2015; revised February 2017)

This Implementing Regulation is intended to clarify the expectations of CoA with regard to “direct observation” as described in the Standards of Accreditation (SoA) for postdoctoral residency programs as follows: Standard II.D.1.b.ii.

“At each evaluation interval, the evaluation must be based in part on direct observation (either live or electronically) of the competencies evaluated.”

Definitions and Guidelines: Direct observation provides essential information regarding trainees’ development of competencies, as well as the quality of the services provided, that cannot be obtained through other methods. This allows supervisors to provide a more accurate assessment and evaluation of observable aspects of trainees’ competency development regarding one or more profession-wide and program-specific competencies (if any) associated with that training experience. Direct observation includes in-person observation (e.g., in room or one-way mirror observation of client contact an intake or test feedback session,), live synchronous audio-video streaming, or audio or video recording. A training site that does not permit live observation, audio or video recording by policy is not a sufficiently unique circumstance to circumvent this requirement. To these ends, all accredited programs must verify on the evaluation form that direct observation is conducted by the immediate supervisor responsible for the activity or experience being evaluated. As indicated in the SoA (Standard I.C.2), at a minimum a residency must provide written feedback on a semiannual basis. Each of these written evaluations must be based in part on at least one instance of direct observation. In the case that a resident completes multiple rotations within a training year, each is considered a unique and separate training experience and requires direct observation as part of the resident evaluation process for that rotation.

C-18 P. Program Names, Labels, and Other Public Descriptors (formerly C-6(a); Commission on Accreditation, January 2002; revised January 2003; November 2015)

What the postdoctoral residency program is called: Because accreditation is available to both doctoral internships and postdoctoral residencies, programs must portray themselves in a manner that does not misrepresent their level of training. Thus, in general, doctoral internship programs should not describe themselves as “residencies,” and postdoctoral residency programs should not describe themselves as “internships.” It is recognized, however, that agencies and institutions providing training at either or both of these levels may have local or state regulations about, or restrictions on, the terms used to portray programs that prepare individuals for practice. In the event that it is not possible to use the term “internship” for doctoral internship training programs, and “residency” for postdoctoral residency training programs, the program in question should include in all public documents (e.g., brochures, materials, web sites, certificates of completion) a statement about the program’s accredited status.

Preferred: “Postdoctoral residency in Clinical Psychology” “Postdoctoral residency in Health Service Psychology”

How the program describes itself: It is recognized that programs have many possible reasons why they choose the self-descriptors or labels that they do. Some are bound by state law, others by institutional regulation, and others simply seek to assign a label to their program to describe their focus to the public. Given that these self-descriptors do not necessarily coincide with recognized areas of accreditation, any program whose label does not reflect the specific area in which it received accreditation must portray its accredited status in a manner consistent with the SoA. Postdoctoral programs accredited in substantive or specialty areas may offer training in areas of emphasis. Areas of emphasis may be described in all public materials except the certificate of completion. Programs will state clearly that accreditation is specific to the substantive or specialty area only.

Preferred: • “Postdoctoral residency in Clinical Psychology” • “Postdoctoral residency in health service psychology”

Example with accurate accreditation status:

• “Postdoctoral residency with an emphasis in geropsychology, accredited as a postdoctoral residency in clinical psychology”

What trainees are called: For postdoctoral residencies, trainees (per the SoA) have a title commensurate with the title carried in that setting by other professionals in training who have comparable responsibility and comparable education and training, consistent with the laws of the jurisdiction in which the program is located. The title assigned to the trainee should not mislead the public about their level of training.

Certificate of completion of residencies: Certificates of completion for postdoctoral residencies reflect only the major area of training in Health Service Psychology (clinical, counseling, or school) or the recognized specialty practice areas in which the program has been accredited. Areas of emphasis may not be identified on the certificate.

Examples:

“completed a postdoctoral residency in clinical psychology” “completed a postdoctoral residency in clinical health psychology”

C-19 P. Accreditation Status and CoA Contact Information (formerly C-6(b); Commission on Accreditation, November 2010; March 2015; November 2015)

Standard V.A.1.b of the Standards of Accreditation for Health Service Psychology (SoA) for postdoctoral residency programs states that the program must include in its public materials:

The program provides its status with regard to accreditation, including the specific training program covered by that status, and the name, address, and telephone number of the Commission on Accreditation. The program makes available, as appropriate through its sponsor institution, such reports or other materials as pertain to the program’s accreditation status.

Programs that are accredited by agencies recognized by the U.S. Department of Education (e.g., CoA) are required to provide the contact information for the accrediting body when the accreditation status is cited. The intent of this Implementing Regulation is to clarify how this information should be presented in order to ensure consistency across programs as well as provide useful information to the public. Accreditation status: • The only official accredited statuses are: “Accredited on contingency,” “Accredited,” “Accredited on

probation,” and “Accredited inactive,” • Programs may indicate their appropriate status (see above) by referring to “APA” accredited or

accredited “by the Commission on Accreditation of the American Psychological Association,” For example, “APA-accredited,” APA-accredited on contingency,” “accredited by the Commission on Accreditation of the American Psychological Association, “accredited on contingency by the Commission on Accreditation of the American Psychological Association,” etc.

• Programs should not use the term “APA-approved,” since at APA this term is used to denote approved sponsors of continuing education rather than accreditation of academic/training programs.

• If there are multiple programs in the same department, institution, or agency, it should be clearly indicated in public materials which programs are APA-accredited. Multiple accredited programs should refer to their accredited status individually and in accordance with IR C-18 P.

CoA contact information: • In ALL public documents, including the program’s website (if applicable), where the program’s

accreditation status is cited as above, the name and contact information for the CoA must be provided. • Information must include the address and direct telephone number for the APA Office of Program

Consultation and Accreditation. Other information (i.e., website, e-mail address) may also be included. • Programs should clarify that this contact information should be used for questions related to the

program’s accreditation status. In doing so, the program should also ensure that its own contact information is clearly indicated in its materials so that the public knows how to contact the program directly with any other questions.

• Programs are encouraged to use the following format to provide this information: *Questions related to the program’s accredited status should be directed to the Commission on Accreditation:

Office of Program Consultation and Accreditation American Psychological Association 750 1st Street, NE, Washington, DC 20002 Phone: (202) 336-5979 / E-mail: apaaccred@apa.org Web: www.apa.org/ed/accreditation

C-20 P. Notification of Changes to Accredited Programs (formerly C-19; Commission on Accreditation, February 2005; revised October 2006; November 2015)

In accordance with Standard V.B.2 of the Standards of Accreditation (SoA) and Section 8.7 P of the Accreditation Operating Procedures (AOP), all accredited programs (doctoral, internship and postdoctoral residencies) whether under a single administrative entity or in a consortium, must inform the accrediting body in a timely manner of changes that could alter the program's quality. The Commission on Accreditation (CoA) must be informed in advance of major program changes such as changes in degree offered, policies/procedures, administrative structure, faculty resources, supervision resources, area of emphases, or tracks/rotations. This includes new, additional, or eliminated rotation or training sites. For example, consortium programs must inform the CoA of any substantial changes in structure, design or training sites. It also includes requests for changes in accreditation status (e.g., request to transition from inactive back to active status prior to recruitment). Programs must submit to the Office of Program Consultation and Accreditation a detailed written description of the proposed change(s) and the potential impact upon the relevant accreditation standards. The CoA will review the program change(s) and may request additional information or a new self-study. In the case of a substantive change (such as a change in consortium membership), the Commission may also determine that a site visit is needed to assess whether the revised program is consistent with the SoA. Upon completion of this review, the Commission will note the proposed change and include the information in the next scheduled review or inform the program of any needed immediate additional actions. The only exception to the policy of informing the Commission in advance is the occurrence of an unavoidable event beyond the reasonable control and anticipation of the program (e.g., educational/training site unexpectedly withdrawing from a consortium because of financial crisis; resources affected by a natural disaster). In such circumstances, it is incumbent upon the program to immediately inform the CoA in writing of the change and to include in its notification a proposed plan for maintaining program consistency with the SoA. The CoA will then proceed as above. Consultation on program changes is available from the Office of Program Consultation and Accreditation.

C-21 P. “Intent to Apply” (Commission on Accreditation, October 2015; revised October 2016)

All programs can seek public notification of “intent to apply” prior to seeking accreditation. The application for intent to apply includes documentation related to key standards of the SoA. This review is a document review only and does not include a site visit. The review is conducted to verify that the essential elements are adequately described. “Intent to apply” is a declaration and is not an accredited status. This declaration does not constitute a judgment by the CoA regarding the quality of the program. Rather, this serves as public notice of the program’s intent to seek accreditation in the near future.

Overview/Logistics:

A program may seek “intent to apply” declaration at any time, including prior to or after admitting residents. The “intent to apply” declaration indicates that once residents are in place, the program intends to apply for an APA accredited status (either “on contingency” or full accreditation). A program may be listed as “intent to apply” for a maximum of two years. The “intent to apply” declaration is effective as of the date of the Commission’s decision to acknowledge the declaration. If the program exceeds its two year period for full-time 1-year programs and four years for full-time programs that are more than 1-year, it will need to inform its publics and residents that it is no longer designated as an “intent to apply” program. Declaration of “intent to apply” is not a requirement for an application for “accredited, on contingency” or “full accreditation.” For programs seeking the” intent to apply” declaration, the application process is intended to provide the program an opportunity to systematically describe the infrastructure upon which it will be building a program consistent with the Standards of Accreditation (SoA). The Commission on Accreditation will provide feedback to the program in response to their application for “intent to apply.” Although the application includes completion and review of only certain sections of Standards I-V of the SoA, the program clearly intends to seek an accreditation status and be in compliance with all aspects of the SoA.

Process to Apply: To apply for this declaration, programs are asked to submit documentation in accordance with the self-study instructions with the provisions listed below. It is recognized that a program will have elements in place and others in development, both of which will be reviewed by the CoA for prospective alignment with the SoA. The program must address the following:

• Standard I, describing the type of program, institutional and program setting and resources, program policies and procedures, and program climate.

• Standard II, describing its aim(s), required Level 1: advanced competency areas, Level 2: program-specific or area-of-focus competencies (if any), Level 3: specialty competencies (if applicable), its learning elements to develop competencies, its plans to measure proximal and distal outcomes, and its plan to review outcome measures to evaluate and improve the program.

• Standard III, describing its plan for resident selection processes and criteria, including a plan for recruitment of residents who are diverse, and its plan for providing evaluation, feedback, and remediation, if necessary, to trainees.

• Standard IV, describing the designated director of the program who is in place, plans for providing a sufficient number of appropriately qualified supervisors to accomplish the program’s aim(s), and plans for the recruitment and retention of faculty/staff who are from diverse backgrounds.

• Standard V in the areas of general disclosure and communication with prospective and current trainees, and its plan for communicating with the doctoral program (in the case of internship programs). Additionally, the program will provide all materials currently available to its publics. These materials must include: o The program’s timeline to apply for “accredited, on contingency,” or “full accreditation;” o The date that the declaration expires; and o The contact information for the APA CoA.

The program is advised to consider its timeline in light of the requirements for application for accreditation status.

C-22 P. “Accredited, on Contingency” (Commission on Accreditation, October 2015)

"Accredited, on contingency" is an accredited status, which reflects a program’s adherence to the Standards of Accreditation (SoA). Programs seeking “accredited, on contingency” will be reviewed for adherence with all aspects of the SoA. Programs will be granted this status if the postdoctoral residency program sufficiently meets all standards with the exception of outcome data on residents while they are in the program and after program completion. Process to Apply: Programs may apply for "accredited, on contingency" status prior to the arrival of residents on site provided that residents will be on site by the time of the site visit. Programs applying for “accredited, on contingency” status are not required to provide outcome data at the time of application, though they must submit any proximal and distal data collected to date. If data are presented at the time of the site visit, the program must send a copy of these data to the CoA. The postdoctoral program will submit a self-study detailing all SoA components except a complete set of outcome data. However, the self-study must include plans for how the program will evaluate proximal and distal outcomes required to demonstrate minimum levels of achievement in profession-wide competencies, program-specific competencies (if any), and specialty competencies (if applicable). Components of the self-study submission for “accredited, on contingency”: With the exception of the provision of complete outcome data (Standard II), each standard will be addressed with respect to the program’s plans and policies to meet the requirements of the SoA. The program must submit its evaluation plans and forms to evaluate resident outcomes and when possible, provide existing outcome data. Term of accredited, on contingency status: The maximum amount of time a postdoctoral program can be “accredited, on contingency” is two years for a program lasting one year, or four years for a program that is more than one year in duration. The program is advised to consider its timeline in light of requirements to apply for full accreditation status. To apply for full accreditation, programs must provide aggregated proximal and distal data. Residents in the program as well as the public must be kept informed of any change in the program’s timeline that could negatively impact accreditation. Such notice must include current information on in all the program’s public documents (e.g., brochure, APPIC Directory listing). Additionally, the program’s public documents must refer all interested parties to the CoA website, on which is maintained a current listing of accredited program statuses. The program must publish the date of expiration of the status in its public materials. In the event that a program does not provide required proximal and distal data at the end of two years, the program will be considered to have voluntarily withdrawn from accreditation. Consistent with 8.2(b)P of the AOP, “failure to do so (provide outcome data) will lead to the program’s being deemed to have withdrawn from accreditation, following completion of the program by the residents currently on-site at the program.” That is, if the program is deemed to have voluntarily withdrawn from accreditation, residents in the program at the time will have completed an accredited program. Programs that submit proximal and distal data will be eligible for an additional three years as a “fully accredited” program.

C-23 P. Trainees Admissions, Support, and Outcome Data (Commission on Accreditation, April 2016)

Standard V.A. of the Standards of Accreditation for Postdoctoral Programs requires that programs provide potential and current trainees and the public with accurate information on the program and on program expectations. This information is meant to describe the program accurately and completely, using the most up-to-date data about important admissions, support, and outcome variables, and must be presented in a manner that allows applicants to make informed decisions about entering the program.

The CoA requires each accredited program to provide information in its public materials regarding program admissions expectations, program support provided to residents, and initial post-training placement in a standardized way. This information is required to be posted in the program’s public material(s) (e.g., website, brochure), and should be updated annually. This information will be reviewed by the CoA as part of periodic program review.

Presentation of Required Information

To ensure that the required information for each program is available to the public in a consistent fashion, programs are required to update information annually, no later than December 1.

• The information must all be located in a single place and be titled “Postdoctoral Residency Admissions, Support, and Initial Placement Data”;

• If the program has a website, the information must be located no more than one click away from the main/home program landing page (e.g., within the program’s online brochure);

• If the program has more than one source of public materials (e.g., website and brochure), the information must be included in the primary recruiting document used to educate potential applicants about the program For instance, if a brief brochure is provided and then applicants are directed to a website, then the information would be located on the website. Alternatively, if a program has a website “introductory page” and then applicants are instructed to download an extensive brochure, the information can be contained in the brochure;

• Table cells must not be left blank; instead, please enter “NA” if not applicable; • The data must be presented in tables consistent with those listed at the end of this regulation.

Programs may choose to provide other data to supplement the requirements of this regulation, but these tables must be provided. If the program chooses to provide supplemental information, it should be provided below the corresponding required tables.

POST-DOCTORAL RESIDENCY PROGRAM TABLES Date Program Tables are updated:____________

Postdoctoral Program Admissions

Briefly describe in narrative form important information to assist potential applicants in assessing their likely fit with your program. This description must be consistent with the program’s policies on resident selection and practicum and academic preparation requirements:

Describe any other required minimum criteria used to screen applicants:

Financial and Other Benefit Support for Upcoming Training Year* Annual Stipend/Salary for Full-time Residents Annual Stipend/Salary for Half-time Residents Program provides access to medical insurance for resident? Yes No

If access to medical insurance is provided Trainee contribution to cost required? Yes No Coverage of family member(s) available? Yes No

Coverage of legally married partner available? Yes No Coverage of domestic partner available? Yes No Hours of Annual Paid Personal Time Off (PTO and/or Vacation) Hours of Annual Paid Sick Leave In the event of medical conditions and/or family needs that require extended leave, does the program allow reasonable unpaid leave to interns/residents in excess of personal time off and sick leave?

Yes No

Other Benefits (please describe)

* Note. Programs are not required by the Commission on Accreditation to provide all benefits listed in this table.

Initial Post-Residency Positions (Provide An Aggregated Tally for the Preceding 3 cohorts)

2012-15 Total # of residents who were in the 3 cohorts Total # of residents who remain in training in the residency program PD EP Community mental health center Federally qualified health center Independent primary care facility/clinic University counseling center Veterans Affairs medical center Military health center Academic health center Other medical center or hospital Psychiatric hospital Academic university/department Community college or other teaching setting Independent research institution Correctional facility School district/system Independent practice setting Not currently employed Changed to another field Other Unknown

Note. “PD” = Post-doctoral residency position; “EP” = Employed Position. Each individual represented in this table should be counted only one time. For former trainees working in more than one setting, select the setting that represents their primary position.

IR C-24 P. Consortium (Commission on Accreditation, October 2016; February 2019)

I. Development of Consortium

A postdoctoral training program may consist of, or be located under, a single administrative entity (e.g. institution, agency, school, department) that controls all program resources, or within a consortium, where more than one administrative entity contributes to the consortium program resources. A consortium is, therefore, comprised of 2 or more independently administered entities, that have agreed to share resources and have developed centralized decision-making for the establishment, implementation, and maintenance of a residency program. The CoA seeks to understand the stability of a consortium's shared resources through this Implementing Regulation which specifically details the components that must be in place and described via a consortial agreement when two or more independent entities meet the above criteria to provide postdoctoral training. The written consortial agreement must include and articulate these components (a-h):

a) The nature and characteristics of the participating entities; b) The rationale for the consortial partnership; c) Each partner's commitment to the training/education program and its aim(s); d) Each partner’s obligations regarding contributions, financial support, and access to resources. e) Each partner's agreement to adhere to central control and coordination of the training program by

the consortium's administrative structure; f) Each partner's commitment to uniform administration and implementation of the program's training

principles, policies, and procedures addressing trainee admission, training resource access, potential performance expectations, and evaluations;

g) Each partner’s commitment to ensure continuation of training for residents in the consortium, particularly if at least one partner leaves the consortium; and

h) Approval by each entity's administrative authority (with authority to sign contracts for the entity) to honor this agreement including signature and date.

Consistent with IR C-20 P, any change in components a-g above and/or in the leadership of the programs in the consortium, must be communicated to the CoA. An individual consortial partner (member entity) of an accredited consortium is not and may not publicize itself as independently accredited unless it also has independently applied for and received accreditation as an independently accredited program. II. General Information for a Currently Accredited Consortium Undergoing Dissolution or the Development of New Consortium When One or More Member Entities is Currently Accredited

Given the differences in consortium programs, transition processes are complex. An accredited program that is seeking to form or dissolve a consortium is strongly advised to consult with the Office of Program Consultation and Accreditation early in the planning process. Further, consistent with IR C-20 P, the CoA must be informed in advance of such major program changes as well as the intended timeframe of the planned transition.

Per Accreditation Operating Procedures and IR D.4-6, the CoA’s responsibility for accreditation extends to programs and not individuals completing programs; therefore, the accreditation status of a program on the final day of the residency year is the status that is to be designated on program completion certificates.

Consistent with Standard V of the Standards of Accreditation, programs must be accurately and completely described in documents that are available to current residents, applicants, and the public. It is especially important for all accredited programs (independent or consortium) to communicate clearly in public materials to current and prospective residents. Such communication should include the current accreditation status of the program, the accreditation status for which the member entity is applying, and the specific training experiences of each program. In addition, communication of the program’s decision to dissolve or develop a consortium during the training year and how these changes may impact accreditation status is to be included. The general procedures and guidelines for reviewing applications are outlined in the Accreditation Operating Procedures. An expedited review process for any transition application for member entities in a consortium cannot be guaranteed. III. Specific Information Related to the Dissolution of an Accredited Consortium Member entities that comprise an existing accredited consortium may wish to separate and become independently accredited. Transitioning from being a member entity of an accredited consortium to becoming an independently accredited postdoctoral program requires each independent member entity to apply for accreditation as a separate program. When the member entity decides to separate from the consortium it may choose to: Option 1) separate entirely from the consortium and then apply for contingent or full accreditation status or: Option 2) remain in the accredited consortium while concurrently applying as an individually accredited program. In making the decision regarding which option to choose, the member entity must consider the impact of such changes on the current and/or incoming resident cohort(s). If the member entity chooses Option 1, the following applies

a) The member entity may decide to apply through two mechanisms available to all applicants: 1) “accredited, on contingency” status or 2) full accreditation. During the time that the program has separated from the consortium and has not yet been independently accredited, the program is not accredited. A member entity may also seek to declare intent to apply, consistent with IR C-21 P.

b) If the member entity applies for “accredited, on contingency” status then it must meet all requirements in IR C-22 P.

c) If the member entity applies for full accreditation it is expected to provide proximal and distal outcome data, consistent with IR C-16 P. These data must be specific to the independent site applying for accreditation. In certain cases, when consortium program data is easily attributed to the program that is seeking independent accreditation, data that has been collected during the consortium time period may be used as part of that included in an application for full accreditation. Programs seeking to do this should consult with the Office of Program Consultation and Accreditation.

If the member entity chooses option 2, the following will occur:

a) The member entity must simultaneously meet all Standards of Accreditation as the consortium member entity AND the requirements for one of the other applicant options: “accredited, on contingency”, as outlined in IR C-22 P or full accreditation.

b) Consistent with IR C-20 P, the consortium must communicate to the CoA how it will be able to meet the Standards of Accreditation without the components that the withdrawing member entity was contributing to the consortium.

c) The consortium agreement must be maintained during the transition period.

In the situation of a two-member consortium, if one-member entity withdraws from the consortium, then neither program is accredited as a consortium or as an individual program at the time of separation unless independent accreditation has already been attained by the separating entity(ies). Each member entity may decide to apply for accreditation as outlined in IR C-22 P for “accredited, on contingency” or for full accreditation. If they have separated, neither of the member entities may advertise themselves as independently accredited programs until the CoA has reviewed and approved the accreditation for each independent program. IV. Specific Information Related to Development of a New Consortium if One or More Member Entity(ies) is Currently Independently Accredited The following parameters do not apply to programs that are already accredited as a consortium and wish to add member entities into the existing accredited consortium. The addition of a consortium member to an accredited consortium should be communicated to CoA as a substantive change, consistent with IR C-20 P. In the event that two or more independent programs (at least one of which is already independently accredited) wish to enter into a newly developed consortial agreement, they may decide to apply through two mechanisms available to all applicants: 1) accredited on contingency status or 2) full accreditation. Two or more independent programs may also seek to declare the intent to apply as a consortium consistent with IR C-22 P. The accreditation status of one independently accredited program does not transfer to any other unaccredited member entity(ies) when programs join together; the unaccredited program is not and may not advertise as an accredited program or member of an accredited consortium until the consortium has applied for and received accreditation. If the independently accredited program chooses to maintain independent accreditation while concurrently applying for an accreditation status as a consortium program, then the independent program must meet all Standards of Accreditation as an independently accredited program AND as a consortium member entity. At a minimum, the basic integrity of the independent program and the training aim(s) and Level 1, 2, and 3 competencies (as appropriate) must be maintained during the transition period. Since more than one already independently accredited program may be transitioning to a consortium, it follows that each program may have additional or more refined aims and program-specific competencies that must be clarified. In addition, the program must clarify the resources (i.e., supervision, space, clerical support) available to the consortium entity and to the independently accredited program. These resources may overlap as long as both programs remain in compliance with the Standards of Accreditation, but there must be sufficient resources to maintain the programs. During the transition period, the independently accredited program and the accredited consortium may not advertise that the independent program is a member entity until the accreditation status for the consortium program has been approved by the CoA.