Computerizing the CPA Examination

Presented to the

2003 APLG Meeting

by

Richard DeVore

Agenda

• Computerizing the CPA Exam• Examination Structure,

Content, and Length• Transitioning and

Conditioning• Overview of CBT• Demonstrations of Simulation

Prototypes

CPA Examination Mission Statement

“To admit individuals to the profession only after they have demonstrated the entry-level knowledge and skills necessary to protect the public interest in a rapidly-changing business and financial environment.”

Continue To Protect The Public Interest

Periodic Changes Needed

• Maintain the confidence of various constituencies

• Provide a comprehensive assessment of the entry-level knowledge and skills required by CPAs in changing business and financial environment

What’s Changed?

• Rapidly Changing Business World

• Rapidly changing role of the CPA– Demands for broader-based financial services

– Expansion of the attest function

– Need for professional adaptability and flexibility

– Rapidly-changing technology environment

Change in Entry-Level Practice

What We Need FromA New Exam

• The Ability to Test:– Recall of foundational

knowledge– Research skills– Analytical skills– Integrated knowledge and skills

What We Need FromA New Exam (cont’d)

• The Availability of:– Access to authoritative literature– Current technological tools– Secure computerized test system– Secure administrative conditions– Year-round testing

Responsibility for Decisions

• AICPA Board of Examiners– Content Committee

• Preparations Subcommittees

– Psychometric Committee– Education Task Force– Communication Task Force– Systems Oversight Task Force– Other task forces and working

groups

Outreach to Stakeholders

• Mechanisms to Guide Policy– Exposure Draft on Structure and

Content (Feb 2001)

– Briefing Paper #2 (Mar 2001)

– Exposure Draft on Transitioning & Conditioning (Nov 2001)

• Focus groups of students and young professionals (Summer 2001)

Outreach to Stakeholders

• Education Task Force– Working with AAA– Looking at curriculum and

other academic-related issues– Help students and educators

prepare for the transition to CBT

Outreach to Stakeholders

• Ongoing Communications– New Web site with practice

materials and tutorials• (www.CPA-exam.org)

– Exam Alert• (contact gjohnson@aicpa.org)

– E-mail updates to stakeholders

BOE Decisions

Examination Structure, Content

and Length

Examination StructureChanges

• New mix of content groupings• New section• Testing of skills• Testing of knowledge integration• Length of Exam

Examination Structure

• 4 Independent Sections– Auditing & Attestation

– Financial Accounting & Reporting

– Regulation

– Business Environment & Concepts

(NB: Full Content Specs available on website)

Auditing & Attestation

• Auditing Procedures• Generally Accepted Auditing

Standards• Other Standards Related to Attest

Engagements• Skills Needed to Apply That

Knowledge

Financial Accounting & Reporting

• Generally Accepted Accounting Principles for:– Business enterprises– Not-for-profit organizations– Government entities

• Skills Needed to Apply That Knowledge

Regulation

• Federal Taxation• Ethics• Professional and Legal

Responsibilities• Business Law• Skills Needed to Apply That

Knowledge

Business Environment & Concepts

• Knowledge of Underlying Reasons and Implications of Business Transactions

• Skills Needed to Apply That Knowledge in Audit and Attest Engagements

• To Be Tested In Context of Entry-Level CPA Practice

Business Environment & Concepts (2)

• Economics

• Planning and budgeting

• Corporate financial management

• Cost measurement and pricing

• Management

• Information identification, control, and analysis

Comparison to Current Exam

AUDIT Audit & Attest

LPR Regulation

FARE Account & Report

ARE – Tax Regulation

ARE – Govt Account & Report

ARE – Mngrl Business Environ

Examination Length

• 14 Hours– In Total for Four Sections

• Primary Considerations– Psychometric Reliability– Adequate Content Coverage

Proposed Section Lengths

• Auditing & Attestation – 4.5 hours

• Accounting & Reporting– 4 hours

• Regulation– 3 hours

• Business Environment & Concepts– 2.5 hours

Examination Format

• Multiple-Choice Questions• Case-Based simulations

– Test integrated knowledge

– More closely replicate real world

– Assess research, analytical skills

– Various question formats

– Business simulations 2005

Exam Expectations

• Exam success will require less memorization, more application

• Candidates expected to do their work on a computer

• Candidates expected to have technology knowledge, as well as computer skills

• Candidates expected to integrate across accounting areas

Conditioning and Transitioning

Recommendations,Rationale,

&Benefits

Conditioning Policy

• Candidates can sit for each section individually and in any order

• All 4 sections – including the first one taken – must be passed within a “rolling” 18 month period

Conditioning

Conditioning Policy(cont’d)

• No minimum score requirement for failed sections

• Failed section(s) cannot be retaken within same window

• Credit for passed section(s) retained for 18 months

Major Benefits of Conditioning Policy

• Protects the Public– Tests breadth of knowledge &

skills within a reasonable time period

• Increases fairness to candidates by eliminating minimum scores– Each section is independent

• Increased flexibility for candidates

Major Benefits of Conditioning Policy (2)

• Allows Candidates to Focus Their Study and Prepare For Each Section

• Overall Cost to Candidate is Reduced– Not required to sit for sections not

prepared to pass

– No retake of passed sections due to score on failed sections

Transitioning

• How do you equate the current sections of the CPA exam to the revised sections?

• Upon launch, how do you treat candidates who have conditioned under the paper & pencil test?

Issues to

Consider

Transitioning

Policies

Paper-Based CBT1. Audit 1. Audit & Attest

2. Fin’l. Acctg. 2. Fin’l. Acctg. and Reporting and Reporting

3. Acctg. and 3. Regulation Reporting

4. Law & Prof. 4. Bus. Environ & Responsibilities Concepts

Transitioning

• Conditioned candidates have same number of testing opportunities within same time period remaining

• Candidate can establish credit for sections passed under CBT even if paper and pencil conditioning is lost

Policies(cont’d.)

Moving Forward

Getting

From Here

To There

Launch Date

• November 2003 -- Last Pencil and Paper Administration

• Early 2004 -- First CBT Administration Window

• Will Exams Run in Tandem?– No

Getting from here to there

Computer-Based Testing

• Roles and Responsibilities of State Boards of Accountancy and AICPA Basically Unchanged

• AICPA Remains the Examination Provider

• State Boards of Accountancy Determine if a Candidate has met Requirements to Test

How CBT Will Work

The Testing Process

• CPA Examination Delivered at Prometric and State-Operated Testing Centers

• Candidate Registers with Prometric (phone or web)

• Candidate Takes the CPA Examination at a Prometric or State-Operated Testing Center

How CBT Will Work

The Testing Process

• The Process (continued)– Prometric Forwards Results to

AICPA

– AICPA Compiles and Forwards Advisory Grades and Diagnostics to Jurisdiction

– State Board Advises Candidate of Grades

How CBT Will Work

Examination SecurityIn A CBT Environment

• Test Administration– 2 IDs– Double log-on to computer– Candidate sees only one item

at a time– Video camera and walk through

surveillance– Exit checks

Critical Path Issues

• Key Milestone Areas Include:– Item Development & Pretesting– Tutorials & practice materials– Systems Development– Communications– Legislation

Getting From HereTo There

Simulations

• Definition:

Assessment of knowledge and skills in a context approximating that found on the job through the use of realistic scenarios and tasks, and access to normally available and familiar resources.

Simulation Question Types

• MC • Drop-down selection lists• Check boxes• Spread sheets (enter, drag & drop,

formula)• Forms completion• Written communication• Research (access to standards,

codes, search engines)

Advantages of Simulations

• Affordable performance testing• Skills assessment• Integration of content • Multiple correct answers • Partial credit• Access to resources• Innovative item types• Workplace realism

Simulations Design

• 20 – 40 minutes in length• Complementary to MC portion of

test• Aimed at entry-level• Developed by educators and

practitioners• Approved by practitioners• Practice related

Simulations Interface

• Tab–style screens• Consistent across simulations and

tests• Mouse-driven response• Full-screen on 17” monitors• On-line resources available• Split-screen capabilities• Review capabilities within a

simulation

Simulations Scoring

• Multiple points per task/question• Contingent scoring accommodated• Partial credit possible• Variable credit per task possible

(i.e., credit related to importance)

Demonstrations of Simulation Prototypes

AND NOW...