Session 34 PD, What Consititutes an Effective Peer Review?

Moderator:

Benjamin Carl Farnsworth, FSA, CERA, MAAA

Presenters: Devon P. Esson, FSA, FCIA

Benjamin Carl Farnsworth, FSA, CERA, MAAA Andrew Chong Jenkins, FSA, CERA, MAAA

Benjamin FarnsworthEffective Peer ReviewApril, 2016

Peer Review

2

Agenda• Overview of Peer Review• Peer Review from Company Product Development

perspective• Peer Review from Regulatory perspective• Peer Review from External Auditor perspective• Closing Thoughts and Q&A

3

ObjectiveProvide overview of Peer Review and insights from three different perspectives

Learning Outcomes:• Identify resources that should be considered in doing a

peer review; • Identify how to effectively perform a peer review; and• Identify levels of documentation to support peer review

conclusions.

4

Overview

5

What is Peer Review?Definition: Evaluation of a professional work project…performed by a second, qualified professional

Purpose of Peer Review: improve work quality and enhance professionalism

Different types of Peer Review• Pre-release project review• Post-release project review• Post-release procedural review• Special review

Source: AAA Peer Review Discussion Paper, September 2005

6

Professionalism and Peer Review

7

Peer Review Item ASOP Code of Conduct

Documentation 21, 41

Precept 1 Annotation 1-1

Precept 4

Precept 8, Annotation 8-1

Precept 10 Annotation 10-1

Precept 13

Assumptions 21, 41

Peer ReviewerQualifications 21

Data Quality 23

Internal Communication 41

ExternalCommunication 21, 41

Reliance On Others 21, 23, 41

What is Effective Peer Review?7 characteristics of effective peer review programs:

1. Strong commitment from top management2. Viewed as positive, constructive, and aimed at improving

quality and professionalism3. Emphasize educational aspects4. Open communication by reviewer and reviewee5. Confidential findings6. Given appropriate priority7. Push towards higher quality product via feedback

Source: AAA Peer Review Discussion Paper, April 1997

8

Peer Review from a Company Product Development PerspectiveAndrew Jenkins, VP Product DevelopmentProtective Life5/16/2016

1

Congratulations!You’ve been assigned to be the peer reviewer!

2All images used in this presentation are free to use, share or modify, even commercially.

"[a]n Actuary shall perform Actuarial

Services with skill and care."

Code of Conduct Tie-In

Checklist for Success“Cover”

Senior management must buy into the peer review process. Include the step in project timelines upfront.

Complete Begin the review once the pricing work is completed. Pricing models, memos, and full financial results.

Competency Is the reviewer qualified?

Collaboration The pricing actuary needs to be accessible throughout.

Correct Check the work, especially models, for accuracy.

Challenge Ask the hard questions about the results and assumptions.

Circling Back A peer review may require multiple rounds.

"An Actuary who performs Actuarial Services shall take reasonable steps to ensure that such services are not

used to mislead other parties.”

Code of Conduct Tie-In

3

Part Of One Team

4

We need to launch! ?• External forces may cause pressure on the

peer reviewer.• An ineffective or incomplete peer review

could lead to disastrous outcomes.• Set reasonable expectations with

stakeholders.• As the peer reviewer, be mindful of

timelines, but not beholden to them.• Leverage consultants as appropriate.

"An Actuary shall perform Actuarial Services with

courtesy and professional respect and shall cooperate with others in the Principal’s

interest.”

Code of Conduct Tie-In

Validating the Pricing ModelAvoiding Garbage In Garbage Out

• Validation of models can take many forms.

• Inspection.• Replication.• Reasonableness.

• Necessary to have familiarity with the pricing software and relevant common pitfalls.

• Ensure access to all model audits and code documentation.

• A technical skillset is required and can be time consuming.

5

“Where a question arises with regard to the applicability of a standard of

practice, or where no applicable standard exists, an Actuary shall

utilize professional judgment, taking into account generally accepted

actuarial principles and practices.”

Code of Conduct Tie-In

Effective Challenge

“It is impossible to separate a cube into two cubes, or a fourth

power into two fourth powers, or in general, any power higher than the second, into two like

powers. I have discovered a truly marvelous proof of this, which

this margin is too narrow to contain.” -Fermat

6

Following the question:• Why did the change in population cause the IRR to

decrease by 2%?• Is that mortality table a reliable basis for the

pricing assumption? Explain it like I’m five:• Reserves changed when the new feature was

introduced? • How do the reserves work exactly for this variable

universal life payout annuity? Respectfully disagreeing:• All risks have been thoroughly evaluated! • “Have you considered the interaction between risk

A and risk B?”

"[d]ifferences of opinion among actuaries may arise, particularly in choices of

assumptions and methods. Discussions of such differences between an Actuary and another actuary, or in observations made by an Actuary to a Principal on the

work of another actuary, should be conducted objectively and with courtesy

and respect.”

Code of Conduct Tie-In

Completing the Review• Findings can be shared as they are identified to

expedite the process.

• After any issues are resolved, it is ideal to repeat some components of the review.

• Public praise and credit to the peer reviewer can help to build “cover.”

• Understand where processes failed or critical thinking was lacking.

• Prepare “after-action” reports for interested parties.

7

Professionalism

Code of Conduct Tie-In

The Customer Impact

• Higher quality products with robust pricing.• Peer review can sometimes improve

competitiveness!

• Fewer reasons to redetermine non-guaranteed elements.

• Greater comfort in the integrity of the life insurance and annuity industry.

8

“…the profession’s responsibility to the public”

Code of Conduct Tie-In

Session 34 PD: What Constitutes an Effective Peer Review?

Presenter: Devon EssonDate: May 16, 2016

2

Agenda

• History of Peer Reviews in Canada• Purpose of Peer Reviews• OSFI Guideline E-15

Overview Work to be Reviewed Peer Review Report Timing Reviewer Selection/Change

• Experience• Questions

3

History of Peer Reviews in Canada

• Introduced in 2003 as “External Review”

• A component of OSFI Guideline E-15• Applicable to federally regulated

insurance companies• E-15 updated in 2012• Approx. 13 years of experience

4

Purpose of Peer Review

• Assists OSFI in its assessment of an insurer’s safety and soundness

• Provides a source of independent consultation advice and professional education

• Maintains and strengthens public confidence in work of the Appointed Actuary (“AA”)

5

OSFI Guideline E-15: Overview

• Describes role of AA and sets out OSFI’s expectations

• Three major sections1. Legal Requirements of the AA2. Qualification Requirements3. Peer Review

6

E-15: Work to be Reviewed• Note: Not intended to duplicate external

auditor’s work• Valuation of Policy LiabilitiesAccepted actuarial practiceAssumptions and methods Changes affecting valuation

• Adequacy of procedures, systems and work of others relied on by AA

• Appointed Actuary Report disclosure• Actuarial assumptions and judgements in

Regulatory Capital• Future Financial Condition analysis and

reporting

7

E-15: Peer Reviewer Report• Written report documenting findings• To Audit Committee or Chief Agent and OSFI• ContentsWork doneTimingMaterialityOpinionObservations w.r.t. changes in methods &

assumptionsAcknowledgement of no additional material

changesRecommendationsRelationship with AA

8

E-15: Timing• Peer Review CycleAt least once every 3 years either all at

once or in phases over 3 year cycleAnnually for changes material to the

valuationAnnually for DCAT scenarios

• Timing of Work and ReportsDifferent timing of AA work multiple

reports possiblePre- or post-releaseReporting to OSFI

9

E-15: Reviewer Selection • Selecting a ReviewerSingle reviewerSame qualification standards as for AAReviewer expected to have sufficient

experience w.r.t. work to be reviewed Reviewer must be objectiveNotify OSFI

• Changing a ReviewerRegular change or rotation expectedNotify OSFI

10

Experience• 2012 Update

OSFI’s supervisory framework updated in 2011 with “Actuarial” added as an oversight function

Peer review requirements updated to aid in assessment of Actuarial oversight function

• Regulatory PerspectiveProcess has worked wellPeer reviews are a key element of OSFI

assessment of the safety and soundness of companies

OSFI’s intention is to periodically meet with peer reviewer to discuss report and findings

Peer Review findings discussed with AAAAs’ feedback is that the process is helpful

11

Questions?

Benjamin FarnsworthEffective Peer ReviewApril, 2016

Peer Review from External Auditor perspective

2

External Auditor Peer Review AssessmentThe auditor’s role is to evaluate the evidence of the peer review. They are NOT the peer reviewer. • Was the scope clearly defined? • Were the assumptions evaluated for

reasonableness? • Does the documentation give findings that are

supported by the analysis? • Is the potential variability of results discussed? • Is the peer review report complete?

3

Peer Review Challenges• Cost of peer review (e.g. resources, time) • Availability of peer reviewer and peer review

qualifications (ASOP 21) • Scope of the peer review• Reconciling differences of opinion• Peer review documentation

The preparing actuary (not the peer reviewer) has primary responsibility for the work product.

4

Effective Peer ReviewClosingThoughtsProduct development Perspective

• Integral and planned part of the product development process• Thorough review of all pricing methods, models, and conclusions

Regulatory Perspective• Strengthens the public’s confidence in the contributions of the AA to

the financial statements.• Assists Regulators in assessing the safety and soundness of financial

institutions• Source of education and independent consultation, benefitting both

the AA and the Peer Reviewer

External Audit Perspective• Emphasis on the work procedures used to ensure a quality work

product of the preparing actuary

5

Q&A

6

Contacts• Andrew Jenkins, Protective Life Insurance Company

• andrew.jenkins@protective.com

• Devon Esson, Office of the Superintendent of Financial Institutions Canada

• devon.esson@osfi-bsif.gc.ca

• Benjamin Farnsworth, KPMG• benjaminfarnsworth@kpmg.com

7

Reference Materials• AAA Peer Review Discussion Paper, 1997• AAA Peer Review Discussion Paper, September 2005• ASB Actuarial Standards of Practice 21, 23, 41• AAA Code of Professional Conduct• ASB Proposed ASOP on Modeling (Exposure Draft)

8