Session 34 PD, What Consititutes an Effective Peer Review? · Session 34 PD, What Consititutes an...
of 36
/36
Embed Size (px)
Transcript of Session 34 PD, What Consititutes an Effective Peer Review? · Session 34 PD, What Consititutes an...
Session 34 PD, What Consititutes an Effective Peer Review?
Moderator:
Benjamin Carl Farnsworth, FSA, CERA, MAAA
Presenters: Devon P. Esson, FSA, FCIA
Benjamin Carl Farnsworth, FSA, CERA, MAAA Andrew Chong Jenkins, FSA, CERA, MAAA
Benjamin FarnsworthEffective Peer ReviewApril, 2016
Peer Review
2
Agenda• Overview of Peer Review• Peer Review from Company Product Development
perspective• Peer Review from Regulatory perspective• Peer Review from External Auditor perspective• Closing Thoughts and Q&A
3
ObjectiveProvide overview of Peer Review and insights from three different perspectives
Learning Outcomes:• Identify resources that should be considered in doing a
peer review; • Identify how to effectively perform a peer review; and• Identify levels of documentation to support peer review
conclusions.
4
Overview
5
What is Peer Review?Definition: Evaluation of a professional work project…performed by a second, qualified professional
Purpose of Peer Review: improve work quality and enhance professionalism
Different types of Peer Review• Pre-release project review• Post-release project review• Post-release procedural review• Special review
Source: AAA Peer Review Discussion Paper, September 2005
6
Professionalism and Peer Review
7
Peer Review Item ASOP Code of Conduct
Documentation 21, 41
Precept 1 Annotation 1-1
Precept 4
Precept 8, Annotation 8-1
Precept 10 Annotation 10-1
Precept 13
Assumptions 21, 41
Peer ReviewerQualifications 21
Data Quality 23
Internal Communication 41
ExternalCommunication 21, 41
Reliance On Others 21, 23, 41
What is Effective Peer Review?7 characteristics of effective peer review programs:
1. Strong commitment from top management2. Viewed as positive, constructive, and aimed at improving
quality and professionalism3. Emphasize educational aspects4. Open communication by reviewer and reviewee5. Confidential findings6. Given appropriate priority7. Push towards higher quality product via feedback
Source: AAA Peer Review Discussion Paper, April 1997
8
Peer Review from a Company Product Development PerspectiveAndrew Jenkins, VP Product DevelopmentProtective Life5/16/2016
1
Congratulations!You’ve been assigned to be the peer reviewer!
2All images used in this presentation are free to use, share or modify, even commercially.
"[a]n Actuary shall perform Actuarial
Services with skill and care."
Code of Conduct Tie-In
Checklist for Success“Cover”
Senior management must buy into the peer review process. Include the step in project timelines upfront.
Complete Begin the review once the pricing work is completed. Pricing models, memos, and full financial results.
Competency Is the reviewer qualified?
Collaboration The pricing actuary needs to be accessible throughout.
Correct Check the work, especially models, for accuracy.
Challenge Ask the hard questions about the results and assumptions.
Circling Back A peer review may require multiple rounds.
"An Actuary who performs Actuarial Services shall take reasonable steps to ensure that such services are not
used to mislead other parties.”
Code of Conduct Tie-In
3
Part Of One Team
4
We need to launch! ?• External forces may cause pressure on the
peer reviewer.• An ineffective or incomplete peer review
could lead to disastrous outcomes.• Set reasonable expectations with
stakeholders.• As the peer reviewer, be mindful of
timelines, but not beholden to them.• Leverage consultants as appropriate.
"An Actuary shall perform Actuarial Services with
courtesy and professional respect and shall cooperate with others in the Principal’s
interest.”
Code of Conduct Tie-In
Validating the Pricing ModelAvoiding Garbage In Garbage Out
• Validation of models can take many forms.
• Inspection.• Replication.• Reasonableness.
• Necessary to have familiarity with the pricing software and relevant common pitfalls.
• Ensure access to all model audits and code documentation.
• A technical skillset is required and can be time consuming.
5
“Where a question arises with regard to the applicability of a standard of
practice, or where no applicable standard exists, an Actuary shall
utilize professional judgment, taking into account generally accepted
actuarial principles and practices.”
Code of Conduct Tie-In
Effective Challenge
“It is impossible to separate a cube into two cubes, or a fourth
power into two fourth powers, or in general, any power higher than the second, into two like
powers. I have discovered a truly marvelous proof of this, which
this margin is too narrow to contain.” -Fermat
6
Following the question:• Why did the change in population cause the IRR to
decrease by 2%?• Is that mortality table a reliable basis for the
pricing assumption? Explain it like I’m five:• Reserves changed when the new feature was
introduced? • How do the reserves work exactly for this variable
universal life payout annuity? Respectfully disagreeing:• All risks have been thoroughly evaluated! • “Have you considered the interaction between risk
A and risk B?”
"[d]ifferences of opinion among actuaries may arise, particularly in choices of
assumptions and methods. Discussions of such differences between an Actuary and another actuary, or in observations made by an Actuary to a Principal on the
work of another actuary, should be conducted objectively and with courtesy
and respect.”
Code of Conduct Tie-In
Completing the Review• Findings can be shared as they are identified to
expedite the process.
• After any issues are resolved, it is ideal to repeat some components of the review.
• Public praise and credit to the peer reviewer can help to build “cover.”
• Understand where processes failed or critical thinking was lacking.
• Prepare “after-action” reports for interested parties.
7
Professionalism
Code of Conduct Tie-In
The Customer Impact
• Higher quality products with robust pricing.• Peer review can sometimes improve
competitiveness!
• Fewer reasons to redetermine non-guaranteed elements.
• Greater comfort in the integrity of the life insurance and annuity industry.
8
“…the profession’s responsibility to the public”
Code of Conduct Tie-In
Session 34 PD: What Constitutes an Effective Peer Review?
Presenter: Devon EssonDate: May 16, 2016
2
Agenda
• History of Peer Reviews in Canada• Purpose of Peer Reviews• OSFI Guideline E-15
Overview Work to be Reviewed Peer Review Report Timing Reviewer Selection/Change
• Experience• Questions
3
History of Peer Reviews in Canada
• Introduced in 2003 as “External Review”
• A component of OSFI Guideline E-15• Applicable to federally regulated
insurance companies• E-15 updated in 2012• Approx. 13 years of experience
4
Purpose of Peer Review
• Assists OSFI in its assessment of an insurer’s safety and soundness
• Provides a source of independent consultation advice and professional education
• Maintains and strengthens public confidence in work of the Appointed Actuary (“AA”)
5
OSFI Guideline E-15: Overview
• Describes role of AA and sets out OSFI’s expectations
• Three major sections1. Legal Requirements of the AA2. Qualification Requirements3. Peer Review
6
E-15: Work to be Reviewed• Note: Not intended to duplicate external
auditor’s work• Valuation of Policy LiabilitiesAccepted actuarial practiceAssumptions and methods Changes affecting valuation
• Adequacy of procedures, systems and work of others relied on by AA
• Appointed Actuary Report disclosure• Actuarial assumptions and judgements in
Regulatory Capital• Future Financial Condition analysis and
reporting
7
E-15: Peer Reviewer Report• Written report documenting findings• To Audit Committee or Chief Agent and OSFI• ContentsWork doneTimingMaterialityOpinionObservations w.r.t. changes in methods &
assumptionsAcknowledgement of no additional material
changesRecommendationsRelationship with AA
8
E-15: Timing• Peer Review CycleAt least once every 3 years either all at
once or in phases over 3 year cycleAnnually for changes material to the
valuationAnnually for DCAT scenarios
• Timing of Work and ReportsDifferent timing of AA work multiple
reports possiblePre- or post-releaseReporting to OSFI
9
E-15: Reviewer Selection • Selecting a ReviewerSingle reviewerSame qualification standards as for AAReviewer expected to have sufficient
experience w.r.t. work to be reviewed Reviewer must be objectiveNotify OSFI
• Changing a ReviewerRegular change or rotation expectedNotify OSFI
10
Experience• 2012 Update
OSFI’s supervisory framework updated in 2011 with “Actuarial” added as an oversight function
Peer review requirements updated to aid in assessment of Actuarial oversight function
• Regulatory PerspectiveProcess has worked wellPeer reviews are a key element of OSFI
assessment of the safety and soundness of companies
OSFI’s intention is to periodically meet with peer reviewer to discuss report and findings
Peer Review findings discussed with AAAAs’ feedback is that the process is helpful
11
Questions?
Benjamin FarnsworthEffective Peer ReviewApril, 2016
Peer Review from External Auditor perspective
2
External Auditor Peer Review AssessmentThe auditor’s role is to evaluate the evidence of the peer review. They are NOT the peer reviewer. • Was the scope clearly defined? • Were the assumptions evaluated for
reasonableness? • Does the documentation give findings that are
supported by the analysis? • Is the potential variability of results discussed? • Is the peer review report complete?
3
Peer Review Challenges• Cost of peer review (e.g. resources, time) • Availability of peer reviewer and peer review
qualifications (ASOP 21) • Scope of the peer review• Reconciling differences of opinion• Peer review documentation
The preparing actuary (not the peer reviewer) has primary responsibility for the work product.
4
Effective Peer ReviewClosingThoughtsProduct development Perspective
• Integral and planned part of the product development process• Thorough review of all pricing methods, models, and conclusions
Regulatory Perspective• Strengthens the public’s confidence in the contributions of the AA to
the financial statements.• Assists Regulators in assessing the safety and soundness of financial
institutions• Source of education and independent consultation, benefitting both
the AA and the Peer Reviewer
External Audit Perspective• Emphasis on the work procedures used to ensure a quality work
product of the preparing actuary
5
Q&A
6
Contacts• Andrew Jenkins, Protective Life Insurance Company
• Devon Esson, Office of the Superintendent of Financial Institutions Canada
• Benjamin Farnsworth, KPMG• [email protected]
7
Reference Materials• AAA Peer Review Discussion Paper, 1997• AAA Peer Review Discussion Paper, September 2005• ASB Actuarial Standards of Practice 21, 23, 41• AAA Code of Professional Conduct• ASB Proposed ASOP on Modeling (Exposure Draft)
8
