Internal Model Industry Forum IMIF. July 2014

Transcription

1 Internal Model Industry Forum IMIF July 2014

2 Agenda Current challenges and best practices with internal models and IMV An Industry view Mark Chaplin 20 minutes Current challenges and best practices with internal models and IMV A Regulator s view Vishal Desai Sebastien Delfaud 20 minutes Internal Models A NED/Board level perspective Ian Marshall 20 minutes The Internal Model Industry Forum (IMIF) Jose Morago 15 minutes IMIF - Discussions All 15 minutes Networking and drinks All END

3 Internal model principles The model must be designed and built to support decision making The model should be as simple as it can be and only changed where the change can be shown to be beneficial The model should be internally consistent The model should be reliable The model should be understood

4 Why validate? Formally: Because we have to Article 124 Insurance and reinsurance undertakings shall have a regular cycle of model validation which includes monitoring the performance of the internal model, reviewing the ongoing appropriateness of its specification, and testing its results against experience. The model validation process shall include an effective statistical process for validating the internal model which enables the insurance and reinsurance undertakings to demonstrate to their supervisory authorities that the resulting capital requirements are appropriate. But really: To ensure that models are and will remain fit for purpose

5 Challenge 1 cost pressures Does your organisation consider its IMV framework to be. Not value adding and not cost effective Not value adding but cost effective Value adding but not cost effective Value adding and cost effective..and this is the view of risk/the actuaries!! 0% 10% 20% 30% 40% 50% 60% 70%

6 Challenge 2 applying proportionality How effectively does the IMV scope address proportionality/materiality elements? Value adding and effective Mostly effective/bau Partially effective/more project work Basic 0% 10% 20% 30% 40% 50% 60% 70%

7 Challenge 3 independence & maintaining it Is the independence of the IMV formally assessed? Don't know No Yes 0% 10% 20% 30% 40% 50% 60% 70%

8 Challenge 4 expert judgment Risk categories where the industry feels most confident in terms of the effectiveness of the IMV approach (% of respondents) Credit risk Operational Risk Insurance risk (e.g. life, GI or both) Market risk (e.g. equity, FX) None of the above 0% 20% 40% 60% 80% 100%

9 Challenge 4 expert judgment (continued) How is expert judgment within the IM validated in practice Other Not been possible Document review Stress & scenario testing Back testing Sensitivity analyses Peer review 0% 20% 40% 60% 80% 100%

10 Challenge 5 use and absence of misuse Main use of the IM which are currently business as usual ( % of respondents) Liquidity management Capital allocation Risk transfer decisions/reinsurance Pricing Investment portfolio optimisation Business and capital planning Concentrations measurement Risk appetites Solvency calculations 0% 20% 40% 60% 80% 100%

11 Agenda Current challenges and best practices with internal models and IMV An Industry view Mark Chaplin 20 minutes Current challenges and best practices with internal models and IMV A Regulator s view Vishal Desai Sebastien Delfaud 20 minutes Internal Models A NED/Board level perspective Ian Marshall 20 minutes The Internal Model Industry Forum (IMIF) Jose Morago 15 minutes IMIF - Discussions All 15 minutes Networking and drinks All END

12

13 Internal Model Industry Forum Validation Vishal Desai Sebastien Delfaud 19 June

14 Agenda 1. Internal Model Validation value proposition Purposes of validation Enhancing value 2. PRA approach to Validation Validation and Governance Framework effectiveness 3. What the PRA has observed Useful Not so useful Areas for consideration? 14

15 1. Internal Model Validation value proposition Purposes of Validation provide assurance to Boards and other internal stakeholders that SCR and other internal model outputs appropriately reflect firm s risk profile; ensure that outputs are used to support decisions that firms are making in respect of their businesses; and ensure internal model appropriateness on an on-going basis. Enhancing value Adherence to Firm s risk appetite in an increasingly complex environment; Understanding model performance and its limitations to support decision making; Improving the quality of capital decisions by reducing dependency risk through enhanced management actions. 15

16 2. PRA Approach to Validation Justification Validation Is the model specified correctly? Are inputs appropriate? Is the model specified correctly? Are inputs appropriate? Many firms are incorrectly calling this validation Challenging outputs to ensure the model properly reflects the risk profile Checking that processes and governance have been followed 16

17 2. PRA Approach to Validation Effective independent challenge Scope Tools Reporting Feedback Proportionality - focus validation on key judgements Independence - promotes an objective challenge of issues considered Testing of the model should confirm robustness Report outcomes and conclusions Feedback reactions to conclusions to the business Lack of quality in any of the above key activities would greatly undermine Validation framework effectiveness 17

18 2. PRA Approach to Validation Effective independent challenge Scope Tools Reporting Feedback Wrong scope: validation requires challenging the approach as well as the output E.g.: validating extent of modelled dependencies but omitting to validate the appropriateness of the adopted model structure 18

19 2. PRA Approach to Validation Effective independent challenge Scope Tools Reporting Feedback Inappropriate use of validation tools: undermines assessment of model performance and the insight it can bring EIOPA preparatory guidelines on use of Validation tools are very useful 19

20 2. PRA Approach to Validation Effective independent challenge Scope Tools Reporting Feedback Inadequate reporting: qualified model limitations are not reported in a structured way that informs management on the model reliance to support decision making 20

21 3. What the PRA has seen: useful validation Evidence demonstrating effectiveness of validation framework Proportionality in issues considered Pass / fail criteria are well articulated Tests reporting against categories such as Model performance and Firm s risk profile Remediation plan includes feedback of Board challenge on split between revalidation, model change or model limitations. Reliance on model (performance & limitations) to support decision making Reporting of validation activities: Not simply about demonstrating that requirements are met (from the capital modelling team) Reporting of activities and outcomes against various validation objectives (performance, risk profile, others..) Use of performance rating Validation reporting recognising range of requirements from stakeholders 21

22 3. What the PRA has seen: useful validation Use of in-depth scenario tests for key risks outlining: Circumstances and severity of each stress How they combine, including lags Estimated impact on the firm, and Reconciliation of loss scenarios to model output percentiles. Increased awareness of scenarios with impacts aggregating across risk categories that are key to Firm, and order of magnitude of losses which could be sustained from combinations of risks Use of expert judgment logs outlining: Compliance with expert judgment policy Reasons for the reliance on expert judgment Justifications / sensitivities behind expert selection used to evidence challenge from the validation process. Validation captures expert judgment and degrees of confidence/reliance 22

23 3. What the PRA has observed: not so useful Lack of Validation MI and/or entry to Board level: undermines the understanding of model strengths and limitations Lack of Proportionality : bottom up approach with lack of focus on the validation of key judgments Tools and testing of model do not confirm or inform model robustness Lack of Independence the challenge of model should be objective 23

24 3. What the PRA has observed: areas for consideration Role of Validation in ensuring the transition from a model owned by actuaries/modelling team to a model owned by business? Can P&L attribution frameworks be enhanced to validate the extent of some model dependencies? Strategic risk and business model sustainability enhanced role for reverse stress testing? Use of validation tools: can the standards of EIOPA Preparatory guidelines CP - 13/011 Chapter 9 be achieved? 24

25 Questions and comments? 25

26 Agenda Current challenges and best practices with internal models and IMV An Industry view Mark Chaplin 20 minutes Current challenges and best practices with internal models and IMV A Regulator s view Vishal Desai Sebastien Delfaud 20 minutes Internal Models A NED/Board level perspective Ian Marshall 20 minutes The Internal Model Industry Forum (IMIF) Jose Morago 15 minutes IMIF - Discussions All 15 minutes Networking and drinks All END

27 Three topics.. A difficult birth Validation! Reconnecting with the Board

28 A difficult birth.. A long time coming! Concerns around the underlying concepts Where is the Anglo-Saxon pragmatism? Cost effectiveness Crowding out of other priorities

29 Validation.. Provide assurance to Boards and other internal stakeholders that the SCR and other model outputs reflect the firm's risk profile

30 IMV- How advanced? 100% 90% 80% 70% 60% Mostly 50% 40% 30% 20% 10% 0% Basic Partial Value Adding

31 How advanced is IMV Governance? 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% Basic Partial Mostly Value adding 0%

32 How effective does IMV reflect risk profile? 100% 90% 80% 70% 60% 50% Mostly 40% 30% 20% 10% Basic Partial Value adding 0%

33 For which categories do you feel most confident in terms of the IMV approach 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Credit risk Market risk Insurance risk Operational risk None

34 How effective has IMV cycle been in improving of IM? 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% Basic Partial Mostly Value adding 0%

35 Cost effectiveness of IMV work 60% 50% Value adding but not cost effective 40% 30% 20% Value adding and cost effective Not value adding but cost effective 10% 0% Not value adding and not cost effective

36 Who does IM validation 60% Both 50% Internal 40% 30% 20% 10% External 0%

37 Board challenge? 100% 80% 60% Mostly 40% 20% Basic Partial Value added 0%

38 Board training? 100% 80% 60% 40% Partial Mostly 20% 0% Basic Value added

39 Board understanding? 100% 80% 60% Mostly 40% Partial 20% 0%

40 Three topics.. A difficult birth Validation! Reconnecting with the Board

41 Reconnecting with the Board - Woods and trees How does it work? What are they key drivers? Where did you get those assumptions from? What happens if I change a key assumption? What happens if I try various different scenarios Is there anything missing? It is only a model what can t it handle well? How do I know that SCR number is even in the right ballpark

42 Agenda Current challenges and best practices with internal models and IMV An Industry view Mark Chaplin 20 minutes Current challenges and best practices with internal models and IMV A Regulator s view Vishal Desai Sebastien Delfaud 20 minutes Internal Models A NED/Board level perspective Ian Marshall 20 minutes The Internal Model Industry Forum (IMIF) Jose Morago 15 minutes IMIF - Discussions All 15 minutes Networking and drinks All END

43 IM/IMV universe of key topics/challenges VALIDATION FRAMEWORK IMV operating model and process (incl. tools, resources documentation, approach to independence, data) IMV governance (e.g. IMV policy, roles & responsibilities, escalation, independence) IMV robustness and on-going improvement process (e.g. IMV controls, IMV cycle) IM s flexibility ( incl. less extreme calibrations) Solvency and capital (incl. business planning, capital allocation and risk appetite) IM USE Alternative uses of IM (pricing, asset portfolio, concentrations etc. ) IMV scope (materiality/ proportionality and firm s risk profile) Calculation Kernel and validation tools (e.g. IMV of op.risk vs. market risk, dependencies, expert judement) METHODOLOGY /TOOLS Model change policy (incl. validation criteria, pass/ fail criteria, communication to regulators etc) IM/IMV reporting to users/board (incl. validation MI) Board/executives IM awareness and training User s/board s challenge of IM/IMV CULTURE/TRANSPARENCY

44 Source: IMIF s survey May 2014 Initial key messages from the Industry: There is an overarching question about ensuring a cost effective and adding-value IM/IMV beyond complying with regulation: 30% companies assessed IM as value adding and cost effective IMV is ticking a regulatory box, we need to get it to the stage where it adds value to the business. This will come but we are probably 2 cycles away from that. IMV is of benefit but can be seen to be an expense that has yet to justify itself IMV - In order to satisfy the regulator too much time is spent documenting all challenges and responses It is yet to be seen if it [IMV] is cost effective depending on the PRA response and the required BAU costs Validation is cost effective. To be value adding it needs to be generating genuine improvement

45 IMIF Survey - Initial key messages from the industry: IMV cycle: The majority of companies do not consider IMV cycle effective in terms of ensuring an on-going improvement and challenge of the IM. Training: 50% of companies report a basic or partially effective approach to Board/Exec s IMV/IM training. Board s understanding and challenge: 40% of companies consider the Board/Execs level of understanding of the IM, its limitation and the IMV process as partially adequate. Interestingly, 68% of companies consider the level of challenge from the Board/Execs of IM/IMV as mostly effective. Calculation Kernel and validation tools: The level of confidence (effectiveness) in the IMV is lower for operational, credit and market risks as well as diversification benefits. Also questions about how to enhance the quality and use of validation tools. IM flexibility and other uses : 43% of companies report a low level of flexibility of the model (e.g. less extreme calibration for other types of decisions). Survey shows potential opportunities for alternative uses of the IM (e.g. concentration, pricing, liquidity management etc.) Source: IMIF s survey May 2014

46 IMIF Survey - Initial key messages from the industry: IMV operating model: Lack of consistency of how IMV is undertaken in terms of internal resources (47%) vs external resources (5%) vs mix of both(47%). Also 45% of companies do not assess independence formally. IMV governance: divergence in terms of maturity: 30% of companies reach a value-adding level, but 25% of companies are at basic/partially effective level. Reporting: 40% of companies report more work to be done to ensure an effective IMV/IM reporting (i.e. granularity, proportionality, timeliness, clarity) to the Board and key users. IMV scope: 30% of companies need to do more to ensure IMV effectively reflect the risk profile of the organisation and address proportionality /materiality. Model change policy: 37% of companies report a basic or partially effective IM change policy. Source: IMIF s survey May 2014

47 IMIF framework Maturity Model VALIDATION 2. IMV robustness and on-going improvement process 3. IMV governance? Approximate score based on the IMIF survey. Red: aligned to partially effective; Amber: close to S2 aligned/ Green S2 aligned and above 4.Solvency and capital USE 5. Alternative uses of the IM 1. IMV operating model and process 6. IM s flexibility 12. IMV scope Basic Partially effective /more project work? 7. Board/ executives IM awareness and training 11. Calculation Kernel and validation tools METHODOLOGY 10. Model change policy Mostly effective S2 aligned Value adding & Effective 9. IM/IMV reporting to users/board 8. User s/board s challenge of IM/IMV CULTURE/TRANSPARENCY

48 IMIF- Purpose of the forum Creating value through internal models Understand the stages of maturity in embedding a value-added IM and IMV Framework ( IM maturity model ) and the current stage of the industry. The IMIF will focus on identifying the current gaps to get to an IMV framework that works in practice. Define best practice for the different uses of an internal model and to provide guidelines on how to support these uses through a value-adding and cost-effective Internal model validation framework. Ensure alignment in the industry with regards to the build-up, use and validation of IMs. Finding a good balance between pragmatic solutions whilst meeting regulatory requirements is critical. Identify and share real life cases and examples for current IM/IMV s challenges among IMIF members The Forum will provide a communication channel with the regulator, industry associations (e.g. IRM, IFoA, ABI) and other stakeholders (i.e. consulting firms). Focus on the IM/IMV s big questions and less on the technical details.

49 Working groups will deliver the various key IMIF s key objectives 1. VALIDATION 2. USE 3. METHODOLOGY 4. CULTURE & TRANSPARENCY Key points - for discussion: Small core group of experts to focus on key topic areas Deliver a short documents on key topics and gap toolkit Inputs based on surveys, roundtables, interviews to IMIF s company members as well as from forum discussions/ presentations If possible, a dedicated consultancy for each topic would help build consensus, ensure momentum and provide the required sponsorship IMIF steering group to be formed

50 Timeline / Next Steps Feedback forms to help identifying people s interests / concerns and who can support the IMIF s deliverables Setting up initial IMIF Steering group define scope and objectives going forward Setting up initial working groups with clear plans and deliverables IMIF maturity framework and insights of the first meeting to be captures in a short paper Organise meetings (e.g. every 6-8 weeks after September )