IBM Watson Financial Services

Transcription

1 IBM Watson Financial Services Risk & Compliance Innovation Forum Trends in Model Risk Governance Ian Francis London 24 May IBM Corporation

2 2 6/7/17

3 Where are we headed and who s setting the destination? Where are we right now? How are we going to get there?

4 Where are we headed

5

6 Direction being set by main parties Regulators Risk Management Accountants 6/7/17

7 IFRS 9 SR-11-7 FRTB TRIM CPS-220 OSFI-E23

8 Supervisory Guidance on Model Risk Management (SR 11-7/OCC )

9 CCAR "BHCs (bank holding companies)should ensure that they have sound model risk management, including independent review and validation of all models used in internal capital planning, consistent with existing supervisory guidance on model risk management (SR letter 11-7)." (Source: Comprehensive Capital Analysis and Review 2015: Summary Instructions and Guidance) The board should direct senior management to provide information about the firm's estimation approaches, model overlays, and assessments of model performance A firm should independently validate or otherwise conduct effective challenge of models used in internal capital planning, consistent with supervisory guidance on model risk management. The model review and validation process should include an evaluation of conceptual soundness of models and ongoing monitoring of the model performance (source: SR15-18)

10 TRIM Targeted Review of Internal Models The ECB s objective is to harmonise the use of the internal model approaches of banks placed under its jurisdiction Targeted review of internal models spread over several years. Complemented by quantitative impact studies (QISs) The ultimate aim of the work is to establish a level playing field for the use of internal model approaches.

11 FRTB Governance Requirements Model Governance Desk Representation, Nomination / Assessment Risk Factor Assessment Internal Risk Transfer Governance Overall FRTB Regulatory Compliance

12 Additional Global References Australia APRA GPS 112,CPS 220 Canada OSFI E23

13 Increase in Costs The rapid growth in regulatory compliance activity over Models has in turn increased the spend by firms on: ü Documentation and Development activities ü Resources to cover enhanced Model validation requests ü Resources to interpreting requirements across multiple jurisdictions ü Resources to apply processes against multiple regimes ( both change and BAU requests)

14 Summary of where we are going Desired Regulatory Outcomes Increased Model transparency / understanding Improved documentation 1 st line Model Risk ownership Evidence of effective challenge Robust and independent 2nd and 3rd line activities Desired Institutional Outcomes Efficiency of process, outcome Improved communications between Model Risk Stakeholders ( 1 st and 2 nd Line) Regulatory compliance Improved Insight and Reporting

15 Where are we now?

16 Where are we now Culture Methodology Technology Regionally

17 Maturity Stages Inventory Management Simple Model Register with Meta data Documentation Aligned Capturing Model Inputs / Outputs Assessing Model Risk Viewing and managing Model interdependencies Dynamic Workflow Integrated Linkage with Data governance Integration with Model Development tools Integration for Model Deployment and Operational control

18 Maturity Stages Monitoring & Review Simple Recording basic Model Validation outcomes Aligned Manually recording Model metrics Model Performance Model Operations Governance Framework scores Resolution activities as a result of Model Validation Validation resources Integrated Integration with Models for automated performance metrics Use of MRG tools to schedule and resource plan Model Validation Activities Data lineage and quality early warning

19 Standard Model Risk Management Organization Chief Risk Officer Title Chief Risk Officer Roles and Responsibilities Responsible for overseeing program and approving procedures Head of Model Governance Head of Model Risk Management Head of Model Risk Management Managing program Managing policies and procedures, including proposing revisions Communication with BOD, including reporting Responsible for representing the group internally Stature requirement May have complete or partial model approval authority Interfacing with regulators Head of Market Model Validation Head of Credit Model Validation Head of Model Governance Head of Market Model Validation Policy and procedure maintenance and drafting Reporting design and execution Model risk tiering Scheduling/PMO Issue and remediation tracking Focused on validation of market risk type models, valuation models based on market variables. Choice of validation groupings should be made for greatest clarity and efficiency. Head of Misc. Validation Head of Credit Model Validation Head of Operational Risk, Capital, etc. Model Validation Focused on validation of credit risk models where credit is defined as default, rather than market risk Focused on validation of other models as determined, which may include operational risk models, CRR models, etc. 19

20 Progress A view from the Fed/OCC Governance Model Design Validation Vendor Models Significant attention and discussion to evaluate current MRM practices & develop remediation plans Healthy Scepticism about Model Outputs & Limitations Attention to monitoring the entire cycle of development, implementation, use, and validation Firm-wide standards for model design, as well as more detailed procedures to guide development of specific model types Clear documentation about the original model design and processing components allows third party to follow the path of development Evidence that model developers have conducted research about industry/academic theory & practice relating to the model Increased adoption of parallel validation with enhanced safeguards Clearly understand model limitations and assumptions of vendor models to determine whether the model is appropriate for the intended use Updates to vendor methodology, data, etc are tracked and reviewed Source: OCC / FRB Presentation: Richmond Federal Reserve Bank MRM Forum - /media/richmondfedorg/conferences_and_events/banking/2016/mrmforum2016_day1slides.pdf

21 How are we getting to our destination 21 6/7/17

22 So how do we get the most efficient / effective route Understanding of the road maps Turn by Turn navigation Continual updates on performance Suitable Inventory Documented, clear process Ongoing Monitoring Traffic updates and re-routing Overview of Progress Real-time / Robust Issue Management 6/7/17

23 A good inventory is based on a solid foundation Elements of a good inventory ü Completeness and transparency using a consistent principles-based model definition ü Careful definition of use ü Robust metadata collection ü Connection to issues tracking; validation activities and documentation

24 Model Risk Management Activities Development Review Model Use Monitoring Remediation Change Management

25 Dimension to Address for an Effective MRM Process Culture Systems Process Organisation

26 Ongoing Monitoring Numerous benefits to those can establish an ongoing monitoring program üpotential Cost benefits üoperational effectiveness reduction in manual effort / errors üearly insight Model Validation Tools Inventory Model Performance Results Model Documentation

27 Model inputs defined and models used for multiple usages

28 Automated tests are set up in the initial validation

29 Automated derivative pricing validation Select relevant existing test from library, including model wrapper and test parameters can be kicked off manually or automated Model wrapper code Graphical and numerical results output for analysis

30 Implementation (Unit) Testing: Calibration Round Trip Tests how well calibration performs when the data is setup for a perfect solution. PDE Test Tests how well the model evaluates its expectation value. Smoothness Test Tests for dangerous oscillations, or excess noise, which can ruin greeks, calculations. Benchmark Test Compares pricing and greeks to those of one or more alternative implementations of the same model. Backtesting: Calibration Error Test Tests how well the model s faces fit real market surfaces. Calibration Stability Tests how stable the model s calibration is. Hedge Performance Test Tests how well the model s hedge removes risk in real market scenarios. PnL Attribution Test Tests how well the model s hedge removes risk in real market scenarios. Challenge Model Test Compares pricing and greeks to those found in a dissimilar, market standard model. Laddered Stress Test Observes behavior under application of a stress with increasing (laddered) intensity.

31 Automated Tests run Regularly

32 alerting / advanced warnings

33 Output reports for analysis and stored within governance system

34 Dashboards for high-level visibility

35 Actionable Insight

36 What else is on the road

37 Opportunity for Machine Learning, AI 37 Watson Financial Services / Presentation Title / Date

38 Centralisation

39 Thank you