Credit Scoring, Response Modelling and Insurance Rating

Transcription

1 Credit Scoring, Response Modelling and Insurance Rating

2 Also by Steven Finlay THE MANAGEMENT OF CONSUMER CREDIT CONSUMER CREDIT FUNDAMENTALS

3 Credit Scoring, Response Modelling and Insurance Rating A Practical Guide to Forecasting Consumer Behaviour Steven Finlay

4 Steven Finlay 2010 Softcover reprint of the hardcover 1st edition All rights reserved. No reproduction, copy or transmission of this publication may be made without written permission. No portion of this publication may be reproduced, copied or transmitted save with written permission or in accordance with the provisions of the Copyright, Designs and Patents Act 1988, or under the terms of any licence permitting limited copying issued by the Copyright Licensing Agency, Saffron House, 6 10 Kirby Street, London EC1N 8TS. Any person who does any unauthorized act in relation to this publication may be liable to criminal prosecution and civil claims for damages. The author has asserted his right to be identified as the author of this work in accordance with the Copyright, Designs and Patents Act First published 2010 by PALGRAVE MACMILLAN Palgrave Macmillan in the UK is an imprint of Macmillan Publishers Limited, registered in England, company number , of Houndmills, Basingstoke, Hampshire RG21 6XS. Palgrave Macmillan in the US is a division of St Martin s Press LLC, 175 Fifth Avenue, New York, NY Palgrave Macmillan is the global academic imprint of the above companies and has companies and representatives throughout the world. Palgrave and Macmillan are registered trademarks in the United States, the United Kingdom, Europe and other countries ISBN ISBN (ebook) DOI / This book is printed on paper suitable for recycling and made from fully managed and sustained forest sources. Logging, pulping and manufacturing processes are expected to conform to the environmental regulations of the country of origin. A catalogue record for this book is available from the British Library. Library of Congress Cataloging-in-Publication Data Finlay, Steven, 1969 Credit scoring, response modelling and insurance rating : a practical guide to forecasting consumer behaviour / Steven Finlay. p. cm. Includes bibliographical references. 1. Credit analysis. 2. Consumer behavior Forecasting. 3. Consumer credit. I. Title. HG3701.F dc

5 To Ruby & Sam

6 This page intentionally left blank

7 Contents List of Tables List of Figures Acknowledgements xii xiii xiv 1 Introduction Scope and content Model applications The nature and form of consumer behaviour models Linear models Classification and regression trees (CART) Artificial neural networks Model construction Measures of performance The stages of a model development project Chapter summary 28 2 Project Planning Roles and responsibilities Business objectives and project scope Project scope Cheap, quick or optimal? Modelling objectives Modelling objectives for classification models Roll rate analysis Profit based good/bad definitions Continuous modelling objectives Product level or customer level forecasting? Forecast horizon (outcome period) Bad rate (emergence) curves Revenue/loss/value curves Legal and ethical issues Data sources and predictor variables Resource planning Costs Project plan Risks and issues 61 vii

8 viii Contents 2.9 Documentation and reporting Project requirements document Interim documentation Final project documentation (documentation 64 manual) 2.10 Chapter summary 64 3 Sample Selection Sample window (sample period) Sample size Stratified random sampling Adaptive sampling Development and holdout samples Out-of-time and recent samples Multi-segment (sub-population) sampling Balancing Non-performance Exclusions Population flow (waterfall) diagram Chapter summary 87 4 Gathering and Preparing Data Gathering data Mismatches Sample first or gather first? Basic data checks Cleaning and preparing data Dealing with missing, corrupt and invalid data Creating derived variables Outliers Inconsistent coding schema Coding of the dependent variable 107 (modelling objective) The final data set Familiarization with the data Chapter summary Understanding Relationships in Data Fine classed univariate (characteristic) analysis Measures of association Information value Chi-squared statistic 125

9 Contents ix Efficiency (GINI coefficient) Correlation Alternative methods for classing interval variables Automated segmentation procedures The application of expert opinion to interval 129 definitions 5.4 Correlation between predictor variables Interaction variables Preliminary variable selection Chapter summary Data Pre-processing Dummy variable transformed variables Weights of evidence transformed variables Coarse classing Coarse classing categorical variables Coarse classing ordinal and interval variables How many coarse classed intervals should 153 there be? Balancing issues Pre-processing holdout, out-of-time and 154 recent samples 6.4 Which is best weight of evidence or dummy 155 variables? Linear models CART and neural network models Chapter summary Model Construction (Parameter Estimation) Linear regression Linear regression for regression Linear regression for classification Stepwise linear regression Model generation Interpreting the output of the modelling 168 process Measures of model fit Are the assumptions for linear regression 173 important? Stakeholder expectations and business 174 requirements

10 x Contents 7.2 Logistic regression Producing the model Interpreting the output Neural network models Number of neurons in the hidden layer Objective function Combination and activation function Training algorithm Stopping criteria and model selection Classification and regression trees (CART) Growing and pruning the tree Survival analysis Computation issues Calibration Presenting linear models as scorecards The prospects of further advances in model 193 construction techniques 7.10 Chapter summary Validation, Model Performance and Cut-off Strategy Preparing for validation Preliminary validation Comparison of development and holdout 202 samples Score alignment Attribute alignment What if a model fails to validate? Generic measures of performance Percentage correctly classified (PCC) ROC curves and the GINI coefficient KS-statistic Out-of time sample validation Business measures of performance Marginal odds based cut-off with average 219 revenue/loss figures Constraint based cut-offs What-if analysis Swap set analysis Presenting models to stakeholders Chapter summary 224

11 Contents xi 9 Sample Bias and Reject Inference Data methods Reject acceptance Data surrogacy Inference methods Augmentation Extrapolation Iterative reclassification Does reject inference work? Chapter summary Implementation and Monitoring Implementation Implementation platform Scoring (coding) instructions Test plan Post implementation checks Monitoring Model performance Policy rules and override analysis Monitoring cases that would previously 252 have been rejected Portfolio monitoring Chapter summary Further Topics Model development and evaluation with small samples Leave-one-out cross validation Bootstrapping Multi-sample evaluation procedures for large 256 populations k-fold cross validation kj-fold cross validation Multi-model (fusion) systems Static parallel systems Multi-stage models Dynamic model selection Chapter summary 262 Notes 264 Bibliography 273 Index 278

12 List of Tables 1.1 Model applications Good/bad definition Roll rate analysis Data dictionary Sampling schema Revised sampling schema Results of balancing experiments Missing values for ratio variables Values for residential status The final data set Univariate analysis report for age Calculating information value for employment status Calculating Spearman s rank correlation coefficient Univariate analysis report for credit limit utilization Revised univariate analysis report for credit limit 132 utilization 6.1 Dummy variable definitions Weight of evidence transformed variables Univariate analysis report for marital status Alternative coarse classifications Univariate analysis report for time living at current 152 address 6.6 Coarse classed univariate analysis report for time in 156 employment 7.1 Linear regression output Logistic regression output Score distribution report KS-test Univariate misalignment report A classification table Calculation of GINI coefficient using the Brown formula Effect of sample bias Through the door population Data surrogacy mapping exercise 234 xii

13 List of Figures 1.1 Linear and non-linear relationships A classification and regression tree model A neuron A neural network Stages of a model development project Likelihood of response by score Application processing system for credit assessment Mailing response over time Bad rate curve Sample window Sample size and performance Clustered data Population and sample flows Insurance quotation table Many to one relationship Entity relationship diagram Basic data checks Relationship between age and card spend Relationship between age and response Classed relationship between age and card spend Classed relationship between age and response Interaction variables Interaction effects Relationship between income and card spend A scorecard Intercept misalignment Gradient misalignment A ROC curve Population flow Population flow through the new system Extrapolation graph Extrapolated trend 238 xiii

14 Acknowledgements I am indebted to my wife Sam and my parents Ann and Paul for their support. I would also like to thank Dr Sven Crone, of the Management Science Department at Lancaster University, for his input to some of the research discussed in the book. Also my thanks to Professor David Hand of Imperial College London for bringing to my attention a number of papers that helped form my opinion on a number of matters. I m also grateful to the Decision Analytics Division of Experian UK (Simon Harben, Dr Ian Glover and Dr John Oxley in particular) for their support for my research over a number of years, as well as Dr Geoff Ellis for providing information about SPSS during a number of interesting discussions about the merits/drawbacks of different statistical packages. xiv