Statistical models for supporting medical diagnosis, therapy and training

Transcription

1 Statistical models for supporting medical diagnosis, therapy and training Gábor Székely Computer Vision Laboratory Swiss Federal Institute of Technology Zürich 27 th January 2011 KÉPAF 2011, Szeged 1 of 48 A historical remark The holy grail of medical image analysis: automatic interpretation of radiological images Numerous reasons huge amount of data: productivity objective criteria reproducibility Original assumption: Segmentation can be performed completely based on image data This is practically never true A radiologist always relies on prior knowledge about the problem anatomy, pathology image formation, tissue appearance 27 th January 2011 KÉPAF 2011 Szeged 2 of 48 1

2 Integration of prior knowledge Classical AI approaches rule-based expert systems expert knowledge is too complex Deformable anatomical atlas a generic anatomical atlas carries all relevant information mapped to individual data through physical deformation driven mostly by feature/intensity similarity problems deformation process is biological nonsense perfect warping is not enough: correspondence!! reflects a static (individual) anatomy 27 th January 2011 KÉPAF 2011 Szeged 3 of 48 Learning from examples Traditional approach: pattern recognition Learning problem structure in high-dimensional feature spaces neural networks knn classification discriminant analysis support vectore machines random forests Compact characterization of the sample distribution principle component analysis independent component analysis 27 th January 2011 KÉPAF 2011 Szeged 4 of 48 2

3 Statistical anatomical models Description of shape variability Starting from a training set of examples result of manual segmentation Generation of shape descriptor parameters point distribution models parametric shape models Normalization in anatomical coordinate system The variability of the anatomy is limited shape variations organ position Characterized by the distribution of the parameters For multivariate Gaussian mean values covariance (eigenvectors - modes of variation) 27 th January 2011 KÉPAF 2011 Szeged 5 of 48 3D example: Basal Ganglia Segmentation of deep gray matter volumetric MRI data (gradient echo: SPGR) Structures to be identified putamen globus pallidus thalamus hippocampus Training set: 30 hand-segmented volumes Anatomical reference midsaggital plane AC/PC line Mean shape and first 10 eigenmodes for segmentation 27 th January 2011 KÉPAF 2011 Szeged 6 of 48 3

4 Basal Ganglia: training data the basal ganglia individual dataset training data for left hipocampi 27 th January 2011 KÉPAF 2011 Szeged 7 of 48 Basal Ganglia: shape parametrization Hippocampus approximation by spherical harmonic functions 27 th January 2011 KÉPAF 2011 Szeged 8 of 48 4

5 Basal Ganglia: statistical model 27 th January 2011 KÉPAF 2011 Szeged 9 of 48 Basal Ganglia: segmentation process 27 th January 2011 KÉPAF 2011 Szeged 10 of 48 5

6 Extension of statistical models Knowledge about organ appearance is also essential e.g. characterization of gray-value profiles along organ boundaries: supports the search for optimal boundary position Unified framework for shape and appearance active appearance models Multi-organ models mutual support through inter-organ shape correlations 27 th January 2011 KÉPAF 2011 Szeged 11 of 48 Prostate irradiation: model building Manual segmentation Surface net Parameterization Profile extraction Spherical harmonic representation Shape and profile model 27 th January 2011 KÉPAF 2011 Szeged 12 of 48 6

7 Bladder segmentation Patient-CT Volume data Initialization Segmentation Mean shape Variation modes Profile model 27 th January 2011 KÉPAF 2011 Szeged 13 of 48 Multi-object model Bladder model Prostate model Jointly trained model 27 th January 2011 KÉPAF 2011 Szeged 14 of 48 7

8 Segmentation results Bladder: 100 Iterations Bladder-Prostate: 50 Iterations 27 th January 2011 KÉPAF 2011 Szeged 15 of 48 Problems of statistical models... Establishment of inter-individual anatomical correspondence much progress but still an open issue Variability should be covered by the training set problems with pathology Less straightforward for more compex distributions different variants of X-component analysis (ICA) They cannot represent all prior knowledge 27 th January 2011 KÉPAF 2011 Szeged 16 of 48 8

9 ... but they are more than just regularizers Spatio-temporal (4D) models for motion and flow characterization Prediction from (very) sparse, incomplete data 27 th January 2011 KÉPAF 2011 Szeged 17 of 48 Statistical models for physiology Anatomy just describe organ geometry In most cases we are interested in more the functional behaviour of the body Physiology is inherently dynamic We need spatio-temporal models for characterization Typical examples: organ motion blood circulation / flow 27 th January 2011 KÉPAF 2011 Szeged 18 of 48 9

10 Dynamic models for clinical support Treatment of moving organs finding pathologies during surgery or radiation therapy significant motion (breathing, heartbeat) shooting on moving target Understanding flow-related pathology formation and development atherosclerosis aneurysm formation 27 th January 2011 KÉPAF 2011 Szeged 19 of 48 Imaging of organ motion Real-time Optical/Photogrammetry: only surface information X-ray: only 2D information Ultrasound: limitations in imaging volume, resolution and image quality Retrospective reconstruction CT, MRI: fully 4D images cannot be applied during interventions Pathology (target) is often not visible implantation of fiducials can be highly invasive 27 th January 2011 KÉPAF 2011 Szeged 20 of 48 10

11 MRI imaging of respiratory organ motion liver lung 27 th January 2011 KÉPAF 2011 Szeged 21 of 48 Strategy for target tracking Respiration is not periodic high variability even for a specific person Respiration can be observed off-line individual cycles can be described as a 3D trajectory field statistical models can be built from the cycles even inter-individual model can be generated again, problem of anatomical correspondence No full volumetric imaging during therapy observation of well identifiable predictors using the statistical model to extrapolate 27 th January 2011 KÉPAF 2011 Szeged 22 of 48 11

12 Liver motion trajectories S P S I I A P R L A R L 27 th January 2011 KÉPAF 2011 Szeged 23 of 48 Clinical relevance of blood flow data Fundamental relevance in pathology formation and development atherosclerosis aneurysm Flow related information needed for diagnosis therapy planning monitoring aneurysm atherosclerosis 27 th January 2011 KÉPAF 2011 Szeged 24 of 48 12

13 Blood flow field imaging Doppler ultrasound Phase contrast MRI 27 th January 2011 KÉPAF 2011 Szeged 25 of 48 Measured flow information Spatio-temporarily resolved flow vector fields Problems low resolution bad SNR especially for low velocities cannot reliably describe complex flow patterns 27 th January 2011 KÉPAF 2011 Szeged 26 of 48 13

14 Image enhancement by simulation mass flows MRI data mesh Numerical problem formulation Inflow: velocity walls: no slip CFD simulation outflow: pressure 27 th January 2011 KÉPAF 2011 Szeged 27 of 48 Improved data quality Derivation of implicit information wall stress wall shear stress pressure Major diagnostic relevance Problems most measured data discarded CFD simulation is not clinically feasible Advantages and problems from MRI simulated (Flow Streamlines at peak diastole) 27 th January 2011 KÉPAF 2011 Szeged 28 of 48 14

15 Solution: statistical flow model CFD simulations map onto base mesh 1 build database of CFD simulations patient-specific PC-MRI 2 segment PC-MRI measurement simulation database PCA of database map onto base mesh 3 enhance measurement find optimal PCA reconstruction map back onto patient mesh enhanced measurement 27 th January 2011 KÉPAF 2011 Szeged 29 of 48 Solution: statistical flow model CFD simulation s map onto base mesh 1 build database of CFD simulations patient-specific PC-MRI 2 segment PC-MRI measureme nt simulation database PCA of databas e find optimal PCA reconstruction map onto base mesh 3 enhance measuremen t map back onto patient mesh maximises amount of measured information enhanced used measureme nt all simulation work done outside the clinic clinically realistic timeframe 27 th January 2011 KÉPAF 2011 Szeged 30 of 48 15

16 2D example Measurement Enhanced Measurement Simulated (original) Recircualtion zones are well captured even though they are invisible in the measurement 27 th January 2011 KÉPAF 2011 Szeged 31 of 48 3D example recirculations not not seen seen in in measurement appear appear in in measured enhanced enhanced simulated measurement 27 th January 2011 KÉPAF 2011 Szeged 32 of 48 16

17 Organ prediction from minimal information Paradigm change in surgery minimally invasive procedures: endoscopy highly reduced damage on healthy tissue reduction of trauma, faster patient recovery 27 th January 2011 KÉPAF 2011 Szeged 33 of 48 Training in endoscopic surgery Major problems for the surgeon no direct access to the operation scene unusual visual feedback and instrument manipulation difficult to perform Special training necessary traditionally learning on patients possible replacement: virtual reality well established and succesful in pilot training Example: intra-uterine endoscopic surgery Hysteroscopy 27 th January 2011 KÉPAF 2011 Szeged 34 of 48 17

18 Components of a surgical simulator Medical background 27 th January 2011 KÉPAF 2011 Szeged 35 of 48 Components of a surgical simulator Model geometry Object textures Vessel systems Tissue parameters 27 th January 2011 KÉPAF 2011 Szeged 36 of 48 18

19 Components of a surgical simulator Immersive environment Haptic interface 27 th January 2011 KÉPAF 2011 Szeged 37 of 48 Components of a surgical simulator Bleeding simulation Cutting algorithms Tissue deformation Collision detection Fluid flow simulation 27 th January 2011 KÉPAF 2011 Szeged 38 of 48 19

20 The virtual patient 27 th January 2011 KÉPAF 2011 Szeged 39 of 48 Anatomical models for training Usually a single (a few) pre-defined models available Does not allow realistic repetitive training Fundamental target: individual, unique anatomy for every training session The variability of uterus anatomy has to be coded Statistical shape model represents the whole population 27 th January 2011 KÉPAF 2011 Szeged 40 of 48 20

21 Selecting a specific example Fully random selection Can we take patient characteristics into account? Ultrasound diagnostics before interventions Characteristic measures (distances, angles,...) can be determined Can we draw samples respecting these measurements? Mathematical complication: non-linear constraints 27 th January 2011 KÉPAF 2011 Szeged 41 of 48 The uterus editor 27 th January 2011 KÉPAF 2011 Szeged 42 of 48 21

22 The uterus editor 27 th January 2011 KÉPAF 2011 Szeged 43 of 48 Uterus deformation 27 th January 2011 KÉPAF 2011 Szeged 44 of 48 22

23 Predicting uterus expansion The cavity has to be expanded Fluid pressure is used during intervention Very complex, highly non-linear deformation difficult biomechanical problem has to be calculated in real-time 27 th January 2011 KÉPAF 2011 Szeged 45 of 48 Learning biomechanics by statistical models Deformations can be pre-calculated off-line Performing for a large number of examples A statistical model can be built Intra-uterine pressure is an additional parameter can be controlled real-time Length Pressure 27 th January 2011 KÉPAF 2011 Szeged 46 of 48 23

24 Simulated polyp removal 27 th January 2011 KÉPAF 2011 Szeged 47 of 48 Summary Statistical models originally used for coding prior anatomical/radiological knowledge in segmentation Basically emulate the learning process of a radiologist In this context an interpolation tool Performance limitations how representative is the training set how good is the intrer-individual correspondence During the past decade the focus is on prediction The prior knowledge coded allows extrapolation from very limited information Very broad application domain Still many challenges selection of optimal predictors quantitative characterization: error margins 27 th January 2011 KÉPAF 2011 Szeged 48 of 48 24