Recent clinical advances and applications for medical image segmentation

Transcription

1 Recent clinical advances and applications for medical image segmentation Prof. Leo Joskowicz Lab website: L. Joskowicz, 2011

2 Key trends in clinical radiology Film/light table Digital images/screen -- early 80 2D X-rays / US 2.5D CT / MRI -- mid D CT / MRI 3D visualization -- mid 00 3D visualization 3D anatomical modeling -- now! 3D modeling Advanced modeling coming soon L. Joskowicz, 2011

3 What is a patient-specific 3D model? Centerline Surface model and lumen Polyp Abdominal CT scan L. Joskowicz, 2011 Colon and intestine surface

4 What is the difference between 3D visualization and 3D modeling? Visualization Model rendering You do the interpretation! You fill/omit missing info L. Joskowicz, 2011 Computer interprets! Explicit delineation

5 Why patient-specific modeling? 3D visualization is great -- but it has limitations no explicit delineation no validation what you see is what is there? limited measurements limited structures discrimination 3D models allow: spatial and volumetric measurements with validation! advanced analysis wide variety of uses in the treatment cycle reduction of radiologist time, easier learning curve L. Joskowicz, 2011

6 Segmentation in commercial systems Copyright L. Joskowicz, 2007 L. Joskowicz, 2011

7 Segmentation in commercial systems Copyright L. Joskowicz, 2007 L. Joskowicz, 2011

8 Segmentation in commercial systems Copyright L. Joskowicz, 2007 L. Joskowicz, 2011

9 3D models in the patient treatment cycle Diagnosis Planning Neurosurgery -- trajectory Orthopaedics -- fixation CAD mammography Virtual colonoscopy Training Learning L. Joskowicz, 2011 Tumor follow-up Implant location MODEL Evaluation Delivery Interventional Radiology Navigation Robotics

10 Diagnosis: computer-aided radiology stenosis thrombus aneurism tumor volume with Prof. J. Sosna L. Joskowicz, 2011

11 Diagnosis: computer-aided radiology 4-phase CT dataset liver contour blood vessels tumors L. Joskowicz, 2011 with Prof. J. Sosna kidneycontour blood vessels urinary vessels with Dr. Y. Mintz

12 Planning: neurosurgery entry point with Dr. Y. Shoshan L. Joskowicz, 2011

13 Planning: neurosurgery L. Joskowicz, 2011 with Dr. Y. Shoshan

14 Planning: neurosurgery Conventional L. Joskowicz, 2011 Our method with Dr. Y. Shoshan

15 Planning: orthopaedics Fracture fixation alternatives L. Joskowicz, 2011 internal fixation external fixation with Dr E. Peleg and Profs. M. Liebergall, R. Mosheiff

16 Planning: orthopaedics L. Joskowicz, 2011 with Dr E. Peleg and Profs. M. Liebergall, R. Mosheiff

17 Planning: orthopaedics L. Joskowicz, 2011 with Dr E. Peleg and Profs. M. Liebergall, R. Mosheiff

18 Delivery: interventional radiology EM real-time tracking US and X-ray imaging add patient-specific models from CT L. Joskowicz, 2011

19 Delivery: intraoperative image guidance augmented continuous X-ray fluoroscopy L. Joskowicz, 2011 with Simbionix and Prof J. Sosna

20 Delivery: intraoperative image guidance L. Joskowicz, 2011 with Simbionix and Prof J. Sosna

21 Evaluation: brain tumor follow-up Tumor internal components solid enhancing cyst Disease progression? T2-weighted T1-weighted Dec 15, 2009 June 9, 2010 L. Joskowicz, 2011 with TA Sourasky and Dana Hospital

22 Key issue: model creation Currently mostly manual delineation slice by slice Desired automatic/nearly automatic a few clicks by physician no technician! accurate and reliable HARD! NO METHOD SUITABLE FOR ALL STRUCTURES L. Joskowicz, 2011

23 Modeling requires segmentation! Identify and delineate anatomical structure contours Very difficult task organ/pathology specific image/scanning protocol anatomical variability intensity values overlap structures proximity

24 Modeling requires segmentation! Identify and delineate anatomical structure contours Very difficult task organ/pathology specific image/scanning protocol anatomical variability intensity values overlap structures proximity

25 Clinical dataset challenges proximity calcifications stenosis

26 Segmentation state of the art Hundreds anatomical segmentation methods for a variety of structures and imaging modalities. Families of techniques: thresholding, region growing, level sets, active contours, and more... Very few, if any, in routine clinical use: huge gap between prototype and clinical use time-consuming, fragile, limited in scope require technical knowledge most have limited validation clinical benefits unproven

27 Observations (1) Segmentation specificity is unavoidable each anatomical structure and pathology has unique characteristics and nearby structures. Organ, structure, and pathology-specific algorithms: heart, liver, long bones, spine,... Very laborious top develop and validate a segmentation algorithm for each trial and error process, many man-months. No universal segmentation method!!

28 Observations (2) Applications have different requirements wrt: Accuracy Robustness User interaction Quality Consider the different requirements of 3D visualization Training simulation FEA simulation Defining application requirements ahead of time is

29 Clinical Goals Automatic or nearly automatic < 5mins user time Can be used by clinician without a technician Produces robust results Technical Requires significantly less time than developing from scratch Takes into account intensity and shape Incorporates shape and intensity priors

30 MIS validation Algorithms without validation are clinically worthless! Validation is with respect to a clinical task Validation requires a ground truth for comparison Physical anatomical models and/or phantoms are typically not available (except sometimes for bones) Ground truth is usually obtained by manual identification and/or segmentation by a user Experts: in most cases, radiologists Extrinsic comparison: compare vs. other methods Copyright L. Joskowicz, 2010

31 MIS validation: issues Large inter- and intra- variability across experts and clinical sites May not be representative of population variability Main quantitative parameters: validation set size number and type of observers intra and inter-observer manual segmentation variability surface-based and volume-based error measures Copyright L. Joskowicz, 2010

32 MIS validation: anatomical variability

33 MIS validation: anatomical variability stenosis narrowing looping

34 MIS validation: metrics Surface-based error measurements Mean surface distance RMS surface distance Maximum surface distance Volume-based error measurements Dice coefficient Volumetric overlap error Copyright L. Joskowicz, 2010

35 MIS validation: metrics Surface-based error measurements Mean surface distance RMS surface distance Maximum surface distance Reference Result Copyright L. Joskowicz, 2010

36 MIS validation: metrics Volume-based error measurements Volumetric overlap error Dice similarity Reference Result Copyright L. Joskowicz, 2010

37 All arteries 1.7mm std=0.9mm Subclavian Arteries 1.3mm (std=1.0mm) Aorta 1.6mm (std=0.8mm) Int. + Ext Carotids 1.5mm std=1.3mm Common Carotid Surface RMS Bifurcations 0.7mm std=0.7mm

38 Copyright L. Joskowicz, 2010 MIS validation: observers variability Intra-observer variability: determine how much variation there is when a single observer produced the ground-truth segmentation repeat 5 times the segmentation Inter-observer variability: determine how much variation there is between multiple observers that produced the ground-truth segmentation ask 3 radiologists to do the segmentation Observer expertise and frequency variability

39 Study: [Weltens 2001] Axial MRI slice Nine independent observers Inter observer variability Repeated delineations Inter/intra observer variability: 30% Key issue: fuzzy tumor boundaries

40 Intra-observer variability 8% 50%

41 Copyright L. Joskowicz, 2010 Carotid Lumen Segmentation and Stenosis Grading Challenge -- The MIDAS Journal

42 Copyright L. Joskowicz, 2010 MIS validation: manual annotation

43 MIS validation: ground-truth generation centerlines (manual) contours (manual) Partial Volume Segmentation Reference standard from 3PVS Copyright L. Joskowicz, 2010

44 Measurements characteristics Ground-truth: not know! Repeatability: intra-observer variability Reliability: Measure: inter observer variability correlation between repeated measurements

45 Inaccuracy vs. Uncertainty Intrinsic uncertainty about the tumor volume. Accuracy for clinical significance is unknown. Inaccuracy > Uncertainty Results can be meaningless Uncertainty may not be improved. Goal: improve accuracy to obtain: Inaccuracy < Uncertainty

46 In summary. Medical image segmentation is on the rise. Basis of many clinical applications Validation is a MUST Many methods and approaches do not re-invent the wheel I expect patient-specific 3D models to be in the clinical mainstream in 3-5 years! Copyright L. Joskowicz, 2011

47 Segmentation challenges Rigorous quantification and evaluation of segmentation algorithms performance Shape priors for pathologies Incorporation of functional information from diffusion/perfusion MRI, fmri, PET into segmentation algorithms. Copyright L. Joskowicz, 2011

48 Summary (1) Patient-specific anatomy model creation is currently a major bottleneck in many clinical applications Automatic anatomical segmentation is essential for model creation Current tools are limited in scope and coverage Clinical use requires the elimination of the technician model generation by the physician!

49 Summary (2) Great opportunity for the development and incorporation of anatomy modeling tools in commercial platforms Growing need and variety of users for anatomical models Training simulators, surgery rehersal Intraoperative guidance Computer Aided Radiology Service providers -- shifting paradigms