Title:Temporal representation of care trajectories of cancer patients using data from a regional information system: an application in breast cancer

Transcription

1 Author's response to reviews Title:Temporal representation of care trajectories of cancer patients using data from a regional information system: an application in breast cancer Authors: Gautier Defossez (gautier.defossez@univ-poitiers.fr) Alexandre Rollet (alexandre.rollet@chd-vendee.fr) Olivier Dameron (olivier.dameron@univ-rennes1.fr) Pierre Ingrand (pierre.ingrand@univ-poitiers.fr) Version:3Date:12 November 2013 Author's response to reviews: see over

2 Author s response to reviews Title: Temporal representation of care trajectories of cancer patients using data from a regional information system: an application in breast cancer Authors: G Defossez, gautier.defossez@univ-poitiers.fr A Rollet, alexandre.rollet@chd-vendee.fr O Dameron, olivier.dameron@univ-rennes1.fr P Ingrand, pierre.ingrand@univ-poitiers.fr Version: 2 Date: Novembre 2013 Poitiers, november 12, 2013 Manuscript: Arlene Pura Journal Editorial Office BioMed Central Dear Arlene Pura, Thank you for considering the above paper and for the reviewers helpful comments. I send you the revised manuscript Temporal representation of care trajectories of cancer patients using data from a regional information system: an application in breast cancer, to your editorial consideration to publication in BMC Medical Informatics and Decision Making. The manuscript is now word long with 45 references (8 added), 3 figures (1 added) and 2 tables. The abstract is 330-word long. This manuscript has been modified according to the suggestions of the reviewers. Our pointby-point responses to their comments are attached. The manuscript was translated and edited by our English-native colleague Angela Swaine Verdier. We hope that our responses have now made the manuscript acceptable for publication. Thank you for your consideration. Sincerely yours Gautier Defossez, MD Unité d épidémiologie, biostatistique et registre des cancers de Poitou-Charentes Faculté de médecine, Centre Hospitalier Universitaire de Poitiers, Université de Poitiers 6, rue de la milétrie - BP POITIERS Cedex, France Tel: (+33) Fax: (+33) Mail : gautier.defossez@univ-poitiers.fr 1

3 Reviewer's report Title: Temporal representation of care trajectories of cancer patients using data from a regional information system: an application in breast cancer Version: 2 Date: 26 September 2013 Reviewer: Peter Raich Reviewer's report: This is a well written paper documenting the value of an algorithm to define the timing of multiple therapies in a group of breast cancer patients in 3 standardized settings. The excellent results achieved are due to the thorough data collection of starting and ending dates for the various treatments given. Unfortunately, this is not the case in many cancer registries throughout the world. Accurately defining the sequence of treatment as well as the time-lapses are very important to assess quality of cancer care, thus this is a very worth wile contribution to the literature. Major Compulsory Revisions none Minor Essential Revisions 1. Clarify the description of the 16 particular cases described on page 24 and Table 2. The grouping of patients is different in the text compared to the table. For example, "Two patients resisted surgery,..." but 3 scenarios are listed after that sentence - A, AK, AR. We apologize for this mistake and have made the correction in the text. It is ten patients (and not two) who refused surgery or whose advanced age constituted a counter-indication for surgery: 6 who received only a biopsy ( A ), 2 who received a biopsy and chemotherapy ( AK ) and 2 who received a biopsy and radiotherapy ( AR ). 2. Add a brief description of the Multidisciplinary Consulting Meetings. I take it that these are the same as Institutional Tumor Boards in the US. These meetings are rightly named tumor boards by the National Cancer Institute, which defines them as a treatment planning approach in which a number of doctors who are experts in different specialities (disciplines) review and discuss the medical condition and treatment options of a patient. We add this sentence: These meetings are named Institutional Tumor Boards in the United States. Discretionary Revisions 1. The authors rightfully address the value of knowing time-lapse periods between different treatments and describe some of the prior studies and literature addressing such time-lapses. It would be interesting for the authors to analyze and report on the actual time-lapses observed in this study population. However, this would require additional review of the literature and an expansion of the current manuscript. 2

4 Hopefully, this will be the topic of a future manuscript, since excessive time-lapses present a major impediment to quality cancer care. We agree with this remark. We analyzed time-lapses in the study population and compared with the literature and guidelines with interest. But if the editor agrees, we prefer to focus this manuscript on the methodology and evaluation of the algorithm and to propose in the future a new manuscript on results drawn from a larger sample. Level of interest: An article of importance in its field Quality of written English: Acceptable Statistical review: Yes, but I do not feel adequately qualified to assess the statistics. Declaration of competing interests: I declare that I have no competing interests 3

5 Reviewer's report Title: Temporal representation of care trajectories of cancer patients using data from a regional information system: an application in breast cancer Version: 2 Date: 5 October 2013 Reviewer: Niels Peek Reviewer's report: The paper by Defossez and colleagues presents an algorithm for extracting care pathways of individual patients from clinical databases. The building blocks of these care pathways were manually derived from clinical practice guidelines. The algorithm was evaluated a sample of 159 female patients diagnosed with non-metastatic breast cancer. The discovery and extraction of clinical pathways from events logs and other clinical datasets is a vivid research area, and has potential benefits for clinical practice and quality improvement in healthcare. However, in my opinion the work by Defossez et al. is currently insufficiently mature to be published in the BMC Medical Informatics and Decision Making, for the reasons explained below. MAJOR COMPULSORY REVISIONS 1. After a brief introduction on care trajectories and clinical practice guidelines, the paper focuses completely on the algorithm, without discussing its clinical significance. I would encourage the authors to close the circle, and discuss (perhaps illustrate, or even show) the meaning and value of their work for clinical practice. The main objective of our study was to represent a concise and comprehensive summary of care trajectories over time in order to produce indicators intended to help to improve care planning. Benefits are expected for institutions first and for health professionals rather than for individual clinical practice. We added to the introduction (section 1.2 Interest of modelling care trajectories ) an additional justification on the meaning and value of our work for clinical practice. Modelling care trajectories offer an explicit process-oriented view of healthcare and will enable routine evaluation of the compliance of observed care trajectories with those set out in guidelines. The production of care trajectories and waiting-time indicators at the regional population level should contribute to improve care planning, ultimately ensuring that all patients have access to the appropriate treatment within an appropriate time-lapse. 2. The extraction of clinical pathways and workflows from clinical data is a vivid research area (e.g., see [1,2]. For instance, Huang and colleagues have recently described methods for discovering and summarizing clinical pathways from clinical events logs [3,4]. Bouarfa and Dankelmand have described a workflow mining approach to derive workflows from clinical event logs and to automatically detect outlier workflows [5]. Wang et al. have proposed a method for creating personalised clinical pathways by semantic interoperability with electronic health records [6], and Combi et al. have provided a formal characterization of temporal similarity measures for querying clinical workflows from clinical databases [7]. Even though this is but a small part of the literature in this area, none of the publication mentioned is referenced in the paper. The 4

6 section on comparison with the literature [p ] briefly touches upon the sequence analysis work by Abbott, and then focuses on clinical registries -- which is not very relevant for the work described. So, the work is insufficiently embedded in the methodological literature on clinical pathway analysis and clinical workflow mining. We agree with the reviewer that many techniques and tools have been applied for clinical pathways analysis and clinical workflow mining. While methods of data mining aimed to discover detail on the pathway in ensuring specialised, standardised and normalised therapy procedures, our work focused on the development and the evaluation of an algorithm representing care trajectories over time restricted to the main stages in the initial care provision for non metastatic breast cancer identified in current updated guidelines. The main difference between our study and other published works is to assess the reliability and the validity of the represented care trajectories with data collected in medical files, regardless of cancer registry data used by the algorithm. The methods proposed in this work respond to needs expressed by institutions and health professionals of routine indicators aimed to improve the quality of patient care. Clinical pathways discover models in a descriptive approach which provides too much detail to give a concise and comprehensive summary of the pathway and are usually restricted to a single hospital information system. Our work focused on the analysis of care trajectory on relevant and accurate information extracted from medical source records coming from every hospitals and health structures involved in cancer care on a large geographical scale and for selected patients according to strict clinical criteria (new cases of non-metastatic breast cancer registered on international rules). We add eight new references in the manuscript: 8. Hu Z, Li JS, Zhou TS, Yu HY, Suzuki M, Araki K. Ontology-based clinical pathways with semantic rules. J Med Syst Aug;36(4): Huang Z, Lu X, Duan H, Fan W. Summarizing clinical pathways from event logs. J Biomed Inform Feb;46(1): Huang Z, Dong W, Ji L, Gan C, Lu X, Duan H. Discovery of clinical pathway patterns from event logs using probabilistic topic models. J Biomed Inform Sep Huang Z, Lu X, Duan H. On mining clinical pathway patterns from medical behaviors. Artif Intell Med Sep;56(1): Rebuge A, Ferreira DR. Business process analysis in healthcare environments: A methodology based on process mining. Information Systems ;37: Bouarfa L, Dankelman J. Workflow mining and outlier detection from clinical activity logs. J Biomed Inform Dec;45(6): Wang HQ, Li JS, Zhang YF, Suzuki M, Araki K. Creating personalised clinical pathways by semantic interoperability with electronic health records. Artif Intell Med Jun;58(2): Combi C, Gozzi M, Oliboni B, Juarez JM, Marin R. Temporal similarity measures for querying clinical workflows. Artif Intell Med May;46(1): We have made modifications in the introduction section (section 1.3): We replaced this section: Most of the research published in the area of health concerning the representation of care trajectories use data mining methods to seek sequential patterns corresponding to the most frequent patient trajectories [6], or else conduct formal analysis of concepts enabling the description of patient flows generating easily understandable visual representations [7]. The main limitation of these studies is the restriction of their analyses to 5

7 a single source of data, and often to a limited period, generally a calendar year. To our knowledge, there is no approach to date that has integrated multiple-source data. By The representation of care trajectories using data mining methods is a vivid research area. These methods are useful to seek sequential patterns corresponding to the most frequent patient trajectories and to conduct formal analysis of concepts enabling the description of patient flows generating easily understandable graphical representations [6-15]. While methods of data mining aim to discover details in clinical pathways in ensuring specialised, standardised and normalised therapy procedures, the number of patterns discovered need to be restricted to the main stages defined by guidelines when the objective is to provide routine evaluation indicators of the compliance of observed care trajectories to guidelines. Clinical pathways discover models usually restricted to a single hospital information system. To our knowledge, there is no approach to date that has integrated multiple-source data coming from every hospitals and health structures involved in cancer care. We added justification in the discussion section (section comparison with the literature): We replaced this section: The research published in the area of health is far more sparse. Several authors have worked on analysing care trajectories for the purpose of classification and planning, but the data used was restricted to a single source, and spanned only one year, on account of difficulties in serialising data. By Many techniques and tools such as data mining, workflow mining or process mining [6-15] give the conceptual framework to clinical pathways analysis. Clinical pathways discover models which provide too much detail to give a concise and comprehensive summary of the pathway and are usually restricted to a single hospital information system. Our work focused on the representation care trajectory to the main stages in the initial care provision for non metastatic breast cancer identified in current updated guidelines, based on relevant and accurate information from medical source records coming from every hospitals and health structures involved in cancer care on a large geographical scale. To our knowledge this work is the first to assess the reliability and the validity of the representation care trajectories with data collected in medical files, regardless of cancer registry data used by the algorithm. The method proposed in this work responds to needs expressed by institutions and health professionals of routine indicators aimed to improve the quality of patient care. 3. As far as I can tell, the evaluation that was conducted with the dataset of 159 breast cancer patients only shows that the authors have correctly implemented their own algorithm. Basically, this was done by manually executing the main steps of the algorithm, and comparing the results with what came out when the computer algorithm was used. This is software testing -- not scientific evaluation. This is an important point for the evaluation that we have probably insufficiently described. We wrote page 20 (first paragraph) that the quality of the representation of care trajectories produced by the algorithm was assessed on a sample of patients for whom the care trajectories were also reconstructed manually from data in the medical files. This means that each sequence was generated twice and independently from regional cancer registry data (as algorithm sequences ) and from medical files reports (as observed sequences ). All pathology reports, surgical reports, reports of treatment consultation as chemotherapy or radiotherapy, multidisciplinary consulting meeting and other medical reports relating to the management of breast cancer patients were directly collected in the medical files in order to 6

8 generate sequences representing real-life patient trajectories for the non-metastatic breast cancer sample. Each dated event was documented and then entered into a database. The observed trajectories were structured according to the simple form and the extended form, to enable confrontation with the trajectories produced by the algorithm. Finally a similarity measure was performed on all pairs of trajectories (algorithm sequences against observed sequences) using the Levenshtein Edit Distance. We made three modifications in the manuscript. First, we added a new sentence in the first paragraph of section 3.3 page 20: Each sequence was generated twice and independently from regional cancer registry data (as algorithm sequences ) and from medical files reports (as observed sequences ). Second, we added a new figure in the manuscript (Figure 3) to explain the evaluation strategy of temporal representation of care trajectories. Third, we added precisions in the section objectives of the manuscript. The main objective of this study was to develop a representation of care trajectories over time for new cancer patients, using the data from the Poitou-Charentes Regional Cancer Registry information system, and to assess the reliability and validity of this representation by confronting trajectories derived from cancer registry data used by the algorithm with observed care trajectories documented from medical files. Figure 3: Evaluation strategy of temporal representation of care trajectories Target trajectories as observed sequences: trajectories manually documented from medical files Algorithm trajectories as algorithm sequences: trajectories automatically produced by the algorithm from cancer registry source data 7

9 4. On p. 23, it is written that 90% of the observed sequences were identifiable in the three standard sequences derived from the guidelines. The observed sequences are listed in Table 2, grouped by standard care sequence. Please describe the criteria for identifiability. For instance, how do you decide that "AD" and "D" belong to care provision for non-metastatic breast cancer in case of good prognosis, and not the other standard sequences? Similar remarks hold for most of the other observed sequences: it is unclear how it was decided to which standard care sequence they belong. Also, what happens when there are missing clinical data (e.g. the "K" is missing from "ADKR")? The description of the sequences observed in the sample corresponds to the real-life patient trajectories generated on documented data derived from medical files. Each of the observed sequences has been elaborated on the basis of collected documents. Among the 81 sequences corresponded to the standard ADR sequence, 21 patients had total mastectomy documented with biopsy ( AD ) or without ( D ) and no relapse factor and hence absence of radiotherapy according to the guidelines. Following the same example, 13 patients had partial or total surgery on the basis of clinical presentation without biopsy and relapse risk factors providing indication for radiotherapy. Missing data may occur in source datasets available for the algorithm along with coding errors. Conversely observed sequences could not contain any missing clinical data. 5. The authors claim that their method works on a "multi-source" information system, i.e. "the data is fragmented across numerous sources in different geographic locations" (p. 34). I do not see how the method accommodates for this situation, it seems to operate a single dataset, obtained from a clinical registry. Perhaps the data were fragmented before they were collected in the registry, but data collection for the registry is not part of the method and therefore claims regarding this aspect should be left out of the scientific conclusion. We described in the background section how the general cancer registry collects multisource data on cancer and process to registering tumours in order to publish cancer statistics on all new cases reported. Successful cancer registration depends on obtaining information from as many sources as possible. The general cancer registry faces with the task of extracting the most relevant and accurate information from source records in order to create the cancer registration. More detailed information on the summarization process and notification algorithm could be finding in the article published by Jouhet and al in 2013, reference [24] (Jouhet V, Defossez G, CRISAP, CoRIM, Ingrand P: Automated selection of relevant information for notification of incident cancer cases within a multisource cancer registry. Methods Inf Med 2013, 52(4)). Each case derived from the notification algorithm is manually checked by registry staff by visual inspection of information sources, assessing the need to refer to a patients medical record to register the tumour. All registered tumours contain relevant information including date of diagnosis, ICD-O3 topography, ICD-O3 morphology and basis of diagnosis, and they are systematically related to the data sources. Following this step, the source data related to non metastatic breast cancers can be selected on the basis of this relationship to enable the representation of the care trajectories. In summary, the tracer events for the representation of care trajectories are identified in hospital discharge data and anatomical pathology data related to non metastatic breast cancers selected for the sample. A single dataset (Figure 1) is then created after ordering data sources related to these tumours. 8

10 MINOR ESSENTIAL REVISIONS 6. On p. 12, it is written that "a state can correspond to the aggregation of several events according to certain rules so as to obtain the desired granularity". These rules are listed in Table 1. Please describe how these rules were assessed, and by whom. How do we know that they are sound and complete? The main stages in the initial care provision for non metastatic breast cancer are given by the national guidelines as the desired granularity. Soundness and completeness are ensured by experts consensus. The rules for aggregating events into states were assessed by us in order to respect and differentiate two distinct care episodes without loss of information. The aggregation complies with the two rules specified section 3.2.3: aggregation is not performed if an event occurs between two events that would normally have been aggregated or if the time-lapse between two events of the same type is too long. For instance the repeated occurrence of several events in chemotherapy is equivalent to the definition of state chemotherapy because no others events occurs between. Evaluation of the validity of these rules was a part of the evaluation strategy. DISCRETIONARY REVISIONS 7. The description of the algorithm focuses too much on implementation details, in particular on p. 19. We deleted the second part of the last section describing the calculation of time-lapse already illustrating into the Figure 2: Each time-lapse is calculated from the sum of the number of characters extracted at the end of the previous state and the start of the following within a sequence, plus one day. For instance, if we wish to extract the time-lapse to radiotherapy in standard ANDR sequences (Figure 1c: care provision for non-metastatic breast cancer in case of voluminous, infiltrating and/or inflammatory cancer), the extraction of the number of characters present between the end of state D and the start of state R corresponds to the timelapse to radiotherapy. 8. A distinction should be made between presentation and representation in step 4 of Section 3.2 and in Section Representation is about the underlying model of reality that is used by the algorithm; presentation is about the things that are shown to the user of a system. We replaced representation of care sequences by presentation of care sequences in step 4 of section 3.2 and in section In the "Results" section, do not literally repeat information that is presented in tables. The description of the sequences observed in the sample is an important part of the manuscript in order to justify how it was decided to which standard care sequence they belong. If the editor agrees, we would like to keep this description for a better understanding for the readers. 10. Use counting (first, second, third, etc) to structure the "Limitations" section. We structured limitations section in the discussion with counting like first, second, etc. 9

11 REFERENCES [1] Rebuge & Ferreira, Information Systems 37 (2012) [2] Huang et al., Artificial Intelligence in Medicine 56 (2012) [3] Huang et al., Journal of Biomedical Informatics 46 (2013) [4] Huang et al., Journal of Biomedical Informatics, in press. [5] Bouarfa and Dankelman, Journal of Biomedical Informatics 45 (2012) [6] Wang et al., Artificial Intelligence in Medicine 58 (2013) [7] Combi et al., Artificial Intelligence in Medicine (2009) 46, Level of interest: An article of limited interest Quality of written English: Needs some language corrections before being Published Statistical review: No, the manuscript does not need to be seen by a statistician. Declaration of competing interests: I declare that I have no competing interests. 10