Argo: a platform for interoperable and customisable text mining

Transcription

1 Argo: a platform for interoperable and customisable text mining Sophia National Centre for Text Mining School of Computer Science The University of Manchester

2 Overview Sharing tools, resources and text mining workflows Challenges Interoperable infrastructure for processing and annotation 2

3 NaCTeM 1 st publicly funded national text mining centre Location: Manchester Institute of Biotechnology Phase I - Biology ( ) Phase II - Biology, Medicine, Social Sciences ( ) Phase III Biology, Medicine, Humanities, Social Sciences; Fully sustainable centre (2011- )

4 Text Types Newswire Scientific Literature Full papers/abstracts Twitter Patents Clinical records, EMR Textbooks, monographs Online forums. Domains Finance/Business Health Biology Social Sciences Humanities. Diversity of Contexts Challenges Technology Sentence Splitter Paragraph Splitter NP Chunkers C-parser D-parser Semantic parser NE recognizers Relation recognizers. Language Technology Tasks Translation Information Extraction Semantic Search Question Answering Sentiment Analysis Summarization Knowledge Discovery. Diversity Open AIRE-COAR of Applications Conference TM Modules TM Workflows Shared! Languages English French German Spanish Portuguese Italian Polish. Chinese Hindu Urdu Japanese Korean. Diversity of Languages 4

5 Metadata Text Types Newswire Scientific Literature Full papers/abstracts Twitter Patents Clinical records, EMR Textbooks, monographs Online forums. Domains Finance/Business Health Biology Social Sciences Humanities. Resource-Rich Language Technology Big Text Big Ontology Big Data Linguistic Resources Knowledge Resources Cloud Computing Tasks Translation Information Extraction Semantic Search Question Answering Sentiment Analysis Summarization Knowledge Discovery. Crowd Sourcing Languages English French German Spanish Portuguese Italian Polish. Chinese Hindu Urdu Japanese Korean OPEN SCIENCE 5

6 Requirements from TM infrastructure Modularity of TM modules Interoperability among TM modules and resources Generic across different languages, domains, and text types Adaptability 6

7 Interoperability and Adaptability Dependency Parser Resources Dictionaries Ontologies Interoperability and Adaptability in Resource-rich TM INFRASTRUCTURES! Module Module Module POS Tagger Named Entity (Annotated) Text Rule Writing Adaptation Languages Text Types Domains Greek English French German Japanese 7

8 Example: extracting proteins, annotations GENIA PennBioIE Problem: Inconsistency Type definitions Texts Incompatibility AIMed GENETAG 8

9 The problem with incompatibility Difficult to evaluate NERs Why so different among different corpora and NERs? Which NER is best for my task? NER A A: 93% B: 36% A is better than B. NER B A: 63% B: 90% B is better than A. Corpus C Corpus D 9

10 Text mining workflows A pipeline that executes particular tools and resources in order Example: semantic search PoS Tagger Dictionary Lookup NE Extraction Chunking Parsing Semantic Query Various versions (language- or domain-specific) of basic components needed for different applications and tasks Different workflows can be created, compared and evaluated by the ability to seamlessly mix and match various versions of components 10

11 Text mining workflows Interoperability Common Data Representation and Types IBM Journal of Research and Development (2011) U-Compare: a modular NLP workflow construction and evaluation system. Kano, Y., Miwa, M., Cohen, K. B., Hunter, L.,, S. and Tsujii, J. 11

12 Common Type System A common type system is required for the complete interoperability A single common type is almost impossible to impose for all developers. Solution: Maintain local type systems and bridge them via a sharable type system bridging bridging Local Type System A U-Compare Sharable Type System Local Type System B 12 12

13 Syntactic Level Document Level Semantic Level U-Compare Type System 13

14 U-Compare: Evaluate and Compare TM Worklfows library Sentence Splitter A Sentence Splitter B POS tagger A POS tagger B Workflow A Workflow B Workflow C NER F-Score A F-Score B F-Score C UIMA SD OpenNLP SD GENIA SD UIMA Tokenizer OpenNLP Tokenizer GENIA Tagger as Tokenizer GENIA Tagger Stepp Tagger OpenNLP Tagger ABNER MedT-NER GENIA Tagger as NER

15 Integrated TM/NLP processing system GUI for workflow creation Library of ready-to-use processing components Statistics, visualizations, developer APIs Supports UIMA Web-based application Interactive creation of workflows Cloud and highperformance computing Database: The Journal of Biological Databases and Curation (2012) Argo: an integrative, interactive, text miningbased workbench supporting curation. Rak, R., Rowley, A., Black, W.J. and, S 15

16 Developers Processing Components Workflow Diagramming Workflow Designer UIMA Compliance Remote Processing Manual Editing Annotator/Curator Structured Data 16

17 Processing Components Approaching 100 components (U-Compare) Additional 50 will be added soon META-NET Developed or co-developed by NaCTeM Planned: Make the library open to others to contribute Generic Listener component Developers can plug in their own locally run UIMA component to a workflow in Argo 17

18 Remote Processing Single machine execution In-house high-performance machines Distributed processing HTCondor VMware vcloud (EBI) EUPMC Planned: EC2, Azure, 18

19 Workflows Users create workflows as block diagrams Workflows can be shared among users Read only Planned: Read & write Planned: downloadable workflows Workflows can be deployed as web services Plain text (input only), XMI, RDF, BioC 19

20 Workflows view 20

21 Workflow Editor 21

22 Sample Use Cases 1 Recognition of chemical entities (chemical NER) 2 Semi-automatic curation of metabolic pathways 3 Evaluation of inter-annotator agreement 4 Information extraction as a Web service 22

23 Use Case 1: Chemical NER Supplies gold standard corpus Removes golden annotations so that they can be created automatically Compares and reports precision, recall and F1 of the different branches against the gold standard corpus Combinations of syntactic and semantic components create annotations

24 Chemical Entity Recogniser Chemical model evaluated at BioCreative IV CHEMDNER challenge The challenge Data: 10,000 manually annotated PubMed abstracts Automatically recognises names of chemical entities in text 24

25 Chemical Entity Recogniser Our solution Ranked unique mentions: ranked 1 st out of 18 groups All mentions: ranked 3 rd out of 19 groups Subtask Precision % Recall % F-score % Ranked unique mentions All mentions

26 Use Case 2: Semi-automatic Curation Search for relevant documents NER for chemicals, genes, process indicators Metabolic Pathways Linking to ontologies: CTD, ChEBI, UniProt Save results in various formats, e.g., RDF for querying and incorporation into databases Manual correction of automatic annotations 26

27 Manual Annotation Editor Create, modify or delete annotations Edit details of annotations Create new annotations by selecting text Open a graphical interface to link annotations to ontologies 27

28 Filtering and converting annotations 28

29 Manual Annotation Editor: linking to Automatic preselection can be modified by the user ontologies Details show ontology entry webpage 29

30 Use Case 3: Information extraction as a Web service Web serviceenabled reader Web serviceenabled writer 34

31 Language Universal Reusable modules Generic TM modules: Competence Annotated Text, corpora: Performance Standards of Data Representation and Types for Resources: Competence Dictionaries, Thesauri, Ontologies: Performance 36