Dow Jones Industry, Region and Subject Taxonomy

Transcription

1 Dow Jones Industry, Region and Subject Taxonomy Features, Benefits and Application Dow Jones provides this taxonomy of industry, region, subject and company/organisation codes under the name Dow Jones Intelligent Indexing. This document describes the taxonomy itself, its features and its benefits. 1 Introduction Background to Dow Jones Intelligent Indexing General description of the Dow Jones Industry, Region and Subject Taxonomy Structure of Dow Jones Intelligent Indexing Taxonomy category structure Industry taxonomy tree Region taxonomy tree Subject taxonomy tree Maintenance of Dow Jones Intelligent Indexing Customer benefits of using the DJII taxonomy Licensing terms Implementing support Code Application Major news elements of an article Coding tools..7 6 Quality Monitoring..8 This document contains information proprietary to Dow Jones and may not be reproduced, disclosed or used in whole or part without the express written permission of Dow Jones. Copyright Dow Jones 2015 Page 1 of 8

2 1 Introduction Dow Jones Intelligent Indexing is the name for the content classification scheme used by Dow Jones to classify the business information in the Dow Jones Factiva products and services. Dow Jones Intelligent Indexing is used by more than 1.5 million subscribers to Dow Jones services world-wide. Dow Jones Intelligent Indexing is made up of two parts: the Dow Jones Industry, Region and Subject Taxonomy and the Dow Jones Companies/Organisations. This document describes the first of these two parts. 1.1 Background to Dow Jones Intelligent Indexing Dow Jones Intelligent Indexing is the result of many years of real-life design and application experience. Inputs to the first version, on the foundation of Factiva in 1999, included: interviews with over 300 customer and non-company organizations around the world; the Reuters Business Briefing and Dow Jones Interactive taxonomies; evaluation of public-sector and competitor taxonomies; feedback and suggestions from a cross-functional team of internal content, technology, editorial, marketing and customer support experts. 2 General description of the Dow Jones Industry, Region and Subject Taxonomy The Dow Jones Intelligent Indexing taxonomy covers over 3000 main terms grouped in to three category trees under the following main headings: industry terms: a structured hierarchy of 17 top-level business sectors such as financial services, technology or industrial goods, drilling down to detailed terms such as online banking, internet telephony or abrasive products region terms: all countries of the world and other major newsworthy territories and sub-regions, with more than 30 supra-national groupings such as North America, Eastern Europe, Asia and Latin America and sub-regional terms for a number of countries such as the U.S., Canada, Australia, China, Japan, India, German and more as well as major cities worldwide subject terms: a highly structured list grouped around corporate, economic, market, sports, general and political news. Also a list of content types such as indepth analysis, rankings, chronologies, transcripts and interviews. Dow Jones Intelligent Indexing also includes over 380,000 company codes and 18,000 organisation codes which are made available within the Dow Jones Company service. Page 2 of 8

3 2.1 Structure of Dow Jones Intelligent Indexing Each of the main category trees in the Dow Jones Industry, Region and Subject Taxonomy has its own hierarchy with parent-child relationships, similar to those in a family tree. The depth and width of the taxonomy varies according to the category tree: Industries 17 level-one codes. Up to 8 levels deep Subjects 7 level-one codes. Up to 8 levels deep Regions 16 level-one codes. Up to 6 levels deep The structure of the DJII taxonomy is designed to be fully hierarchically inclusive. So where a lower-level term is assigned, all higher-level terms in the same tree should also be assigned. A search on a top-level term will therefore automatically retrieve all lower-level terms in the same tree, unless the search strategy is constructed specifically to exclude the lower-level terms. Please note that categories may appear in multiple locations in the tree structure. The purpose of this is to ensure that users can locate the category easily, even though they may be looking in more than one place in the tree. For example, the subject code for Climate Change, GCLIMT has two parent codes assigned GGLOBE and GENV. This means that in a display of the taxonomy hierarchy, the term Climate Change can be found beneath the term Global/World Issues and under the term Environment. The same also applies with industry codes. So for example the industry code for Online/Direct Banking, i81404, appears under both Banking and Internet/Online Services. Similarly, the region code EGYPT, for that country, resides in more than one place in the region hierarchy. Note that in this case the code appears at different levels under different branches of the taxonomy. So Egypt appears at level 3 in the African Countries branch and at level 2 under the Middle Eastern Countries branch. Dow Jones refers to this multiple parent structure as polyhierarchy. 2.2 Taxonomy category structure Industry taxonomy tree The industry tree of the taxonomy currently consists of over industry terms grouped around a structured hierarchy of 17 top-level industry groups, with up to eight levels. The 17 top levels and the hierarchical structure are proprietary to Dow Jones Factiva and were established based on extensive research among Dow Jones Factiva s global business information user base. While public-standard industry taxonomies such as the UK and U.S. SIC and NAICS were examined, user feedback was that no such scheme provided the organizational structure, level of detail and rate of update appropriate to the needs of the customer base. Page 3 of 8

4 2.2.2 Region taxonomy tree The Region tree of the DJII taxonomy currently consists of over region terms, covering all countries of the world and other major newsworthy territories, with more than 30 supranational groupings such as North America, Eastern Europe, Asia and Latin America. Dow Jones codes also exist for major cities world-wide and sub-regions and territories of some countries, such as the U.S., Canada, Australia, Russia, UK, Germany, France, Spain, India, etc. There are 16 top-level Region codes and the tree is up to 6 levels deep. The list of countries/areas for which codes exist is broadly based on the ISO (International Organization for Standardization) 3166 countries standard, with some additions for other territories based on persistent newsworthiness, such as the Palestinian Territories Subject taxonomy tree The Subject tree of the DJII taxonomy currently consists of over 1000 subject terms, in a highly structured list grouped around corporate, economic, market, general and political news. The tree also contains a list of content types such as in-depth analysis, rankings, chronologies, transcripts and interviews. There are seven top-level Subject codes and the tree is up to eight levels deep. The following is a detailed description of the seven sub-sections of the Subject tree. Commodity/Financial Market News This sub-section covers all information about commodities and financial instruments traded on markets. Content Types This sub-section covers information that has a distinct format, such as a table, or is of a particular generic type, such as a chronology, analysis, ranking, or interview. Some categories in this sub-section are often employed as filters. This means they are added to search queries prefixed by an exclusion command such as NOT in order to eliminate certain kinds of material from search results. Some frequently used examples are Routine General News, NRGN, to exclude items such as sports/recreation or weather news, food/cooking items or obituaries, or Routine Market/Financial News, NRMF, to exclude regular updates from financial and commodities markets. Corporate/Industrial News This sub-section covers general corporate-industrial subjects. Economic News This sub-section covers national, regional and local economic information. Selection of Top Stories/Trends/Analysis This sub-section covers trend stories, profiles, analysis and commentary that provide valuable overviews or unique insight about vertical markets. Political/General News This sub-section covers political and general subjects. Page 4 of 8

5 Sports/Recreation This sub-section covers different types of sports. 3 Maintenance of Dow Jones Intelligent Indexing The taxonomy continues to develop through a structured review process taking place every two months in line with the demands of Dow Jones customers, the evolution of Dow Jones content set, new Dow Jones products and the changing business information landscape. Input is welcome at all times and Dow Jones greatly values its continuing dialogue with users of the taxonomy. The Metadata team is responsible for adding new categories, maintaining and modifying existing categories, and providing alternative terms and language translations in accordance with user needs. This is done through continuous consultation with a large number of Dow Jones staff around the world and draws on customer feedback, content coverage, user search metrics and analysis of accuracy of application. These inputs are complemented by direct customer research, independent analysis by external experts, and monitoring of public-sector and commercial taxonomies, public events and business information developments, in order to ensure the taxonomy continues to reflect the evolving business world. Main criteria for adding a new code: sufficient client demand fit in existing taxonomy hierarchy and policy concept should be clearly definable as being distinct from existing codes concept should not be already searchable by simple use of current codes. concept should be universal and not restricted to a particular content set or geography. If you would like to suggest a new code or comment on the existing taxonomy, please contact Customer Support. To keep up to date with the new codes and code changes introduced into DJII please subscribe to FAQ Customer benefits of using the Dow Jones Industry, Region and Subject taxonomy Dow Jones Intelligent Indexing taxonomy is applied all business news content, in all formats, in all languages, in our business news information services. By applying Dow Jones Intelligent Indexing codes to all content, Dow Jones products can deliver only the most relevant content to users, saving them time and helping them quickly find the answers they are looking for. All content is universally indexed with company, industry, geographical and subject codes from Dow Jones Intelligent Indexing: Page 5 of 8

6 380,000 company codes 18,000 organisation codes industry codes subject codes region codes It is possible to search or browse Dow Jones Intelligent Indexing lists, and add them to your search request. By using the Dow Jones codes as building blocks searches will return with accurate and focused results. Other benefits are listed below: Keyword search challenges Spelling errors Synonyms and different spelling variants Different languages Cannot browse through concepts Different meanings of terms, ambiguity Retrieves articles with any mention of keyword Complicated free-text search in complex queries Solutions by using DJII codes Controlling spelling errors Covering synonyms and spelling variations Searches in one / many languages at the same time Browsable hierarchy can be explored Context for terms with more than one meaning Retrieves relevant content Multiple concepts easily combined/excluded in a query (industry, region, subject, companies, organisations) Unstructured data is believed to account for 80 to 85 percent of a typical organisation s internal information. Unless the data is categorized with standard taxonomy terms, it can be extremely difficult to find this precise information needed to make effective business decisions. Dow Jones Intelligent Indexing allows you to create seamless pathways for accessing documents or reports originating from multiple repositories. Customers licensing DJII benefit from all the experience and effort which have gone into its creation, and which continue to go into its ongoing maintenance and development. 4.1 Licensing terms Users of the taxonomy must first accept the terms of a licence agreement governing terms of use and update procedures. Page 6 of 8

7 Use of the taxonomy within Dow Jones services, solely to retrieve Dow Jones Factiva content, is included within the standard Dow Jones Subscription Agreement. Depending on other purposes of use, annual licence fees may be payable. Where the taxonomy is used to retrieve non-dow Jones Factiva as well as Dow Jones Factiva data, and/or in a commercial product or service, licence fees will be due. Please contact your Dow Jones representative for further details. 4.2 Implementation support When integrating the DJII taxonomy into your information environment, you may require advice and practical assistance. To help customers through the taxonomy deployment process, Dow Jones has developed a simple support framework covering strategy, licensing and implementation services. The services are modular and can be employed in any combination. This modular approach allows organizations at any point in their taxonomy development to benefit from Dow Jones experience and expertise. Dow Jones staff will be happy to offer assistance when required. For more information, please contact your Dow Jones representative. 5 Code Application The underlying principle of code application of Dow Jones Intelligent Indexing is aboutness coding, also known as high-relevance coding or high-importance-level coding, i.e. the function of our coding systems is the categorization of information based on what it is about. 5.1 Major news elements of an article the subject, or focus, of the article (subject codes) the location / regional context of the main events (region codes) major company / organization references: article focusing on a company / organization or where that entity features prominently (company / organization codes) the industry sectors to which the story relates, for corporate/industrial news (industry codes) the way in which the content is presented or organized (subject codes: content types) 5.2 Coding tools Dow Jones Intelligent Indexing is applied to Dow Jones content using a combination of different automatic categorization tools. This ensures coding is timely and consistent across all content and languages. These tools can be rule-based or linguistic analysis software programs, set up to determine the main news elements (see Major news elements of an article) Page 7 of 8

8 The primary system used to apply company coding is known as ThingFinder. It works by analyzing a predetermined amount of text and extracting those text strings which it identifies as company names. It then compares each text string found to a catalogue of company names and associated Dow Jones Company Codes. If an exact match is found, the relevant code is generated. The primary system used to apply region, subject and industry coding is known as Categorizer. It s based on linguistic analysis technologies, which determine the correct coding by comparing new documents with a collection of thousands of correctly indexed documents put together by Dow Jones editors. The system calculates a confidence score for incoming articles and classifies them into the appropriate category, based on the degree of similarity to the articles contained in the training sets, in terms of vocabulary and sentence structure. In addition, Dow Jones uses a system called Rules Based Coding (RBC), which complements the above autocoding tools. For example, where two companies, with two separate codes, have the same name, ThingFinder will not be able to determine which one is the required code. Instead, a rule can be written, incorporating supporting evidence for one code or the other, which will ensure that the correct code is selected and applied. Rules are also created to deal with items consistently repeated in content such as market reports or sports stories. RBC can apply company, industry, subject and region codes. Rules range from being simple to very complex and can combine input provider metadata with textual evidence present in the article. 6 Quality Monitoring The Content Quality group evaluates code application and reports internally on performance on a regular basis indicating which areas are susceptible of improvement. Coding must be consistent across all coding streams and codes must be applied accurately all codes on an article should be relevant and all relevant codes should be present. These two principles are reflected in precision and recall scoring. Page 8 of 8