KNOWLEDGE ARCHITECTURE AND BIG DATA

Transcription

1 KNOWLEDGE ARCHITECTURE AND BIG DATA How to Apply Knowledge Architecture to Big Data Federal Reserve June 15, 2016 David Meza Chief Knowledge Architect NASA Johnson Space Center 2015 IHS. ALL RIGHTS RESERVED.

2 AGENDA Knowledge Architecture NASA Data Strategy CogniPve CompuPng 2

3 The most important contribupon management needs to make in the 21st Century is to increase the producpvity of knowledge work and the knowledge worker. PETER F. DRUCKER, 1999

4 To convert data to knowledge a convergence of Knowledge Management, InformaPon Architecture and Data Science is necessary. Knowledge Management InformaPon Architecture Data Science 4

5 Knowledge Architecture The people, processes, and technology of designing, implemenpng, and applying the intellectual infrastructure of organizapons. What is an intellectual infrastructure? The set of acpvipes to create, capture, organize, analyze, visualize, present, and uplize the informapon part of the informapon age.. InformaPon + Contexts = Knowledge InformaPon Architecture + Knowledge Management + Data Science = Knowledge Architecture KM without applicapons is empty (Strategy Only) ApplicaPons without KA are blind (IT based KM) Data Science transform your data to knowledge 5

6 Knowledge Management "Knowledge management is the process of capturing, distribupng, and effecpvely using knowledge. This definipon has the virtue of being simple, stark, and to the point. A few years later, the Gartner Group created another second definipon of KM, which is perhaps the most frequently cited one (Duhon, 1998): "Knowledge management is a discipline that promotes an integrated approach to idenpfying, capturing, evaluapng, retrieving, and sharing all of an enterprise's informapon assets. These assets may include databases, documents, policies, procedures, and previously un-captured experpse and experience in individual workers. 6

7 InformaPon Architecture The intent is to achieve a variety of capabilipes to enable the Agency to efficiently acquire or generate, find and access, use and reuse, share and exchange, manage and govern, and store and repre our data. 7

8 Data Science Data science is an interdisciplinary field about processes and systems to extract knowledge or insights from data in various forms, either structured or unstructured, which is a conpnuapon of some of the data analysis fields such as stapspcs, data mining, and predicpve analypcs, similar to Knowledge Discovery in Databases (KDD). The Knowledge Discovery in Databases (KDD) process is commonly defined with the stages: (1) SelecPon (2) Pre-processing (3) TransformaPon (4) Data Mining (5) InterpretaPon/EvaluaPon. 8

9 Data Strategy Key Recommendations : Data Management Unified Data Lifecycle Data Governance Data Analytics Lab Data Fellows Program Data Stewards 9

10 Data Strategy Framework Challenge Example Opportunity RecommendaEon Lack of an explicit data management framework, fragmented data lifecycle and lack of data integrapon Need for new emerging data analypcs technologies and capabilipes to address mission specific challenges Data experpse gap No Agency-wide architecture and standards for informapon interoperability. Much of the data NASA produces is inaccessible or human-readable only, with no method to draw-in, parse, organize, or make use of this data. Many of NASA s current data systems are significantly outdated and cannot scale to meet demand. Data scienpsts are in low supply and high demand, and NASA will need to compete with industry to apract the best & brightest. Improved architecture, standards and accessibility permimng quicker and more effecpve collecpon, digipzapon and discovery; increased focus on mission-specific data needs and type-specific approaches ExperimenPng with new algorithms, applicapons, and techniques CollaboraPve partnerships to build internal capacity and experpse and uplize external talent, tools, and informapon 1. Data Management 2. Unified Data Lifecycle 3. Data Governance Program 4. Data AnalyPcs Lab 5. Data Fellows Program Need to effecpvely address culture and policy issues alongside technology In many cases, individuals are not mopvated to share data for collaborapve use with others. Increased cross-agency and crossstakeholder ownership and approach to data management and data analypcs challenges 6. Data Stewards 10

11 KNOWLEDGE ARCHITECTURE ANALYTICS FRAMEWORK Source Data AcquisiPon & CreaPon Data Management Data Warehousing Data AnalyPcs, BI (Knowledge ExtracPon) Knowledge PresentaPon and VisualizaPon User Sources: Sensor Experimental Computed (modeling & simulapon) Forms: Digital Text Visual OrganizaPon: Structured Semi-Structured Unstructured FuncPons: Governance Taxonomy Ontology Comm. Plan OperaPons Management Security Master Data Management Content Management Metadata Data Quality Tools & Environments: Large scale storage RDBMS Parallel RDBMS NOSQL Hadoop OrganizaPon: Structured Semi-Structured Unstructured Tools & Environments: ComputaPon & data access Data Mining Text Mining OpPmizaPon Net Algorithm New Algorithm VisualizaPon Access Papern: Structured Semi-Structured Unstructured Predictable Unpredictable Data Products: PredicPons Models VisualizaPons Decision Analysis Wiki Security, Data Quality, Workflow Management, Data Management, Resource Management IT & Intellectual Infrastructure 11

12 We have an opportunity for everyone in the world to have access to all the world s informapon. This has never before been possible. Why is ubiquitous informapon so profound? It is a tremendous equalizer. InformaPon is power. ERIC SCHMIDT (FORMER CEO OF GOOGLE)

13 46% Workers can t find the informapon they need almost half the Pme. Source: IDC 30% of total R&D spend is wasted duplicapng research and work previously done. Source: Na+onal Board of Patents and Registra+on (PRH), WIPO, IFA 54% of decisions are made with incomplete, inconsistent and inadequate informapon Source: InfoCentric Research

14 Knowledge Architecture: The Next Phase 14

15 15

16 16

17 Push versus Pull 17

18 WHAT COULD YOU ACCOMPLISH IF YOU COULD: Empower faster and more informed decision-making Leverage lessons of the past to minimize waste, rework, re-invenpon and redundancy Reduce the learning curve for new employees Enhance and extend exispng content and document management systems 18

19 JSC Knowledge Architecture Services: AnalyPcs Web NOSQL VisualizaPon Repository Wiki Code Training Plauorm for Analysis and VisualizaPon - Neo4j and MongoDB Services - Business Intelligence Specific Search Farm Sharing and Project collaborapon 19

20 Contact Information David Meza Twitter Linkedin - hpps:// Github davidmeza1 Blog davidmeza1.github.io 20

21 Contents QUESTIONS? Report Name / Month IHS. ALL RIGHTS RESERVED. 21