A Recipe for Data Fusion. Russell Neimy

Transcription

1 A Recipe for Data Fusion Russell Neimy

2 Introduction And Agenda Establish Governance Warehousing Vs. Federation Adopt Or Create An XML Framework Design & Execute Solution Strategy Design & Execute The Test Strategy Deployment & Training Maintenance Discussion

3 How Did We Get Here & Where Are We Going Law Enforcement Fractured By Design Adjudicate At The Local Level But RMS Data Is Unintentionally Fractured As Well Many Vendors, Systems and Law Enforcement Agencies Security of Data at Law Enforcement Sensitive and Secret Levels Lack Of Standards Sharing Is Hard Healthcare Fractured By Business Model Provide Patient Care At The Local Level But RMS Data Is Unintentionally Fractured As Well Many Vendors, Systems and Healthcare Practices Security of Data Compliance with HIPAA Lack Of Standards Sharing Is Hard

4 9/11 Catalyst LE Forced by Events to Respond Over the course of 3 years 700+ Agencies Using Over 50 Disparate Vendor RMS Solutions 3 Fusion Centers Share Information Across Jurisdictional Boundaries On a Nickel and a Prayer If Poor Old Law Enforcement Can Do It Healthcare Can Too

5 Establish Governance Understand The Fiefdoms Business Process Owners Have Legal Responsibilities IT Technical Process Owners Have Technical Stewardship Responsibilities Both Are Generally Resistant To The Exposure Data Sharing Can Manifest Identify And Involve The Owners Business & Technical Business For Funding, Resourcing And Conflict Resolution Technical For Access To And Support Of Data Integration Having Both Covers The Bases Create Authorized Oversight Executive Steering Committee Working Group with REAL USERS & IT Representation Empower Them And Give Equal Voting Rights Create a Memorandum Of Understanding

6 Sample MOU Background: Explains The Background Of The Project Such As From Who/How/Where The Idea Was Formulated Concept: Describes The Goals Of The Project Including Things Like Providing Connectivity, Hardware And Software Necessary For X Departments To Share Y Information Purpose: Defines Who The Agreement Encompasses: For Example This Agreement Is Between Agency X, Y And Z Responsibilities: Defines The Roles And Responsibilities Of All Business And Technical Data Source Owners Including Who Will Supply Resources, Create Systems And Protect Information. Stick To Just 1 Change For One And You End Up Changing For All Host MOU Meetings Provide Food Seriously What s In It For Them: Show Pro-type Of Outcomes Provide Testimonials EVANGILIZE Funding: Explains Funding Sources How Costs Will Be Shared Between Each Business Party. Acknowledgement: Secures Adherence To The MOU Through A Signature Block

7 Warehousing Vs. Federation Warehousing Is System Based Data Replication Federation Is User Initiated Access To Host Sources Warehousing Insulates System Of Record From Heavy Query Load Federation Can Accommodate Data Policy Or Legal Restrictions Choice Is Rarely Mutually Exclusive Both Lend Themselves Well To SOA Architectures

8 If A Standard Exists Use It Adopt/Create An Exchange Framework Law Enforcement National Information Exchange Model (NIEM) Banking & Finance Open And Interactive Financial Exchanges (OFX/IFX) Health Care Seems this is still being worked out (HL7, RHEx, NHIN,??) Law Enforcement uses and Event Drive Justice XML Model Based On National Information Exchange Model for First Responders Elements Include Person, Vehicle And Property Descriptions Records Types Include Incidents, Warrants And Arrests Etc. The Model Describes Both AND How Information Can Be Added, Modified, Deleted And Queried

9 Design & Execute Solution Strategy Three Factors That Impact Architecture Choice: Number Of Data Sources To Be Fused (Breadth) Number Of Elements Being Fused Across The Sources (Depth) Number Of Applications Used To Generate/Manipulate Data (Complexity) Service Oriented Architectures Principles: Loose Coupling: Creates Self Contained Services Designed To Access Data Abstraction: Insulates the Dependent Systems Reusability: Can be inventoried and reused Autonomy: Services Supports Themselves Regardless Of Dependent System Availability

10 Information Sharing SOA Architecture Data Collection Replicated Warehouse Intelligence Analysts

11 Testing Solution Validation Part 1 Cover All Major Test Cases For Add/Modify/Delete/Query Validate That The Fusion System & Services Work BEFORE Real Applications Point To The Solution Use Test Scripts To: Reset Database To Known Valid State Trigger Changes From Mock/Harness Applications Validate SOA Services Handle Requests Correctly At The Record And Element Level in the Data Repository

12 2 Testing Vendor Certification Part Use The REAL GUI(s) From Each Application s Source Data Repository More Interfaces = More Complex/Time-consuming Test Efforts Reuse Scripts From Harness Systems With Real Systems Validate That The Host System & Services Work With The Real Applications That Will Point To The Production Solution

13 Testing User Acceptance Part 3 Conduct With REAL Users Of Both: Upstream Contributing Applications New Downstream Consumer Applications Proves The Original Upstream Applications Remains Valid Proves The New Down Stream Warehouse And Apps Are Valid Garners Support And Commitment From End User Community Give Project Milestone Closure Signals Readiness For Production Deployment

14 Deployment & Training Instructor Led Or Computer Based Training We Provided Food That ALWAYS Helps! End User Documentation Is Crucial PDF Online Hard Copy Float Support Staff During Go Live Word Of Mouth Creates The Snowball Affect Usage Rates RISE

15 Maintenance Fusion Systems By Their Very Nature Are NOT Static They Are Intended To Grow Growth In Effect Equates To Success Maintenance Is Crucial Unplanned Interrupts Will Kill System Perception & Usage Develop Usage Statistics To Plan For Future Growth Plan AND Communicate Routine DB Optimization Execute Data Archival & Retention Develop Tools To Monitor & Detect Anomalies

16 Discussion