PIEE and the Data Lake

Transcription

1 DEFENSE PROCUREMENT AND ACQUISITION POLICY PROCURE-TO-PAY TRAINING SYMPOSIUM PIEE and the Data Lake Presented by: Adarryl Roberts Defense Logistics Agency Enterprise Sourcing, Medical, and Contingency Portfolio Manager May 30 June 1, 2017 Hyatt Regency Orlando FL 1

2 Agenda Procurement Integrated Enterprise Environment (PIEE) What is the PIEE? PIEE Current State Procurement Systems View Level 8 Architecture Data Lake What is the Data Lake? Solution Architecture Capabilities Overview Current Report List How are Reports Determined? What is Possible? How are Capabilities Designed? Gaining Access to Production 2017 Procure-to-Pay Training Symposium 2

3 What is the Procurement Integrated Enterprise Environment? An efficient effective hosting environment for functional procurement capabilities (e.g. WAWF, JCXS, CAGE, FEDMALL) PIEE will enable: Agile development and deployment Lower maintenance cost Standard and reduced system interfaces Ability to leverage data across applications Streamlined secure role based user access 2017 Procure-to-Pay Training Symposium 3

4 Procurement Integrated Enterprise Environment Current State Established a common hosting environment for enterprise applications and data (FY17) ATO granted for FEDMALL and Data Lake Next Steps: Enable enterprise role based access to environment Determine other cloud ready applications to move to the environment Ensure technology framework for EDA is current, efficient and effective 2017 Procure-to-Pay Training Symposium 4

5 EMMA RA CLS (2015) BI Extracts (2014) BI Reports (2014) PSD-OCC Selection Tool Purchase Card Program FBO CFDA FSRS / esrs WDOL FPDS PPIRS / CPARS / FAPIIS USASpending Treasury (2008) AIM DM DMDC System for Award Management (SAM) DPAP (2009) Common Services / APIs / Reports Mobile Apps Line Item Service Labor Code Crosswalk (2015) DPAP DLA GSA SAM - Entity Mgmt. (2012) Common Services Platform Future Phases: TBD Award Mgmt. Performance Information Wage Data Assistance Data DLA DoD Entity Identification PDS Validation (2008) DoDAAD CAGE (CONUS) Vendor Family Tree (2016) PRDS Validation (2010) WAWF ebusiness Suite DLA GEX Contract myinvoice Distribution Validation (2003) (2015) IRAPT (2003) Contract File (EDA, 2003) IUID / GFP Registry (2005) CORT Tool (2013) myinvoice (2015) D-C emipr workflow (2015) NCCS DD254 (2015) Automated Contract Closeout (2015) CDR (2016) Manual Closeout (2017) Automating Solicitations (2018) Data Lake Phase 1 (2017) Contract Administration Enterprise Solutions etools Navy MOCAS API MOCAS NG PPIRS-SR (Supplier Risk) (2016) IWMS JCCS Integrated Environment DCMA CBAR Component Managed Enterprise Solutions Vendor Vetting Opportunity Posting Contract Reporting Dollars & Sense CERP 3-in-1 (2010) TBC (2013) AGATRS (2014) JCOP GFLSV (2015) PDREP / W&S of Repair Solicitation / Proposal Management Purchase Request / MIPR Workflow Management Niche Contract Writing (20-40% of transactions) Workload / Workforce Management Business Intelligence Records Management BPR: Virtual File Mgmt, Info SLINs, ELINs, & Reconciliation Tools Catalogs / Online Malls EMALL GSA Advantage GSA Legend IBEX Merged DLA DLA casm Yet to be DLA In-Process Determined 5 FEDMALL (2017) DLA Bolted / Separate ly Built Module Fully operational

6 What is the Data Lake? A capability that ensures data from procurement enterprise systems is searchable agnostic to system operations (e.g. consolidate MRS instances across WAWF) Ensure data aggregation can be separate from operational systems Successful proof of concept completed in FY16 Extraction capability developed Data Lake is currently being populated with data from WAWF Limited initial access Verification and Validation underway Designed for government users Developing extracts for DFAS to support audit 2017 Procure-to-Pay Training Symposium 6

7 Data Lake Solution Architecture Built entirely on open source software, eliminating license and maintenance costs Scales linearly on demand to manage massive datasets Runs on commodity, reusable hardware Ingests, enriches, and relates structured and unstructured data sets Supports advanced distributed analytics to derive knowledge Enables teams to focus on developing a customer solution instead of worrying about the plumbing of data ingest, storage, and search Provides an integrated analytic engine to allow users to load and execute analytics tuned to their needs 2017 Procure-to-Pay Training Symposium 7

8 Data Lake Phase 1 Capabilities Overview Brings together data from various sources irapt EDA myinvoice CORT Tool emipr IUID Registry DCMA EDRMS and Contract Closeout Database WAWF CDR WAWF Contract Closeout Provides method of storing and indexing multiple data formats Uses Data migration Reports - Samples follow Contract Execution History Contract Closeout 2017 Procure-to-Pay Training Symposium 8

9 Data Lake Current Report List A list of completed Data Lake use cases available for users after initial deployment Notification of Contract File Disposal from Data Repository Contract File Disposal from Data Repository Query Data* (handful of reports within this capability) Export Query Results Save Queries Track IDIQ Contracts Awarded and Closed by Base Award and By Order Track Recent Contracts Awarded and Closed (Annual) Track Older Contracts Awarded and Closed (Annual) Generate Record of Destruction of Contract File Contract Execution History Report A list of highlighted use cases planned for future Data Lake development efforts Total of 23 planned future use cases Contract Purple File Repository CORT Submission of Status Reports to Contracting Officer Service Contract with CORs Appointed in DoD CORT Tool Training Requirements for CORs Appointed in DoD CORT Tool Contract Phonebook DCAA Contract Brief Capability Bulk Download and Highlight 2017 Procure-to-Pay Training Symposium 9

10 2017 Procure-to-Pay Training Symposium

11 Contract Execution History Detailed Data 2017 Procure-to-Pay Training Symposium

12 Contract Execution History Contract Data

13 Contract Execution History CLIN change by Mod

14 Contract Execution History irapt Data by CLIN

15 Procurement Instruments Awarded and Closed

16 What is possible with the Data Lake? Provides a robust view of a contract s full lifecycle Provide data analytic capabilities beyond traditional static reporting across multiple enterprise-level data sets Improved data accessibility, integration and in-depth analytical capability from various emerging data sources into a single data repository Substantial benefit of growing data variety and metadata for analysis and reporting Services associated with business function Various level of automated reporting solution across DoD, Service, and MAJCOM levels 2017 Procure-to-Pay Training Symposium 16

17 How are Lake capabilities designed? Step 1: An organization sponsors a Lake capability Step 2: A use case proposal is developed by the sponsor Step 3: The proposed capability is reviewed by the PBORG Step 4: If approved, a wireframe is developed Step 5: Priority and resources are determined Step 6: Development of new capability inserted into future sprint sessions Step 7: Submitter conducts testing and V&V of new capability Step 8: Capability is deployed for use to production users Sponsors may be required to resource the Lake capabilities 2017 Procure-to-Pay Training Symposium 17

18 How to Gain Access to Data Lake? Data Lake access is managed through WAWF and require approval from the Service/ Agency lead irapt GAMs Users must self register/ add a Data Lake role through WAWF There are three types of Data Lake roles: Standard User: Most users will have this role. It grants users access to all reports that do not have sensitive pre-award data Enhanced User: Provides users will all standard reports plus the ability to view select reports that contain pre-award data Note: there are no current reports that contain pre-award data Executive User: Provides users with all standard reports plus the ability to view enhanced query capability This is limited to the PMO user base 2017 Procure-to-Pay Training Symposium 18

19 DEFENSE PROCUREMENT AND ACQUISITION POLICY PROCURE PAY TRAINING TO SYMPOSIUM May 30 June 1, 2017 Hyatt Regency Orlando FL 19