REQUEST FOR INFORMATION FOR Data Quality System RFI 2012-24 Florida Department of Education 325 West Gaines Street Tallahassee, Florida 32399-0400 Please email submissions to: Florida Department of Education Attn: Christina Davis Email: Christina.Davis@fldoe.org
Florida Department of Education Data Quality System Software Purpose and Scope I. INTRODUCTION The Division of Accountability, Research, and Measurement (ARM) within the Florida Department of Education (FDOE) provides education information to inform policy and support high standards that will support continually improving student achievement and opportunities throughout Florida s PK-20 education system. ARM is responsible for maintaining the Statewide Longitudinal Data System (SLDS) with PK20 Education Data Warehouse (EDW) as its key component. The mission of the EDW is to provide stakeholders in public education including, but not limited to, administrators, educators, parents, students, state leadership, and professional organizations with the capability of receiving timely, efficient, and consistent responses to inquiries regarding Florida's pre-kindergarten through University education. The EDW is capable of tracking students over time and across delivery systems, performing trend analysis, querying summarized data, and providing policy makers with tools and information to make informed, data-driven decisions about education. II. BACKGROUND With input to the EDW from multiple sources with varying formats, the quality and consistency of data stored in the EDW becomes critical. Presently, over one hundred data providers are required to perform data cleanup based on data quality specifications. Due to the large number of source systems involved, it is difficult to maintain consistent, high quality data. As part of the modernization of the EDW, a conceptual stand-alone system termed the Data Quality (DQ) system is planned as shown in Figure-1. This system will have the necessary software components to collectively ensure high quality and consistency of data before it is loaded into the EDW. These components will have features to enable FDOE to setup quality standards and validation rules. They will also permit setting up necessary feedback to data providers via the chosen mechanisms. Since the process is centralized, it will ensure consistent quality of data. This Request for Information is for software that validates data before it is loaded to the DQ System that is a commercial off the shelf tool customized for FDOE needs, designed from the ground up for FDOE, or a combination of both. Page 2 of 7
Figure-1: Data quality system as a front end system to data warehouse. III. GOALS The primary goal of implementing an enterprise-wide DQ system is to establish a centralized process to perform data validations at FDOE and/or at the data provider end and enable FDOE to perform additional validations. The DQ system should also permit timely feedback to data providers so that they can iteratively make corrections to their data to be certified for loading into EDW. IV. CONTEXT The DQ system is intended as an independent software unit which will ensure incoming data is compliant with the standards of data quality established by the FDOE. This system is intended to cover all processes from the time data is received from Data Providers until it is ready to be loaded into the EDW. It is comprised of several integrated software functional components listed below. Interaction between the components is as shown in Figure-2 below. Each component is a commercial off-the-shelf tool customized for FDOE needs, designed from the ground up for FDOE, or a combination of both. Typical software components of a Data Quality system are: Data Quality Tool Relational Database Management System (RDBMS) System Interfaces User Interfaces Pre-processors Reporting components Page 3 of 7
Portal Job Scheduler 4. Web User Interface External Source Systems Submit Data FLDOE BPOs Setup DQ Rules 7. FLDOE Portal / Web Page 3. System Interface 4. Web User Interface Communication (email ) 2. RDBMS 1. Data Quality Tool DQ Rule Base DQ Validation Results 6. Reporting Component DQ Staging ODS Data Conditioning 5. Pre-processors (Custom) 8. System Scheduler Figure-2: Interaction between software components of Data Quality system. Typical steps involved in the data quality process are: Receive data from Data Providers Convert the data received from various sources into common format Run data quality validations at different levels Communicate exceptions to external Data Providers Certify data ready to be loaded to EDW V. FUNCTIONAL REQUIREMENTS Functions served by each component of the Data Quality system are: 1. Data Quality Tool: This is the central component of the DQ system which will support: 1.1. Rule Engine which performs quality checks using the defined rules and workflows 1.2. Interfaces for business process owners to define 1.2.1. data quality rules 1.2.2. work flow for sequencing validation rules 1.2.3. work flow for handling exceptions 1.2.4. flow based on result of validation 1.2.5. exception reporting 1.3. Creation of a standalone data quality module that is platform independent and that can be implemented at individual remote Data Providers 1.4. Support audit and tracking of quality validations 2. RDBMS: is used for defining and maintaining the tables where the data from the Data Providers is staged for use by the data quality tool. 3. System interfaces: Used by the Data Providers to submit the data to the data quality system in small and large volumes 4. User Interfaces: Web based user interfaces for 4.1. BPOs to define data quality rules 4.2. Data Providers to review DQ process results, and, 4.3. Manual certification of data 5. Pre-processors: Included or custom developed processes to - 5.1. Validate data transmissions and process acknowledgements 5.2. Move data between hardware elements (FTP to App server etc) 5.3. Web services to receive data from Data Providers Page 4 of 7
5.4. Processes to identify data format and convert to standard format for processing by Data Quality system 5.5. Save the data in intermediate staging tables 6. Reporting components: 6.1. For linking exception reports from DQ tool with data received from the Data Providers 6.2. Generating and bursting exception reports for LSS 6.3. Sending communications to LSS. 7. Portal: Presentation Layer for hosting DQ Validation reports for access by LSS and FDOE staff 8. Job Scheduler: System Job Scheduler to facilitate setting up of at least the following types of data quality jobs 8.1. Event triggered 8.2. Time triggered 8.3. Manually triggered VI. SCOPE OF RFI The scope of this RFI is for vendors to provide information on their Data Quality tool and how they would enable the FDOE to fulfill its goals as stated in the GOALS section above (Section III) and as elaborated in the FUNCTIONAL REQUIREMENTS section also above (Section V). Vendors can also provide information on other supporting software components to the data quality tool (as listed in CONTEXT section above (Section IV) in addition to the tool itself for consideration by FDOE at its discretion. FDOE intends to integrate the necessary components required to enable functioning of the DQ system as a whole if these supporting tools are not part of the offered product or product suite. VII. PROCESS ARM will form a work group comprised of program personnel and other ARM stakeholders to review and analyze information received from this Request for Information (RFI) to determine the feasibility of issuing a competitive solicitation to acquire these systems. Any request for cost information is only to gain a perspective of the potential budgetary magnitude and is not binding. Responses to this request will be reviewed for informational purposes only and will not result in the award of a contract. Vendors submitting answers to the RFI are not prohibited from responding to any related subsequent solicitation. VI. RESPONSE FORMAT Potential vendors are asked to address all the following requirements in their response: a. Explain clearly how each objective listed above will be met by the product being proposed. b. Company literature, such as corporate overview, describing company structure, financial soundness and stability, software specifications, and services provided, experience, and upgrade capability. c. General product information regarding data structure and formatting, ease of use and functionality, system documentation and user help, system security and technical support. d. Detailed product information about the Data Quality system and its features which may not be publicly available on the Company s website e. Provide server, hardware, licensure, and any other infrastructure requirements. Hardware specifications should indicate how many concurrent users can be accommodated by the recommended infrastructure as well as the anticipated average storage requirements f. Provide an estimated of the average bandwidth that would be required to support submission of about 30 million records. g. Provide a description of how security will be implemented and administered to limit access to the system. Page 5 of 7
h. Provide a basis of cost of software and services requested, and any time constraints in terms of staff availability for setup and technical support. Basis of cost to include but not limited to: 1. Licensing: a. Licensing Unit Cost b. Licensing Unit of Measure (e.g. per user, per server, enterprise, etc.) c. Licensing Period 2. Installation and Configuration: a. Installation and Configuration Cost (if included in licensing cost, please mention) b. Additional Hours Rate 3. Maintenance: a. One-time Cost b. Recurring Cost (indicate Period and Rate/Period) i. Provide basis of cost for ARM staff training and customer support. Basis of cost to include but not limited to: 1. Technical Support: 2. Training: a. How many hours of Support included in Licensing cost b. Rate for Additional hours a. How many hours of Training included in Licensing cost b. Type of Training provided (e.g. User, Administrator, etc.) c. Venue for Training (e.g. on-site, off-site, etc.) j. Provide information or suggestions regarding necessary requirements. k. Human resource requirements for on-going maintenance and administration. l. Provide product information regarding compliance with Section 508 of the Rehabilitation Act of 1973 and Sections 282.601 282.606, Florida Statutes. m. Provide information and pricing structure on software updates, customer support, and maintenance services. VII. Division of Accountability, Research, and Measurement Website: Additional information about the Division of Accountability, Research, and Measurement can be found at: http://www.fldoe.org/arm/. SCHEDULE OF EVENTS A. Procurement Time Schedule The following timetable shows the approximate dates for this procurement. All times indicated are Eastern Time (ET). Page 6 of 7
Request for Information Issued 12/05/2011 Questions Due to no later than 12/08/2011 by 2:00 p.m. Answers to Vendors on or before 12/15/2011 Receipt of e-mailed RFI responses 12/22/2011 by 3:00 p.m. PLEASE PROVIDE RESPONSES VIA EMAIL B. Questions And Restrictions The Department of Education may be contacted via email or fax regarding the submission of questions concerning this RFI. Any respondent s questions must be submitted in writing and received by the Department on or before the specified due date at the following email address or fax: Please deliver questions to: Florida Department of Education Attn: Christina Davis 325 W. Gaines Street, Suite 332 Tallahassee, Fl 32399 Fax number: 850-245-9189 Telephone number: 850-245-9191 Email: Christina.Davis@fldoe.org The Department will provide written answers to all questions that respondents submit by the specified due date. Questions and Answers and notice of changes (addenda), will be posted on the Florida Vendor Bid System (VBS) at www.myflorida.com (click on Business & Industry, under Doing Business with the State of Florida click on State Purchasing, click on Everything for Vendors and Customers, then Vendor Bid System and Search Advertisement, select the Department of Education in the Agency window and initiate search), under this RFI number. It is the responsibility of all respondents to monitor this site for any changing information prior to submitting a response. Page 7 of 7