Louisiana DOTD. Enterprise Data Warehouse

Transcription

1 2007 Transportation Engineering Conference Louisiana DOTD February 13 th, 2007 (Meeting Room 1) Louisiana DOTD Enterprise Data Warehouse Business Intelligence at DOTD

2 Overview Louisiana Department of Transportation and Development The Challenge Solution Area Business Case Source Systems Data Repository Reports and Queries Tools Summary

3 Louisiana DOTD The Louisiana Department of Transportation and Development (DOTD) develops, implements, and administers programs and projects that impact the State s highways, bridges, airports, waterways, rail and public transportation systems and administers an annual budget of over $2.3 billion. DOTD needs to analyze information to manage its complex activities, but finds that the data needed for analysis is located in different systems, stored in different formats, and often difficult to access.

4 Louisiana DOTD To meet this challenge, DOTD implemented technology to enable: A more feasible mechanism to retrieve, report, export, or perform analysis on adhoc queries A checkbook view that shows the status of financial transactions Improved decision support for cash management practices, such as scheduling of expenditures Ability to easily generate key reports and query results for decision-makers.

5 The Challenge Disparate data sources Operational data stored in very functional silos Difficult for users to access Little integration between financial and project systems Manual process to extract and integrate data for reports when data from multiple systems required Data is not clean

6 The Challenge Defined The Data Warehousing system must provide: A more feasible mechanism to retrieve, report, export, or perform analysis on ad-hoc queries of relevant data, updated on a frequent basis to meet our increased reporting requirements due to the hurricanes The Process Improvement Initiative (Team 1) on Performance Measurements requirement for an automated method to generate an easy-toretrieve and easy-to-understand representation of department performance on key indicators

7 What is a Data Warehouse? Historical data repository for reporting and analysis across time Combines data from multiple systems Builds on dimensional data model that facilitates analysis across different subject areas Extracts data from operational systems, does not change operational systems Transforms operational data into meaningful information Applies business rules to data and eliminates bad data Single version of the truth, one common data repository for all users

8 What is a Data Warehouse? Operational vs. Analytical Operational Functionally Oriented Disparate Time Specific Volatile Redundant Incompatible Analytical Subject Oriented Integrated Time Variant Non-volatile Detail Data

9 The Data Warehouse Architecture Separates Operational Systems and Analytical Systems Operational Environment Business Applications Operational Data Store ETL Enterprise Data Warehouse Report Queries Analytical Environment ETL ETL Data Flow

10 Solution Area Business Case Reporting around projects is very high priority Complete picture of a project Aggregate views with ability to drill to detail views Finances (expenses) Project costs Departmental Costs Cash management Outlays and funding Human Resources HR related to projects and finances

11 Solution Area Business Case Major Focus Areas Finance Projects Source Systems Eight systems Integrates data from major functional areas Satisfies needs identified during interviews

12 Source Systems SOURCE DESCRIPTION FOCUS AREA DAJR Daily Journal of Financial Transactions Finance SMGR/ESTI Construction Projects Projects/Finance PMFS Capital Outlay Financing TOPS Project Master File Project LETS Project Letting Schedule Project/Finance Project/Finance PCST Project Cost History Project/Finance FAID Federal Aid Finance

13 Data Repository Utilize existing DB2 installation on z/os Create two new databases Staging Data transformation and load activities Production End user production environment for reporting Managed by LaDOTD DBA s No additional software acquisition required

14 Reports and Queries Data Warehouse delivers reports designed to answer the questions defined and agreed upon in the business requirements phase. Those questions were: How was a project funded? For capital outlay funds, what are the budgeted, committed, encumbered, expended and balance remaining amounts? How much does/did a project cost? How were the funds spent? When were the funds spent?

15 Tools Reports were created and delivered using Information Builders Web Focus. Tool benefits include: Managed report caching and delivery Security mechanisms and end user management Integration with the Web Focus Dashboard Simplified administration Ease of use for end users Report delivery via common web browser No additional software acquisition costs were required

16 Tools continued For ETL processes, custom SQL language scripts are run via our scheduling system. The ETL process document details the exact order the scripts must be run in and any exception processes associated with the scripts. While custom scripts were not recommended for longterm data warehouse projects with high volumes of data, they would suffice for our needs at this time. No additional software acquisition costs were required.

17 On-Line Demo

18 Data Warehouse Welcome Page

19 Key Performance Indicators

20 Projects Bid When Scheduled

21 Projects Bid Within Estimate

22 Project Delivery Gauges

23 Projects Completed on Time

24 Projects Completed Less Than 110% of Bid

25 Construction Project Gauges

26 Traffic Fatalities on Louisiana Public Roads

27 Project Reports

28 Active Construction Projects

29 Active Construction Projects

30 Active Construction Projects

31 High Impact Projects

32 Unrecorded Liabilities

33 Prompted Report High Impact Projects

34 Prompted Report High Impact Projects

35 Prompted Report High Impact Projects

36 Prompted Report High Impact Projects

37 Prompted Report High Impact Projects

38 Fund Reports

39 Capital Outlay Fund Balances

40 Capital Outlay Fund Balances Summary

41 Operating Fund Balances

42 Operating Fund Balances Summary

43 Prompted Reports

44 Prompted Reports HTML or Excel

45 Prompted Reports - Excel

46 Prompted Reports Excel, in Browser

47 Prompted Reports Excel, Standalone

48 Prompted Reports HTML

49 Capital Outlay Funds Summary

50 Operating Funds Balance Drill Down

51 Operating Funds Balances by District

52 Budget Reports

53 Budget Status by Fund, District, and Financial Group

54 Budget Status by Fund, District, and Financial Group

55 Budget Status by Fund, District, and Financial Group

56 Budget Status by Fund, Program, and Financial Group

57 Hurricane Reports

58 Hurricane Reports

59 Hurricane Reports

60 Hurricane Reports

61 Hurricane Reports

62 Hurricane Reports

63 Help Pages

64 Future Plans In FY 2005, we worked with consultants in developing the base system; in FY2006, IT personnel continued adding to the architecture and adding additional data sources; in FY2007, we plan to use consultant assistance. Expansion areas to be addressed include implementing additional performance indicators, more detailed payroll information and including maintenance data sources. We will also plan to include additional operational data stores.

65 Summary The Data Warehouse project has provided the platform to facilitate the analysis and reporting requirements that occurred as a result of hurricanes and process improvement. Having a designated area (data warehouse) on the Intranet has eliminated the where do I find it search for DOTD employees. With regard to timing, the Data Warehouse was able to address several high priority needs and heightened awareness for continued expansion. The demands for data continue to increase while the tolerance to wait for results continues to dwindle.

66 Questions