Five-Tier Data. Warehouse. Architecture For. South African. Government. Researchjournali s Journal of Information Technology

Transcription

1 1 Five-Tier Data Warehouse Architecture For South African Government Edzai Kademeteme Department of Informatics, TUT, Soshanguve, South Africa Billy Mathias Kalema (PhD) Department of Informatics, TUT, Soshanguve, South Africa Pieter Pretorius Department of Informatics, TUT, Soshanguve, South Africa

2 2 ABSTRACT Due to ever demanding decisions that needs to be made right on time technologies that enhance decision making have evolved. These technologies have evolved to integrate heterogeneous information sources for analysis purposes. Transactional databases (Information sources) are progressively becoming autonomous and they change their content rapidly due to continuous transactions (data changes) and may change their structure due to continual users' requirements evolving (schema changes). Using such data sources for decision making becomes strenuous and nearly impossible. Governments all over the world are faced with challenging decision making which is the very livelihood to their very existence. For almost a decade now, discussion and controversy is ongoing with relation to the topic, which is the best DW architecture. Bill Inmon and Ralph Kimball being the pioneers of data warehousing, are at the heart of the disagreement. Inmon advocates the hub-and-spoke architecture, while Kimball promotes the data mart bus architecture. However, as research advances not only this architecture proposed by Inmon and Kimball exist but rather more are emerging. Other researchers developed other architectures for specific environment for example the two tier architecture and the three tier architecture. In this research work, we propose a five tier architecture for the South African government. There is a need to improve South Africa government information inventory system and hence decision making in order to make right on time decisions and to adapt to the ever changing and competitive environment. This proposed architecture will increase efficiency and effectiveness in maintaining the cycle of activities, in their planning, decision-making processes, and analytical needs. Keywords: user resistance, user adaptation, individual performance, age difference 1. INTRODUCTION Lately, data mining is a becoming a bubbling research field where innovations find their ways quickly into industrial products. Many companies and other organizations have already gained beneficial insights into data which helped them to improve their business. High performance is a key factor for data mining systems in order to allow a sufficient exploration of data and to cope with the huge amounts of data that an organization has accumulated over the years. High performance of a data mining system is not only a question of choosing the proper algorithms and efficient implementations of them but also the architecture which has a remarkable impact on performance. And for one to manage data efficiently and effectively there is a need for a DW. Data warehousing (DW) systems are used as decision support systems to help enterprises in strategic and intelligent business decision making. However, initially DWs where used as an enterprise repository to upkeep top management in business decision making in an efficient manner[1, 2]. Data warehousing can also This work was supported in part by the Department of informatics, Tshwane University of Technology, South Africa. (I, therefore acknowledge the financial support of informatics department, TUT).

3 3 be defined as logical collection of information gathered from many different operational data sources used to create business intelligence that supports business analysis activities and decision making tasks. It provides the basic infrastructure for decision making by extracting, transforming, cleansing and loading huge amount of data of the enterprise. This classic definition of the DW focuses on data storage. However, the means to retrieve and analyses of data, to extract, transform and load data and to manage dictionary data are also considered essential components of a data warehousing system. DWs support business decisions by gathering, combining and shaping of data for reporting and future analysis with tools such as online analytical processing (OLAP) and data mining. The size of DW fluctuates from hundreds of gigabytes to terabytes. Different scans, joins and aggregates are performed while querying the DW. The queries on DW are ad hoc and multiple faced. Throughput of query determines the success of data warehousing project. The query response time is also important factor in DW success. The allocation of facts and dimensions in a certain schema also effect query success. Several data warehousing architectures have been proposed in literature i.e. one tier architecture whereby a system is completely client based. Basically all data mining systems of the first generation are based on this architecture. The user has to select a small subset of DW data and load it on the client in order to make it accessible to the data mining tool[3]. The second architecture is the two tier architecture, in a two-tier architecture the data mining tool completely resides on the client but there is no need to copy data to it in advance. The data mining application may choose to load parts of the data during different stages of the mining computation[4]. The last architecture is the three tier architecture; which addresses the problems remaining with two-tier architecture. This is achieved by an additional layer holding the data access services and parts of the data mining services[5]. Data mining services may also be present on the client. Where part of the data mining services should be client based depends on data mining techniques and algorithms. South Africa is one of the leading developing countries in the African continent as well as the world at large. Its economy is growing as fast as other countries like Brazil, USA, and China to mention but few. As its GDP is growing South African government is faced with a lot of decisions that they need to make at the right time. This can only be possible if the SA government starts using the data warehousing in their decision making. As it stands the SA government relies on operational databases in order to make decisions. This is time consuming since data in an operational database is not summarized. On top of that an operational database consists of a large number of tables that are highly normalized for the sake of reducing data redundancy. However a DW provides with a phenomenal concepts of the tables being lowly normalized as compared to the operational database. This gives a DW an advantage that queries are quickly executed as they are being processed on few tables as compared to an operational database where queries have to be executed on so many tables and this will consume time.

4 4 SA is made of 44 government departments each is meant for its purpose as a ministry. Of the 44 this paper is going to illustrate using the department of Rural and Land Reform (DRDLR). The DRDLR is responsible for the leasing out of state land to organisations, citizens, and other government departments. It has corporate service departments within the department and these are; human resource personnel, finance, salaries, communications, legal services, supply chain, labour relations, public relations, transport and logistics, performance management and IT. These corporate service departments use internal data generated from each other and external data from other departments. The corporate service departments go through tedious processes of getting the required information since most of the data is stored on independent computers and datasets instead of a centralized storage system. [6] asserts that, DWs are widely used within the largest and most complex businesses in the world. Use within moderately large organizations, even those with more than 1,000. However [6] was addressing the use of use of DW in the corporate world rather than in the nonprofit making environment like government. As much as a DW provides advantages in the corporate world it will provide more or less the same advantages in the nonprofit making industry like government departments. Instead of gaining profit government will save on citizens taxes, the departments will also gain competitive advantage as much as the corporate world does. This is all achieved by the rapid decision making that is achieved by the use of the DW. In this paper we propose a five tier architecture for the South African government. This proposed architecture will increase efficiency and effectiveness in maintaining the cycle of activities, in their planning, decisionmaking processes, and analytical needs. The rest of the paper is organized into different sections. Section 2 gives related work. Section 3 presents the proposed five tier architecture and provides a clear discussion of the proposed architecture and section 5 provides the conclusion for the proposed architecture and future work. 2. RELATED WORK [7] researched on a practical approach for implementing a multi-tier DW in a large international firm. The research describes the strategy and processes used to define the DW architecture and development methodology used to model, map, populate and refresh the data in the DW. The practical approach is threetier architecture with the first-tier being the operational data store, the second-tier being the DW, the third-tier being the data mart. However, the proposed tier did not address the client-tier. Sharma (2009), proposed the use of a 3 tier architecture for advanced applications of a DW [8]. The 3-tier proposed had the first-tier as the extraction and transformation tier (DW Server layer), the second-tier as middle or connective tier (data mart layer), and the third-tier as data access and retrieval tier (Client Layer). The architecture proposed here did not include the operational database layer which would have made the whole architecture to be four-tier architecture.

5 5 [9] defines a DW architecture as a subset of the enterprise architecture is the DW. He goes on to emphasize that a well-documented architecture is a logical organisation of information pertaining to the corporate level. [9] describes the following as the advantages that are presented with a well-documented architecture; facilitate change management, enable strategic information to be consistent and accurate, promote data sharing. Reduction of data redundancy, improve productivity. While on the other hand [6] wrote paper answering the question Why You Need a DW, he states the advantages that are associated with the use of a DW. [6] states that the presence of a DW costs less and delivers more value than direct connection DW architecture is a way of representing the overall structure of data, communication, processing and presentation that exists for end-user computing within the enterprise [10]. [11] adopted a three tier level DW architecture which he recommended to the department of Ghanaian Petroleum Industry. The architecture proposed here had level 1 as the data conversion (scanning and manual entry of documents), level 2 as the data staging (processing, cleaning and conversion) and level 3 as ETL tools (loading to the DW and DM). However this architecture lacked the client level. In 2009 [3] proposes a three-tier architecture for high-performance data mining. In this architecture there is the front end/client architecture, the middle tier (the data mining) architecture and the back end or DW tier. In this architecture the operational database layer is not presented[3]. While in 2010, [12] did a research on the most effective and successful DW architecture. They identified five architectures: independent data marts, data mart bus architecture with linked dimensional data marts, hub-andspoke, centralized DW (no dependent data marts), and federated. They concluded by the use of quantitative approach that the bus, hub-and-spoke, and centralized architectures earned similar scores on the success metrics[12]. This research paper will adopt the hub-and-spoke architecture that was developed by [13]. The architecture developed by Inmon was developed on a 3-tier architecture. However, in this paper a 5-tier on a hub-andspoke architecture is going to be proposed in a South African government scenario. 3. PROPOSED ARCHITECTURE Level 1 Level 1 represents the first Tier and consists of operational databases for each department in South Africa as well as from stake holders to the government. Operation databases are comprised of the data created and maintained by the day to day operations of an organization. These systems keep track of specific transactions, customers, products, vendors, and resource information on a transaction by transaction basis. They are the systems which support the operations of an organisation on a day to day basis.

6 6 Level 2 Figure 1: A proposed 5-tier architecture for SA Government Level 2 represents the second Tier and consists of the ETL process and the DW. The ETL process is responsible for the extraction of data from the operational databases into the DW. As it extracts the ETL also cleanse and normalize data before it is moved to the DW. Therefore the DW stores the summarized and cleansed data from the operational databases. Also part of the second tier is the government meta-data server. A meta-data server makes sure that facts about data in the government DW are made available for decision makers. Level 3 Level 3 represents the third Tier and consists of a departmental Data Mart and a departmental meta-data server. This tier is customized for a specific department like the department of Rural Development and Land Reform. This department has a specific set of data requirements and wants the systems that support decision making be customized according to their requirements. The meta-data server at this tier is meant for keeping facts about the data stored in the departmental data mart. This will help in decision making as well.

7 7 Level 4 Level 4 represents the fourth Tier and consists of a corporate service Data Mart and its meta-data server. A corporate service is a section or division that belongs to a government department. The department of Rural Development and Land Reform consists of corporate services like human resource personnel, finance, salaries, communications, legal services, supply chain, labour relations, public relations, transport and logistics, performance management and IT. This tier is customized to a specific set of data requirements and wants the systems that support decision making be customized according to their requirements. The meta-data server at this tier is meant for keeping facts about the data stored in the corporate service data mart. This will help in decision making at the corporate level. Level 5 Several DW architectures have emerged over the last decade and are being used beyond the research community and business organizations. Organizations should prepare to exploit such architectures for decision-making. This can be facilitated by using the five tier architecture of data warehousing. This five tier architecture is applicable in situations where two or more businesses need to merge or in the scenario used in the paper which is the South African government and the department rural development and land reform. The five tier architecture will provide several advantages to decision makers in the government and respective departments. The architecture proposed here fits well with the South African government. However, other nations can adopt it if their setup is the same with the setup in South Africa 4. CONCLUSION AND FUTURE WORK Several DW architectures have emerged over the last decade and are being used beyond the research community and business organizations. Organizations should prepare to exploit such architectures for decision-making. This can be facilitated by using the five tier architecture of data warehousing. This five tier architecture is applicable in situations where two or more businesses need to merge or in the scenario used in the paper which is the South African government and the department rural development and land reform. The five tier architecture will provide several advantages to decision makers in the government and respective departments. The architecture proposed here fits well with the South African government. However, other nations can adopt it if their setup is the same with the setup in South Africa. Acknowledgment I wish to express my sincere appreciation and gratitude to Dennis Luke Owuor and Dr B.M. Kalema whose efforts have contributed to the completion of this article.

8 8 5. REFERENCES [1] E. M. A. Butt, et al., "Star Schema Implementation for Automation of Examination Records," ed: WORLDCOMP, [2] M. AhmedButt, et al., "DW Implementation of Examination Databases," International Journal of Computer Applications, vol. 44, pp , [3] R. Rantzau and H. Schwarz, "A Multi-Tier Architecture for High-Performance Data Mining," in Datenbanksysteme in Büro, Technik und Wissenschaft, 1999, pp [4] P. D. Manuel and J. AlGhamdi, "A data-centric design for< i> n</i>-tier architecture," Information Sciences, vol. 150, pp , [5] S. Helal, et al., "A three-tier architecture for ubiquitous data access," in Computer Systems and Applications, ACS/IEEE International Conference on. 2001, 2001, pp [6] J. Guerra, et al., "Why you need a DW," Andrews Consulting Group, www. rapiddecision. net/pdfs/why-you-need-a-data- Warehouse. pdf, [7] B. RAI, "IMPLEMENTING A GLOBAL MULTI-TIER DATAWAREHOUSE ARCHITECTURE," [8] P. Sharma, "Advanced Applications of Data Warehousing Using 3-tier Architecture," DESIDOC Journal of Library & Information Technology, vol. 29, pp , [9] A. Perkins, "DW Architecture: A Blueprint for Success," ed, [10] J. Deku, et al., "Three Tier level DW Architecture for Ghanaian Petroleum Industry," International Journal, vol. 4, [11] J. Yao Deku, et al., "Three Tier level DW Architecture for Ghanaian Petroleum Industry," International Journal of Database Management Systems, vol. 4, [12] T. Ariyachandra and H. Watson, "Key organizational factors in DW architecture selection," Decision Support Systems, vol. 49, pp , [13] W. H. INMON, "Building the DW," 2005