SERVICE ORIENTED BUSINESS INTELLIGENCE FOR MONITORING ACADEMIC QUALITY

Transcription

1 SERVICE ORIENTED BUSINESS INTELLIGENCE FOR MONITORING ACADEMIC QUALITY Irya Wisnubhadra Universitas Atma Jaya Yogyakarta Jl Babarsari No 6, Depok, Sleman, Special Region of Yogyakarta ABSTRACT Academic Quality Assurance to control and audit academic quality is already a compulsory activity for University in Indonesia. This activity has already written in National Education System Law, number 20 year Under this rule, University in Indonesia has to be aware and assure its academic quality achieves the quality objective or performance excellence Education Criteria for Performance Excellence was founded and derived from Malcolm Baldrige Quality Management System. These criteria are very useful to help organization to improve their performance based on integrated approach for customer services, effectiveness, organization holistic capability and organization learning capability. Malcolm Baldrige using 7 category criteria that have to be achieved by university in order to reach the best academic performance. This paper will present design and implementation of Web Based Business Intelligence application that display university achievement about Malcolm Baldrige Education Criteria in nearly real time. Rector or University s Director could use this application to monitor their performance and then make the right decision to improve their performance. Design and implementation will be focused on Business Intelligence with Service Oriented Architecture model, that using Services as an integrator, extractor and collector information from distributed island of information. Keywords: Business Intelligence, Service Oriented Architecture, Academic Quality, Malcolm Baldrige Education Criteria. 1. Introduction: Academic Quality Assurance in Indonesia Academic Quality Assurance to control and audit academic quality is already a compulsory activity for University in Indonesia since The Indonesian National Education System Law (UU No 20 year 2003 on National Education System) [1] has been passed. Under this rule, University in Indonesia has to be aware and assure its academic quality achieved the quality objective or performance excellence. The Indonesian National Education System Law, article 50 verse 2, explicitly expresses that Indonesian Government have the responsibility defining the national policy and standard of education. Directorate General of Higher Education as Indonesian Government representative in education defines the National Policy of Education in Higher Institution Quality Assurance System (Sistem Penjaminan Mutu Perguruan Tinggi: SPM-PT). SPM-PT system synergized Evaluation of Study Program Based on Self Evaluation (EPSBED), Higher-Education Institution Accreditation System and Institution- Based Quality Assurance System. SPM-PT as a quality assurance system comprises three (3) subsystem [2],[3]: a. National Higher Institution Warehouse or Pangkalan Perguruan Tinggi (PDPT) Nasional, which is a systemic activity for collecting, saving and managing education data transaction in higher education institution. This activity has to be managed by directorate general of higher education. b. Internal Quality Assurance System, which is internally driven activities to control and ISBN: SDIWC 136

2 audit the implementation of higher education by higher institution itself. The education process has to be in continuous improvement and sustainable methodology. c. External Quality Assurance System, which is a systemic activity to assess higher institution/college performance. This activity implemented by BAN-PT or independent institutions outside the university which is recognized by the government. Subsequent discussion in this paper relates to the Internal Quality Assurance System and External Quality Assurance System by BAN-PT, with no intention to eliminating another sub-system. Internal Quality Assurance System is intended to give higher institution/college the opportunity to exceed the National Education Standards in a sustainable way. The higher institution has to do this in continuous improvement mechanism. This effort is to reach the internal needs of stakeholders (students, teachers, educators). Meanwhile, the External Quality Assurance System through assessment by BAN-PT has been designed to meet the needs of external stakeholders (parents, employers, community, and government). 2. The Malcolm Baldrige Performance Excellence The Malcolm Baldrige National Quality Award [4][8] is an annual award that recognizes U.S. organizations in the business, health care, education, and nonprofit sectors for performance excellence. It is administered by the Baldrige National Quality Program, which is based at and managed by the National Institute of Standards and Technology, an agency of the U.S. Department of Commerce. Figure 1. Baldrige Education Criteria for Excellence Framework [4] The Award promotes awareness of performance excellence as an increasingly important element in competitiveness and information sharing of successful performance strategies and the benefits derived from using these strategies. To receive a Baldrige Award, an organization must have a role-model organizational management system that ensures continuous improvement in the delivery of products and/or services, demonstrates efficient and effective operations, and provides a way of engaging and responding to customers and other stakeholders. The Award is not given for specific products or services. The main uses of the Baldrige Criteria for Performance Excellence are education and organizational self-assessment and selfimprovement that also in line with The Indonesian National Education System policy. Malcolm Baldrige Criteria for Excellence is comprises seven categories (figure 1). The seven categories of this criteria are: 1. Leadership 2. Strategic Planning 3. Customer Focus 4. Measurement, Analysis, and Knowledge Management 5. Workforce Focus 6. Process Management 7. Results The Baldrige Criteria for Performance Excellence provide organizations with an integrated approach to organizational performance management that results in ISBN: SDIWC 137

3 1. delivery of ever-improving value to customers and stakeholders, contributing to organizational sustainability 2. improvement of overall organizational effectiveness and capabilities 3. organizational and personal learning These criteria are very useful to help organization to improve their performance based on integrated approach for customer services, effectiveness, organization holistic capability and organization learning capability. The seventh category of Malcolm Baldrige criteria is the Result, that examines organization s performance and improvement in all keys areas, including student learning outcomes; customer-focused outcomes; budgetary, financial, and market outcomes; workforce-focused outcomes; process effectiveness outcomes and leadership outcomes. Performance levels also examined relative to those of competitors and other organizations with similar program and services Student Learning Outcomes: What are the student learning results? Summarize the organization s key students learning results. Segment the results by student and market segment, as appropriate. Include comparative data relative to competitors and to comparable organizations and student populations. Provide data and information to answer the following questions: Student Learning Results a. What are the current levels and trends in key measures or indicators of student learning and improvement in student learning? How do these results compare with performance of your competitors and comparable organizations and other appropriate student and market segments? The result of student learning is measured by Student GPA, Retention Rates, and Graduates GPA 2.2. Customer-Focused Outcomes: What are the student-and stakeholder-focused performance results? Summarize the organization s key student-and stakeholder-focused result for student and stakeholder satisfaction, dissatisfaction, and engagement. Segment the result by educational program, offering, and service features, and by student, stakeholder, and market segments, as appropriate. The result of customer focus is measured by Employers Rating of Graduates, Students Satisfaction by Academic Program, Students Satisfaction with Curricula, Students Satisfaction with Complain Management, and Student Engagement 2.3. Budgetary, Financial, and Market Outcomes: What are the budgetary, financial, and market performance results? Summarize your organization s key budgetary, financial, and market performance results by student segments, stakeholder groups, or market segments, as appropriate Workforce-Focused Outcomes: What are the workforce-focused performance results? Summarize the organizations key workforcefocused results for workforce engagement and environment. Segment your results to address the diversity of the workforce and to address the workforce groups and segments, as appropriate. Include appropriate comparative data. The result of workforce-focused is measured by workforce engagement, and workforce satisfaction From the result category, Top Management could see the achievement of the higher institution, whether or not the higher institution achieves the performance excellence. This result or this information then could be delivered to Top Management using Business Intelligence (BI) tools. BI Tools could deliver the information interactively, effectively with good ISBN: SDIWC 138

4 performance. BI tools also deliver information in real time accommodating the changing data. 3. Business Intelligence for analysis and monitoring performance excellence Business Intelligence (BI) applications can be defined as a decision support tools that enable real-time, interactive access, analysis and manipulation of mission-critical corporate information[10]. Users are able to access and leverage vast amounts of information to analyze relationships and understand trends that, ultimately, support business decisions. The desired outcome of BI projects is the continuous improvement of the organization through timely information that enhances decision making. BI also supports internal enterprise users in the assessment, enhancement and optimization of organizational performance and operation [5]. BI relies on Warehousing (a data repository designed to support an organization's decision making), making cost-effective storing and managing of warehouse data critical to any BI solution. Without an effective data warehouse, organizations cannot extract the data required for information analysis in time to facilitate expedient decision-making. The ability to obtain information in real-time has become increasingly critical in recent years because decision-making cycle times have been drastically reduced. Figure 2. Business Intelligence and Warehouse [9] Figure 1 depicts the component of Business Intelligence system. BI process begin with storing of raw data from on-line transaction processing (OLTP), legacy system, and external source. Then, is extracted, transformed, and loaded to information warehouse. is integrated and is designed for particular subject, non volatile, and optimized for query processing. Subsequent process is data mining, query and analytical tools to generate intelligence. Various data mining, query and analytical tools generate the intelligence that enables companies to spot trends, enhance business relationships, and create new opportunities. Organizations use intelligence to make strategic business decisions, with this intelligence, organizations can make effective decisions, and create strategies and programs for competitive advantage Web-Business Intelligence and Service Oriented Architecture The emergence of the Web phenomenon and Service Oriented Architecture in 1990 and 2000 has changed the way computing and communications as well as Business Intelligence. Web-Business Intelligence appears to deliver information within the framework of Business Process Management using the Internet infrastructure including intranet, internet, and extranet. Web-Business Intelligence application increases the data warehouse accessibility, which raises the level of enterprise knowledge. The emergence of service oriented architecture also increases the accessibility of data from various sources in a specific service that loosely couple and interoperable. Traditional BI has ETL process that batch-driven process focused on integrating data during business downtime. In today's connected marketplace, business does not have a quiet time for this process to occur. ETL must be flexible enough to emerge from its batch-driven roots to deliver data on an event-driven basis. Within a service oriented solution, these events are readily routed, consumed, and integrated as part of an event-driven architecture. BI grew out of the inability of operational systems to efficiently handle analytical capabilities (aggregation, trend, exception, and so on), hence, distinct BI systems were developed to meet this need [6][7]. The architecture of Service Oriented Business Intelligence could be depicted in this figure. ISBN: SDIWC 139

5 Mrican Campus Faculty D Catur Tunggal Campus Faculty A Communication Link Communication Link Intranet/Internet Communication Link Main Campus Babarsari Campus Faculty B Faculty C Communication Link figure 3. Architecture for Service Oriented Business Intelligence [6] s This paper will explain the design and implementation of Service Oriented Business Intelligence using this kind of model for monitoring performance excellence which is depicted in figure Case Study University Xyz (UX) is a university in Indonesia that was established in This university has about 9000 students. This university has four (4) faculties and ten (10) study program. UX has four campuses: main campus is located in Mrican district, second campus is in Catur Tunggal district, and the third and fourth campus is located in Babarsari district. Each campus has around 20 kilometers average distance from main campus. UX has IT system with distributed technology that covers all campuses. The UX IT Architecture is depicted in this figure. Each Faculty in UX has Online Transaction Application. Each faculty has its database and server for running application. This application ranging from Academic Information System, Web based E-Learning System, Alumni Information System and so on. The UX also has applications that cover the centralize business process like students admission, research and community services, public relation, counseling, and so on. figure 4: UX IT architecture UX implementing Service Oriented Business Intelligence (UX SOA-BI) for display and monitor performance excellence that comply with Malcolm Baldrige Standard Performance Excellence. UX SOA-BI created to monitors the following information: 1. Student GPA 2. Retention Rate 3. Graduation Rate 4. Graduate GPA 5. First Job Period 6. Alumni Participation Rate 7. Employer Rate 8. Job Related 9. Employers Rating of Graduates 10. Students Satisfaction by Academic Program 11. Students Satisfaction with Curricula 12. Students Satisfaction with Complain Management 13. Student Engagement 14. Curricular/Extra Curricular Student Engagement UX SOA-BI has designed and implemented with the components depicted in figure 5. ISBN: SDIWC 140

6 Analytics Application Multidimentional base ETL Services BI QA Warehouse HR dimensions and facts form a schema that called Star Schema. All list of information in the first result above (Student GPA, Retention Rates, Graduation Rates, and Graduates GPA, etc) should be delivered to Top Management to support the decision making. The Star Schema for several information is depicted in figure 6, figure 7,and figure8. Once the star schema design has been finished, it deployed to become Warehouse/ Marts. Marts load data from OLTP source to become ready to give information for Top Management. Source figure 5: Business Intelligence component Warehouse and Mart were created by services for extract, transform, and loading process from source data. Source data coming from multiple sources. Source data is ranging from Academic Transaction/Operational that coming from Academic Information System in each faculty, Admision Transaction coming from University Admision Office, etc. Competitor that also have to display in result, could be pulled from National Institution (Statistics Central Bureau) through their Web Services as an external data. Extract, Transform and Loading Process using Staging base to save temporary data before it load into Warehouse and Marts Analysis and Design for Warehouse and Marts Analysis and Design of Warehouse is often said to be Dimensional Modeling. The way of design Warehouse is different with the design of Online Transaction Processing (OLTP) because of the nature of Business Intelligence for information delivery. BI Analysts have to think like a manager. They have to think in business dimensions, and then formulating it to the information packages including: business dimensions, hierarchies/categories and key business metrics or facts. The business Information Delivery After loading data into Marts has been done, the information is ready to use. UX uses desktop and web based tools for reporting the information. Information displayed and reported in various ways such as tables, graphs, and dashboards. The next tables and figures show several education criteria for performance excellence using solid graphs and dashboards. TimeDim TimeId Semester Month AlumniDim AlumniId StudentId DateOfBirth PlaceOfBirth Sex StudentDim StudentId DateOfBirth Sex RetentionStatus OfEntry StudentPerformanceFact,FK1 StudyProgramId,FK2 TimeId,FK3 StudentId GPA CummGPA GraduationFact,FK1 TimeId,FK4 StudyProgramId SupervisorId,FK2 AlumniId GPA LengthOfStudy FK1 StudyProgramDim StudyProgramId FacultyId FacultyDim FacultyId SupervisorDim SupervisorId Degree FieldofStudy RetentionStatus figure 6: Student Performance and Graduation Star Schema ISBN: SDIWC 141

7 AlumniDim TimeDim AlumniId StudentId DateOfBirth PlaceOfBirth Sex FieldOfJob OfEntry TimeId Semester Quarter Month RespondentDim RespondentId Institution InstitutionType EmploymentFact,FK1 TimeId,FK2 AlumniId EmployerRate FirstJobPeriod ParticipationRate SatisfactionFact,FK1 TimeId,FK3 RespondentId AcademicSatisfactionRate CurricullaSatisfactionRate ComplaintRate EngagementRate FacultyDim FacultyId StudyProgramDim StudyProgramId figure 7: Employment and Satisfaction Star Schema Faculty/Studi Program Industrial Technology 91,00 90,67 92,3 Industrial Engineering Electrical Engineering Chemical Engineering Information Technology 91,00 91,67 93,3 Informatics Eng Information System Software Engineering Laws Law Science Table 2: UX Pass Rates comparing with competitors, and national top institution No Institution UX 91,1 91,4 92,9 2 Peer Peer Peer Province National UnitDim UnitId Unit TimeDim TimeId Semester Quarter Month EmployeeSatisfactionFact,FK2 StudyProgramId,FK1 TimeId,FK3 EmployeeId AcademicSatisfactionRate CurricullaSatisfactionRate ComplaintRate EngagementRate Employee EmployeeId DegreeLevel AcademicLevel figure 8: Employee Satisfaction Star Schema Top Management could use this to monitor their academic institution performance real time and in an effective way. Table 1: Pass Rates by Faculty and Study Program Faculty/Studi Program Economics 91,33 91,33 93,33 Accounting Management Economics Science figure 9 : Graphs report for student pass rates Table 3: UX Student GPA comparing with competitors, and national top institution No UX 2,96 3,03 3,02 2 Peer 1 2,86 2,88 2,98 3 Peer 2 2,97 2,95 2,99 4 Peer 3 2,85 2,95 2,95 5 Province 2,95 2,97 2,98 6 National 2,96 2,98 3,00 ISBN: SDIWC 142

8 knowledge supporting the decision making. The better the information we have, the better our decisions to achieve quality objectives of education. figure 10 : Dashboard report for Alumni GPA, Student GPA, and Pass Rates 5. Conclusion This paper described implementation Service Oriented Business Intelligence for monitoring Malcolm Baldrige Performance Excellence. Business Intelligence using Web Enabled and Service Oriented Architecture has several advantages including: real time data monitoring, enhanced data warehouse accessibility, and using services that have interoperability, and loosely couple attribute. Using this tool, higher academic institution could have more information and higher level of 6. References 1. Undang Undang Republik Indonesia, tentang Sistem Pendidikan Nasional Sistem (2003) 2. Peraturan Pemerintah Republik Indonesia, nomor 19 tahun 2005, tentang Standar Nasional Pendidikan (2005) 3. Departemen Pendidikan Nasional, Sistem Penjaminan Mutu Perguruan Tinggi (2009) 4. National Institute of Standards and Technology, Department of Commerce, Education Criteria for Performance Excellence, (2011) 5. Ponniah, Paulraj, Warehousing fundamentals, John Wiley and Sons, (2004) 6. Gordon, S., et. All, Service-Oriented Business Intelligence, Microsoft Architect Journal (2009) 7. Rotem, Arnon, Bridging the gap between Business Intelligence and SO, (2007) 8. National Institute of Standards and Technology, Department of Commerce,2009, Nightingale College of Nursing Case Study, 9. Anonymous, Business Intelligence and Warehouse overview, (2008) 10. Inmon, W.H, Building the warehouse Second Edition, John Wiley & Sons, Inc, New York. (1996) ISBN: SDIWC 143