Development of evaluation Criteria. Design and Implementation of a database of

Transcription

1 Examination of the Best Practices in Administrative and Organizational Functions of the Greek Universities

2 Research Team George Tsiotras, Professor, Secretary General of the Region of Central Macedonia, Greece Katerina Gotzamani, Lecturer Project Director Antigoni Papadimitriou, Researcher Elias Dinas, Researcher Athanasios Tsadiras, PhD, Researcher

3 Maria Koemtzi, researcher April June 2005 Efi Kapassakali Administrative Assistant

4 Further goals of this research Investigation of current application Development of evaluation Criteria Identification of Best Practices Design and Implementation of a database of Best Practices

5 Project framework Catalog of Greek universities and a presentation of the selected administrative and organizational functions Record of the current situation of the chosen administrative and organizational functions Evaluation of the applied procedures Collection and analysis of the information Development of the data base Results of the research project

6 Chosen organizational and administrative functions Library Information Technology Center Research Committee Career s Office Department of Public and International Relation

7 Defining the evaluation criteria Education Criteria for Performance Excellence by the Baldrige National Quality Program (2004) EFQM Excellence Model Higher Education Version by Sheffield Hallam University in collaboration with EFQM (2003) Academic Scorecard evaluation of academic organizations ( balanced scorecard )

8

9

10

11 EVALUATION CRITERIA QUANTITATIVE Financial i and other Leadership resources Customers-Users Strategy QUALITATIVE Personnel User satisfaction i Processes (function- specific) Personnel Society Processes Other data (not included in previous categories) Resources

12 BEST PRACTICES Quantitatively high performing practices High score achievements in qualitative factors

13 Problems: Pilot study - reluctance by most functions head officers to participate in the project in the form of evaluation so as to record best practices was observed - questionnaire related comprehension difficulties during completion by the representatives

14 Solutions - Development of second qualitative questionnaire which accompanies the interviews - identify and record best, good, or effective practices

15 Looking for best practices These practices are not directly associated with the terminology (definition) of best practices, as these are found in the literature, but they are based on the personal experience of the participants and the experience and subjective judgement of the research team

16 Comments from the 1 st International Conference Professor Brent Ruben, Rutgers University USA and Professor Al Shagana Kadim, Sheffield Hallam University, UK this project is a challenge for the research team

17 Project Schedule Phase 1 Phase 2 March June 2004 Catalog of Greek Universities Presentation of the selected administrative and organizational functions July October 2004 Investigation of current practices Phase 7 NOV- DEC 2006 Results of the Research Project Phase 3 Phase 4 Phase 5 Phase 6 Nov Feb.2005 Development of evaluation Criteria March Jun 2005 Best Practices from foreigner Universities Jul Jun 2006 Pilot study, Collection & analysis of selected data July- Oct 2006 Development of the Database

18 Geographical distribution ib ti of the Greek Universities

19 1. National and Kapodistrian University of Athens, 1837, 2. National Technical University of Athens (Metsovio), 1836, 3. Athens University of Economy and Business, 1920, 4. Pantion University, 1927, 5. Agriculture University of Athens, 1920, 6. University of Piraeus, 1938, 7. Athens School of Fine Arts, 1837, 8. Harokopio University, 1929, 9. Aristotle University of Thessaloniki, 1925, 10. University of Macedonia, 1948, 11. University of Patras, 1964, 12. University of Ioannina, 1964, 13. Democritus University of Thrace, 1973, 14. University of Crete, 1973, 15.Technical University of Crete, 1977, 16. University of the Aegean, 1920, 17. Ionian University, 1984, 18. University of Thessaly, 1984, 19. University of Western Macedonia, 2002, 20. University of Peloponnese, 2000, 21. University of Continent Greece, 2003 Source: A. Papadimitriou, Quality Assurance in Greek Higher Education, Ph. D. Dissertation, University of Macedonia, Thessaloniki, Greece, in process.

20 Preliminary DATA ANALYSIS For the above reasons the first questionnaire was regarded as not appropriate for the use of evaluating and identifying of best practices and it will only be used for an explorative recording of the current quality management practices in the specific academic functions services

21 Data analysis Leadership Strategy Personnel Processes Resources User Satisfaction

22 Data collection until Jan Library 11 Research Committee 4 Information Technology Center 5 Career s Office 4 Department of Public and International Relation 3 27

23 Population Administrators 24 Library Research Committee Information Technology Center Career s Office Department of Public and International Relation Faculty 3 Library Research Committee Information Technology Center Career s s Office Department of Public and International Relation

24 1) leadership Scale -items Appplication rate Importance rate Scaled response - Score

25 Leadership (app. & imp. rates) scales distribution 6 Leadership 10 Leadership Std. Dev =,37 Mean = 4,57 2 Std. Dev =,81 Mean = 4,02 0 4,00 4,25 4,13 4,38 4,50 4,75 4,63 4,88 5,00 N = 21,00 0 1,00 2,00 3,00 4,00 5,00 1,50 2,50 3,50 4,50 N = 22,00 LEADIMP importance rate LEADAPP Application rate

26 2) Strategy Scaled response - score Importance rate Application rate Q3: Do you collect and estimate within your strategy planning, data concerning corresponding Services of other Constitutions or Foundations of Education? App:3.45/imp.4.18

27 Strategy (app. * imp. rates) scales distribution Strategy Strategy Std. Dev =,82 1 Std. Dev =,43 Mean = 4,40 0 Mean = 3,83 N = 22,00 0 N = 19,00 1,00 2,00 3,00 4,00 5,00 3,50 3,75 4,00 4,25 4,50 4,75 5,00 1,50 2,50 3,50 4,50 STRTIMP importance rate STRTAPP application rate

28 User Satisfaction Scaled Response - Score Importance rate Application rate Q6:Do you apply a comparative evaluation method for Users Satisfaction in relation to the satisfaction provided by corresponding Model-Services (Benchmarking) imp:3.78/app.2.2 Q7:Do you establish relationships (eg. collaborations with corresponding Services of other Educational Institutions) aiming to satisfy the Service Users? imp: 3.54/app:2.56

29 User satisfaction (app. & imp. Rates) scales distribution 6 User satisfaction 8 User satisfaction ,25 3,50 3,75 4,00 4,25 4,50 4,75 5,00 Std. Dev =,51 Mean = 4,17 N = 21, ,00 1,50 2,00 2,50 3,00 3,50 4,00 4,50 Std. Dev =,75 Mean = 3,30 N = 22,00 USERSIMP Information rate USERSAPP application rate

30 Personnel Scaled response - score Importance rate Application rate Q3: Do you apply an official system of recognition of Service Staff? imp:3.6/app:2.3

31 Personnel (app. & imp. rates) scales distribution 5 Personnel 10 Presonnel Std. Dev =,44 Mean = 4,44 0 N = 18,00 3,63 3,88 4,13 4,38 4,63 4,88 3,75 4,00 4,25 4,50 4,75 5, ,50 2,00 2,50 3,00 3,50 4,00 4,50 5,00 Std. Dev =,68 Mean = 3,76 N = 18,00 STAFFIMP importance rate STAFFAPP application rate

32 Processes Scaled response - score Importance rate application rate Q3:Do you apply specific measurement indicators for the control and improvement of procedures? imp:4.1/app:2.5 Q4:Do you apply some official procedures evaluation method? imp:4/app:2.5

33 Processes (app. & imp. rates) scales distribution 6 Processes 6 Processes Std. Dev =,48 Mean = 4,34 1 Std. Dev =,96 Mean = 3,17 0 3,25 3,50 3,75 4,00 4,25 4,50 4,75 5,00 N = 21,00 0 1,00 1,50 2,00 2,50 3,00 3,50 4,00 4,50 N = 21,00 MANGEIMP importance rate MANGEAPP application rate

34 Resources Scaled response -scored Importance rate Application rate Q4: Do you apply programs of mutual development and instruction (eg. mutual exchanges) imp:3.6/app:2.1

35 Resources (app. & imp. Rates) distribution 6 Resources 6 Resources Std. Dev =,48 Mean = 4,47 1 Std. Dev =,82 Mean = 3,53 0 3,50 3,75 4,00 4,25 4,50 4,75 5,00 N = 15,00 0 1,50 2,00 2,50 3,00 3,50 4,00 4,50 N = 17,00 RSRSIMP Importance rate RSRSAPP Application rate

36 Scale reliability Table1: Scale reliability: Reliability coefficients (Cronbach s α) ) of six criteria Leadimp Strtimp Usersimp Staffimp Mangeimp Rsrcsimp Leadapp Strtapp Usersapp Staffapp Mangeapp Rsrcsapp Application rates show, in general, greater reliability; Resources scale is, in both cases, the most problematic

37 Table 2: Descriptive statistics from EFQM variables (importance & application) N Minimum Maximum Mean Std.Devation LEADIMP STRTIMP USERSIMP STAFFIMP MANGEIMP RSRCSIMP N Minimum Maximum Mean Std.Devation LEADAPP STRTAPP USERSAPP STAFFAPP MANGEAPP RSRCSAPP

38 Average scores (app. & imp. rates) Mean responses Scaled response - score Importance rate Application rate In general, mean responses are greater for importance rates rather than for application rates

39 Average scores (app. & imp. rates) Std. Deviations 1,2 1 0,8 06 0,6 0,4 0, Importance rate Application rate In general, application scales show more variability than importance scales (as was also indicated by each criterion histogram)

40 Are these Differences statistically significant? Table 3: Resutls from difference of means tests (two-tailed) tailed) between the key EFQM variables. Leadimp Strtimp Usersimp Staffimp Mangeimp rsrcsimp Leadimp - Yes Yes No Yes No Strtimp - Yes No No No Usersimp - Yes Yes No Staffimp - No No Mangeimp - No Rsrcsimp - In most cases (9/15), they are not.

41 Correlation between the six criteria Table 4: Pearson correlations between the 6 criteria Leadimp Strtimp Usersimp Staffimp Mangeimp rsrcsimp Leadimp Strtimp Usersimp Staffimp Mangeimp -.79 As it was expected, all criteria are intercorelated (imp. Rates) although greater correlation is observed between User satisfaction & personnel/processes Table 5: Correlation between importance and application rate Leadimp Strtimp Usersimp Staffimp Mangeimp Rsrcsimp Leadapp.63* Strtapp.56* Usersapp.41* Staffapp.18 Mangeapp -.06 Rsrcsapp.238 * p<.05 Although suspected, importance rates responses are not driven by application rates responses (and vice versa)

42 Questions, comments and suggestions are welcomed THANK YOU