RAE2015 OF THE UNIVERSITY OF TURKU PEER-EVALUATION REPORT RESEARCH UNIT FOR THE SOCIOLOGY OF EDUCATION (RUSE) Faculty of Social Sciences
RAE2015 OF THE UNIVERSITY OF TURKU PEER-EVALUATION REPORT RESEARCH UNIT FOR THE SOCIOLOGY OF EDUCATION Faculty of Social Sciences Panel: Arve Egil Asbjørnsen (chair), Clotilde Calabi, Maria Lähteenmäki, Björn Wittrock 1. OVERALL RATING OF THE UNIT: Excellent 2. Description of the research activity during the evaluation period 2010 2013 The Research Unit for the Sociology of Education (RUSE) has been established as an independent unit to assess empirical facts for making evidence based policy. The research is described as multidisciplinary, as in addition to sociology of education it also involves elements of economics, philosophy, computer science and history. The mandate of the RUSE are: 1. to conduct research related to education, especially the system of higher education, and higher education policy; 2. to develop research methods for social sciences and methods for evaluating education and research; 3. to offer academic postgraduate education; 4. to advance international and multidisciplinary research cooperation in the field of research on education; and to develop different methodological solutions for advancing the consolidation of social sciences and information and communications technology (ICT) in both research and development work; to develop methods applicable to research on strategic operation and applications possibly leading to business activity. RATING: Good to excellent Describe the panel s view on the scientific quality and innovativeness of the research activity Key research topics include relations between education and transition to working life, education policy, equality of educational opportunities, relational methodology, pragmatist philosophy, human evolution, productivity of research, youth marginalisation, in addition to the activities of the Research Laboratory of Strategic Actions. The unit has been prolonged following a meeting with the rector and vice rector in 2013, justified by the close cooperation with the Ministry of Education in producing and providing evidence-base. The unit has training of doctoral students as the only educational responsibility. The quality of the output is varying, but is mostly of good to excellent quality.
2 / 10 How do you see the main achievements in relation to the main resources (for personnel, see 3, for funding, see 4, and for infrastructure, see 5) invested in the research activities of the unit? The main achievements are low, considered that the infrastructure and support personnel are equal to what is seen for the regular departments of the Faculty of Social Sciences. Professor Kivinen has a respectable output, but the output from the unit at large is not as impressive, taking into consideration that the employees have no teaching obligations besides the doctoral training, and PhD-graduation should mainly be training in writing for publication. Indicate strengths, weaknesses and other remarks Probably a strong leader is the main asset, in addition to a strong basic funding. The weakness can be seen as this is basically the project of one strong professor, and there is no welldeveloped plan for the future development of the unit. On the average, the publication rate is approximately one publication per head, with Kivinen s own publishing skewing this figure with excellent scientific output. For a research unit of this size, one could expect a higher output, and also a larger success rate in competition for funding. The basic/external funding was 10.76 in 2011, and although down to 5.24 in 2012 and 4.34 in 2013, this is far above what is seen for the regular departments of the Faculty. The number of publications in total is approximately as seen for each member of other departments in total. However, since the basic funding of the unit in general consists of project funds from the Ministry, it is not given that the comparison to other units comprise a good evaluation approach. 3. Organisation and personnel during the evaluation period 2010 2013 RATING: Good Describe your view on the following points: The unit s recruiting policy of researchers at various career stages The unit employs a substantial number of early stage researchers on temporary contracts and with the opportunity to achieve a doctoral degree. Possibilities for researchers to proceed in their career
3 / 10 Probably not as good as for the initial training, as the unit has access to two post-doctoral fellowships, and have few other more advanced positions available. SUBRATING: Sufficient Are the teachers and the researchers also actively involved in research and teaching, respectively? How do you see the balance between teaching and research? The unit s only responsibility is doctoral training, and the only information is that the professor is responsible for the supervision of the doctoral students. It would probably be of benefit to the faculty, and also for the further consolidation of the unit, that the members of this unit engage in teaching activity at the graduate level (Master and doctoral courses). SUBRATING: Insufficient Gender and equality issues: pay attention e.g. i) to good gender balance as well as the proportion of foreign personnel in the personnel groups in Appendix A The gender balance and proportion of foreign personnel was below the average in the faculty during the assessment period, but it is now reported to be in balance. and ii) to the parental leave issues and gender balance in Appendix B The parental leaves are less frequent compared to the faculty at large, and only female members are reported to be on parental leave. This may indicate a biased recruitment, or a work environment that is not adjusted to the need of researchers with small children. SUBRATING: Insufficient RUSE s permanent staff includes one professor, in addition to one senior research fellow on a long term appointment. One coordinator and one planning officer are employed on permanent contracts. Other staff includes two postdoctoral researchers with 5-year contracts, a senior research fellow and a changing number of project researchers, doctoral students, research assistants and other project staff. RUSE has also software and computer analysts. The total number of staff varies between 20 and 25 depending on the number of projects. The recruitment to the RUSE should be taken into consideration to assure gender balance both in temporary and permanent staff.
4 / 10 4. Research funding during the evaluation period 2010 2013 RATING: Good to Excellent Describe your view on the following points: The success and strategy of the unit in the competition for funding Since the majority of the unit s funding is coming from the ministries as project funding, it is hard to compare the funding of this unit towards the other departments, where the basic funding is also supporting teaching obligations. The strategy of RUSE has mainly been collaboration with other national and international research groups with a similar mandate, and collaborative and comparative research within the main areas of the unit. This strategy has been somewhat successful, and has generated research funding and given good output. Is the division of external funding into the different categories as shown in Appendix C appropriate? A portion of the total budget comes from international funds, and some of these are collaborative parts of programmes in countries of cooperation. The unit has had some success in competitive domestic and international funds (Academy of Finland, European Social Fund and Bundesinstitut für Berufsbildung). The Appendix C is probably not representing an appropriate way of reporting the situation for this unit, as also the basic funding in general is project based. to excellent The external and competitive research funding is low for a research unit with few teaching obligations, as the unit has been relying on the continuing funding from the ministry. 5. Infrastructure during the evaluation period 2010 2013 RATING: Excellent Describe the panel s view on the following points: Major strengths and weaknesses in the unit s infrastructure SUBRATING: Excellent Development of the infrastructure 2010 2013 in relation to research needs SUBRATING: Excellent Possible impact of the unit in developing research infrastructures
5 / 10 The unit reports the infrastructure and access to necessary facilities to be satisfactory. The unit has developed computerized models and laboratory procedures that may influence similar analyses internationally. 6. Scientific quality of research during 2010 2013 RATING: Good to Excellent How would you evaluate the scientific quality of the unit s research in relation to top national/international research? Some of the publications are of excellent quality, but only few are published in highly recognized international journals and in journals categorized at the highest level in the JuFo system. The major production is unclassified publications, and too many publications of the total production are in low categorized journals or books. A larger proportion of the research results should have been published in higher categorized journals for quality assurance and impact. 7. Publications 2010 2013 RATING: Good to Excellent How would you evaluate the publication policy and quality based on the examples of publications in the self-evaluation report and Appendixes D and E? Estimate the overall quality of the publications that the unit estimates as its most important ones. to Excellent Several of the published papers are of high quality and with impact on the field of study (higher education). Also, results of some other studies could have been submitted to more general journals publishing educational research, as the results are of more general interest.
6 / 10 8. Doctoral training 2010 2013 as part of the scientific activity in the unit RATING: Good How would you estimate the organisation and success of doctoral training in the unit? The doctoral students work is an integrated part of the unit s research portfolio, which is excellent. However, the number of doctorates that are trained seems to be approximately equal to what is going on in the regular departments of the Faculty, and not as one should expect of a unit with no other teaching obligations. It is hard to get a grasp of how the doctoral programme is organized, and the report is not made to make the production clear. The output has been approximately one PhD per year. With one professor, two senior researchers and two post doc fellows, one should expect there to be capacity to train more doctoral students in parallel. 9. Researcher mobility RATING: Insufficient How would you evaluate researcher mobility in the unit? SUBRATING: Insufficient Has the unit succeeded in attracting international researchers? SUBRATING: Insufficient The reported mobility rate both in and out has been low, and the unit has not been able to attract international researchers, or to a large degree participate in exchange or international research activity. Two members of the unit have spent time in other locations, and this is low considering the number of early stage researchers in the unit. 10. International co-operation and partners 2010 2013 RATING: Excellent Describe the panel s view on the following points: Success and extent of international co-operation SUBRATING: Excellent Has international co-operation provided clear extra value for the research? SUBRATING: Excellent
7 / 10 The members of the unit participate in international research, mainly funded through ministries in the Nordic countries. This strategy has been fruitful in the past, but increased focus on external and competitive funding may be crucial to the future existence of the unit. 11. National co-operation partners 2010 2013 RATING: Good Describe the panel s view on the following points: Success and extent of national co-operation The unit describes quite substantial cooperation with national units, and mainly within research on higher education. The cooperation is established both as partners and also project leaders in joint research projects. Has the national co-operation provided clear extra value for the research? Cooperation has extended the foci of the unit, and has probably also assured more impact SUBRATING: Excellent The unit could benefit from enhanced efforts to also increase their success in more competitive funding, also from the point of quality assurance of their research efforts. 12. Wide-range impact of research during 2010 2013 RATING: Good to Excellent How would you estimate the impact of the unit s research on basic and applied research? The research results produced by the unit feed into the international knowledge and literature on higher education in particular, and also findings from basic education are of international interest and worth publishing in international journals. The unit has also one project that started out as basic research, the ReadIT that has evolved into a practical tool for assessment and adaption of learning material to individual skills. However, in accordance with the mandate, the main research approach is applied in nature.
8 / 10 13. Innovations RATING: Good to Excellent How would you evaluate the innovations (e.g. social innovations, co-operation with companies and TEKES projects, established companies, patents, innovation announcements) described in the unit s self-assessment? According to the self-assessment, RUSE has developed and tested its own ReadIT learning solution. It can be described as a web-based intelligent learning system for improving learning skills. ReadIT adapts to students activities and directs them toward practices that enhance learning by differentiating learning material to every student's current level. This represents excellent innovation, with probably high impact. Further, RUSE has developed a Productivity analyser which is an online toolset for studying and teaching how to conduct empirical social scientific analyses leaning on relational methodology and real-time data. The database builds on real-time, comparable input and output measures of 10 000 disciplinary units hosted by 700 universities in 42 countries. Productivity analyser combines input and output data from on-line open access sources. This also represents excellent innovation with expected impact on the field. RUSE has also developed a real-time documentation tool and a modified ERP for nonprofit/voluntary organizations and associations. A social innovation for engaging young (especially those in danger of marginalization) into organized free-time activities, is the circle line ride model tested in the NOTKE project, and need to be published in high quality journals for quality assurance and impact. RUSE s RoSA-lab (Research Laboratory of Strategic Action) is described as innovation with its digital learning environment and a unique space created for integrating social sciences and ICT in order to develop new solutions and methods for research and teaching. 14. Special impact on the national and international scientific community 2010 2013 RATING: Good How would you evaluate the unit s impact on scientific leadership as well as on the national and international scientific community? Professor Kivinen has an excellent standing in the scientific community, but other members of the unit are nearly invisible. Concerning the important focus of training early stage researchers, there seems to be a lack of opportunities for the junior members of the unit to participate in external activities, both to gain experience and to learn from international outreach.
9 / 10 The unit would probably benefit from more outreach as well as from enabling other members of the unit to participate in external activities both nationally and internationally. 15. Research activity plan 2015 2018 RATING: Good Describe the panel s view on the future success of the unit, paying particular attention to the following points: Is the research activity plan feasible? How do you see the ratio of the planned investments and the aimed outcomes and impacts? Does the unit describe potential weaknesses that may affect reaching these aims? How are alternative approaches being considered? The self-assessment report indicates that RUSE has had more success in applying for external funding, although the budgets for the new projects starting in 2015 is not given. The ambitions of funding through Horizon 2020 are good, but will also require both infrastructure and effort in addition to already initiated projects. The activity plan does not discuss obstacles to the activities, and it is neither discussing how to develop the junior staff members to include them in the activities with increasing challenges. Estimate the potential of this research activity plan for significant new outcomes, scientific breakthroughs, scientific progress in this field, as well as its potential outcomes and impacts. The projects are mostly based on analyses of existing data, and with high value for planning and policymaking, but they do not represent high-risk, ground-breaking research. However, some of the projects that are conducted represents important information of value to the international community as well, and could probably justify a larger number of international, peer-reviewed publications in better journals publishing research in education compared to what is presented so far. Can the panel foresee any potential scientific breakthroughs not considered by the unit itself? Several of the studies they are working on do come up with results that should be of interest to the general and also the international society. If based on application of relevant theory, these studies could be published in good, international journals (A1) in education research. Estimate the planned national and international co-operation and networks in light of the aimed outcomes and impacts. SUBRATING: Sufficient
10 / 10 Estimate the potential of the described strategy for success in competition for funding. It is hard to read from the self-assessment document if they have a strategy for future development and success, but they continue to rely on funding from the ministry. However, as a significant part of the activities of the RUSE is depending on the leader, the structure is vulnerable, and it is advised that a backup plan for the continuation is developed. SUBRATING: Insufficient Did you find enough information for your evaluation of each topic in the unit s self-evaluation and the background material provided? If not, what was missing? Many sides of the activity of the unit was described in vague terms, and also for a unit like this, with a different funding model, the table work of the appendices were not sufficient as background.