Light-touch evaluation of the Principles of Workforce Redesign. May we re helping employers to redesign their workforces

Transcription

1 Light-touch evaluation of the Principles of Workforce Redesign May 2012 we re helping employers to redesign their workforces

2 Contents Executive Summary 1 Introduction 3 Evaluation findings 8 Learning outcomes 11 Summary and conclusions 17 Light-touch evaluation of the Principles of Workforce Redesign Published by Skills for Care, West Gate, 6 Grace Street, Leeds LS1 2RP Skills for Care 2012 Reference no. R042 Copies of this work may be made for non-commercial distribution to aid social care workforce development. Any other copying requires the permission of Skills for Care. Skills for Care is the employer-led strategic body for workforce development in social care for adults in England. It is part of the sector skills council, Skills for Care and Development. This evaluation was undertaken by Jenny Swift, Project Manager (Evaluation) at Skills for Care, with assistance from Jim Thomas, Programme Head (Workforce Innovation), and Marie Lovell, Project Manager (Policy). Bibliographic reference data for Harvard-style author/date referencing system: Short reference: Skills for Care [or SfC] 2012 Long reference: Skills for Care, Light-touch evaluation of the Principles of Workforce Redesign (Leeds, 2012)

3 Executive Summary Background The principles of workforce redesign are a package of resources developed by Skills for Care with the aim of providing employers with the confidence and competence to remodel the way in which their employees work. A series of resources were launched between 2008 and 2011 and were promoted via Skills for Care s area-based staff, who work with employers locally and via networks. The materials were also promoted at Skills for Care conferences, events and exhibition stands. In January 2012 Skills for Care decided to carry out a light-touch evaluation of its activity around the principles of workforce redesign. The aim of the evaluation was: to elicit employer and stakeholder feedback on the principles of workforce redesign to assess whether the principles of workforce redesign work has fulfilled its aims and objectives. to understand how the principles of workforce redesign are working in practice to understand how employers have adapted the principles of workforce redesign tools to meet their own objectives. In order to undertake an evaluation that was cost effective and proportional to the scale of the project, we opted to conduct the evaluation in-house and limit the research method to a concise, post-test beneficiary survey, imposing the minimum burden on employers to respond. The survey was sent to 208 people who had downloaded various principles of workforce redesign materials in the six months prior to evaluation, and who had given consent to be contacted. Fifty-four responses were received. Given the post-test design, and the relatively small sample size, the evaluation is limited, particularly in relation to what we can say about additionality. It has however yielded valuable feedback which can be used to inform future Skills for Care practice. The key observations that can be made from the data include: Take-up of the Principles of Workforce Redesign has been reasonable, with penetration estimated at up to nine percent of the target market. A large majority of respondents felt that the principles of workforce redesign work meets its objectives. Employers were especially positive in their views on this topic. Reported confidence in relation to workforce remodelling grew significantly following the use of the principles of workforce redesign materials. Competence levels in relation to workforce remodelling also grew slightly. The mean distance travelled (using fourpoint Likert scales) was a moderate of a point on the confidence scale, and on the competence scale. 1

4 Reported learning outcomes appeared greater for those who started out less confident and competent in relation to workforce remodelling. 76% of respondents felt that the principles of workforce redesign materials had helped inform their practice; and 11% felt that they had been able to directly apply learning from the principles of workforce redesign in their workplace. The materials had been used by employers to support their workforce in transformation, and by commissioners to promote good practice to provider organisations. Overall, it appears that the principles of workforce redesign have been welcomed by employers, who feel that the materials have fulfilled their objectives. The principles of workforce redesign materials have assisted in boosting confidence (and to a lesser extent, competence) in relation to undertaking workforce remodelling, and have helped inform practice in a range of settings. In order to further increase take-up and widen the observed benefits, there is therefore a case to be made for further promotion of the principles of workforce redesign. Given the need to reduce unit costs, which are relatively high, this should be undertaken costeffectively, e.g. within Skills for Care s existing engagement work with employers. There is a case to be made for a targeted approach, identifying employers who may be more likely to lack confidence and skills in this area, or who are likely to be considering service redesign. For further information contact Jim Thomas, Programme Head - Workforce Innovation, Skills for Care on or jim.thomas@skillsforcare.org.uk 2

5 Introduction The Principles of Workforce Redesign were developed from Skills for Care s past work and learning around the new types of worker required for the changing workforce. The seven principles were developed to set out the key things employers need to take account of when changing the way in which their staff work, to meet the challenges of personalisation. The principles recognise that the quality of any service delivered by a social care organisation is directly linked to the skills, knowledge, expertise, values and attitudes of the people who make up the workforce. The principles also recognise that those employees have a significant role to play in any transformation that takes place. The Principles of Workforce Redesign are that employers should: 1. take a whole systems view of organisational changes 2. recognise how people, organisations and partnerships respond differently to change 3. nurture champions, innovators and leaders 4. engage people in the process - acknowledge and value their experience 5. be aware of the way adults learn 6. change minds and change systems 7. develop workforce strategies that support transformation and recognise the shape of resources available in the local community. Rationale, aims and objectives The Principles of Workforce Redesign grew out of Skills for Care s workforce innovation programme. At the time the programme included up to 20 small-scale innovation projects per annum and it was clear from the case material generated by these projects that the biggest barrier many of those organisations faced when trying to implement workforce innovation was culture change. The Principles of Workforce Redesign were a direct response aimed at helping people in those organisations to understand how to take a workforce development approach to managing transformation and transition. They were also a pragmatic response to those projects telling Skills for Care about techniques that they had found helpful. The aim was to provide employers with the confidence and competence to remodel the way in which their employees work in line with requirements for new ways of working. The objectives were: to set out a workforce development focused model to support employers with remodelling the way people work in the context of transformation to develop a practical framework that enables employers to use the principles of workforce redesign when changing the way in which people work. 3

6 Activities, outputs and costs The Principles of Workforce Redesign, with accompanying explanation, were printed in a free A4 promotional leaflet and published by Skills for Care in October 2008 both online and in hard copy. In 2009 this was followed with the Principles of Workforce Redesign: Notes and Resources, which outlined the management science underlying the principles. This too was published in hard copy and online. In 2011, Skills for Care published Using the Principles of Workforce Redesign to support the transformation of services in adult social care and support. This follow up document, with its focus on workforce development and involvement, was intended as a practical guide to help people who are engaged in reshaping services. It was available free to download on the Skills for Care website or could be purchased in hard copy (on a non-profit basis) for 12. It included a range of tools to help work through change constructively, transforming services to meet the needs and expectations of the people who use them. Its publication was timely, coming during a period when all providers of social care services are facing the dual challenge of reshaping their services and managing significant budget cuts. In November 2011, additional free online resources were added to the site: the Workforce Redesign Pathway a template managers can complete to help them work through the principles Practical Exercises in Workforce Redesign available to adapt and use for different organisations. The materials were promoted via Skills for Care s area-based staff, who work with employers locally and via networks. The materials were also promoted at Skills for Care conferences, events and exhibition stands. At end March 2012, there had been approximately 1,500 downloads of the Principles of Workforce Redesign. There are 17,320 adult social care employers in England 1, so if only one copy has been downloaded per organisation, this figure would equate to a penetration rate of nearly nine percent. In practice, it is likely that at least some multiple downloads have been made so the actual figure may be lower. 1 Skills for Care (2010) State of the Adult Social Care Workforce, p.16. This figure refers to organisations employing workers and excludes individual employers of PAs. 4

7 Table 1: Key outputs to end March 2012 Downloads Sales Total Principles of Workforce 1,500 2 n/a 1,500 Redesign Using the Principles of Workforce Redesign Notes and resources 193 n/a 193 Workforce Redesign 202 n/a 202 Pathway Practical Exercises in Workforce Redesign Between 121 and n/a Between 121 and 291 Table 2 below provides details of the costs to Skills for Care of the principles of workforce redesign work. This included the costs of production and printing, and supporting the vanguard sites to test the principles in a real life organisational change and provide feedback to Skills for Care. Assuming from Table 1 above that in the region of 1,500 organisations have benefited from the principles of workforce redesign to date, this intervention has come at a unit cost of approximately 130. This figure will decrease as take-up increases, if take-up can be promoted cheaply. Table 2: Inputs to the Principles of Workforce Redesign, Expenditure 2008/ ,000 Expenditure 2009/ ,000 Expenditure 2010/ ,000 Expenditure 2011/2012 5,000 Value of staff time (approximate) 60,000 Total 197,000 The evaluation In January 2012 Skills for Care decided to carry out a light-touch evaluation of its activity around the principles of workforce redesign. The aim of the evaluation was: to elicit employer and stakeholder feedback on the principles of workforce redesign materials to assess whether the principles of workforce redesign workhas fulfilled its aim and objectives. to understand how the principles of workforce redesign are working in practice to understand how employers have adapted the principles of workforce redesign tools to meet their own objectives. 5 2 Estimate as prior to 2011 page views rather than downloads were recorded. 3 Varies by each exercise.

8 As described above, the Principles of Workforce Redesign were first published in Supporting materials were added in the intervening time. Skills for Care has historically had an internet policy whereby web visitors do not have to register prior to downloading resources. This meant a lack of information about who downloaded the resources or how they were used. In November 2011 when the additional resources were added to the site, Skills for Care took the decision to ask visitors for some key details about themselves, prior to proceeding to download. By 5 th March 2012 this had resulted in a database of 353 visitors who had provided at least some details. Two hundred and eight of these (59%) had also provided an address and consent to be contacted for follow-up. Because we did not capture data on the downloads of the Principles prior to November 2011, we cannot compare the characteristics of the achieved survey sample to the total population, however we can compare it to the 353 visitors whose details were captured between November 2011 and March Table 3 below illustrates this comparison. It is apparent that there are some significant differences between the achieved sample characteristics and the larger group of visitors. This should be borne in mind in the analysis that follows. We did not re-weight the sample since, with a relatively small sample, it could distort the findings; moreover, there is no way to test whether the larger group of visitors are representative of the total population. An invitation to participate in an online survey was sent to all 208 in March A reminder was sent two weeks later and a final reminder at three weeks. In total 54 responses were received to the online survey, a response rate of 26% which is an excellent response rate for a survey of this type. 6

9 Other respondents comprised mainly consultants and researchers. As the principles of workforce redesign are aimed at establishments rather than individuals who pay for their own care and support service, there were no downloads by the latter group. To complement the survey, a number of qualitative case studies were carried out by Melissa Balman, an independent researcher. These are available at 7

10 Evaluation findings Use of the materials Table 4 below shows that the most popular download was the Principles of Workforce Redesign itself, with nearly 90% of the sample having downloaded it. In each case there was a gap between the number who had downloaded an item and the number who had actually gone on to read the material. Table 4 Use of the principles of workforce redesign materials N=54 Fit with original objectives Survey respondents were asked whether they felt that together, Skills for Care s collected work on the Principles of Workforce Redesign materials meets its objectives. That is, does it: set out a workforce development-focused model for transforming services support employers with remodelling the way people work set out a framework for workforce redesign that is practical enable employers to adopt the principles when remodelling the workforce? 8

11 Table 5 below shows the response. An overwhelming majority felt that the materials did meet the objectives set. Table 5 To what extent do you agree or disagree that Skills for Care s collected work on the principles of workforce redesign? (All responses) N=54 Only six percent or less disagreed in each case that the relevant objective had been met, and interestingly, these negative responses came only from the training providers, strategic and other groups in the sample. As shown in Tables 6 and 7 below, none of the employers in the sample felt that the materials had not met their objectives We need to sound a note of caution here however due to the small sample sizes. There were only 17 employers in the survey sample and the differences shown in Table 7 between employers and the wider sample are not statistically significant at 95% confidence levels.

12 Table 6 To what extent do you agree or disagree that Skills for Care s collected work on the principles of workforce redesign? (Employers only) N=17 Table 7 To what extent do you agree or disagree that Skills for Care s collected work on the principles of workforce redesign? Comparison between employers and all responses Employers only All responses Sets out a workforce development-focused model for transforming services Supports employers with remodelling the way people work Sets out a framework for workforce redesign that is practical Enables employers to adopt the principles when remodelling the workforce Agree or strongly agree Disagree or strongly disagree Agree or strongly agree 82% 0% 70% 2% 88% 0% 72% 2% 71% 0% 67% 6% 76% 0% 69% 4% N=17 N=54 Disagree or strongly disagree 10

13 Learning outcomes Thirty-two respondents in the sample (59%) stated that as part of their job role, they had responsibility for service transformation and/ or remodelling the workforce. This group were asked about their confidence and about their competence in relation to workforce remodelling, and how it had changed since using the principles of workforce redesign materials Table 8 below shows the results. The proportion who were quite or very confident in workforce remodelling grew from 47% of the sample (15) prior to reading the materials, to 72% (23) afterwards. Competence in workforce remodelling increased from 56% (18) of respondents feeling quite or very competent to 72% (23). 5 Table 8: Change in confidence and competence in relation to workforce remodelling, before and after using the materials N= The change in confidence reported is statistically significant (at 95% confidence levels). The change in competence reported however is not sufficiently large to register as significant with these sample sizes.

14 An alternative method of viewing these findings is shown in Table 9 below. This looks at the distance travelled by respondents up or down the four-point Likert scale used for the question: Please rate your (a) confidence (b) competence: before you downloaded the principles of workforce redesign [materials], and now. The scale offered four options: Not at all confident/ competent Not very confident/ competent Quite confident/ competent Very confident/ competent. For the purpose of the distance travelled analysis we have excluded those who answered Not sure or gave no response. This gives two small samples of 23 respondents each who answered the questions around confidence and competence in full. The table shows that just over half of respondents reported a positive change in both confidence and competence, but just under half did not report any positive progress on the scale. This group was mainly made up of respondents who said they were already quite or very confident or competent the four point scale was insensitive to small movements, e.g. within the range of confidence indicated by Quite confident. Only one respondent who said they were not very confident and not very competent before using the principles of workforce redesign reported no change. The mean distance travelled was therefore positive, but marginal, at of a point on the confidence scale, and on the competence scale. Table 9: Confidence and competence in workforce remodelling: distance travelled Confidence Competence No. respondents % Mean change No. respondents % Mean change Negative 0 0% 0 0% change No change 11 46% 11 48% Positive 12 55% 12 52% change: One point 9 38% 10 43% on the scale Two points 3 13% 2 9% Total % %

15 Given that this was of necessity a post-test design, it is difficult to demonstrate attribution, and the concise online survey form did not probe respondents on additionality, further to the before- and after- rating. However, respondents who said they had read the different materials were asked whether and how they had been able to apply the learning from the principles of workforce redesign. The responses to this question are shown in Tables 10 and 11 below. Positively, between 86% and 89% of respondents in each case felt that the materials helped inform practice, or that they were able to directly apply learning in the workplace. Table 10: Application of learning N=37,26,16,18,23 13

16 Table 11: Application of learning N=54 (30 responses coded from open question) 14

17 Practical uses of the principles of workforce redesign Respondents were asked in an open-ended question how they had used the principles of workforce redesign in action. Responses are summarised in Table 12 below. Table 12: Please describe how you have used the principles of workforce redesign in action summary of responses. The most common response was for research. Four respondents each said they used the principles of workforce redesign to influence service providers; to inform consultancy work; and to redesign services. Other responses included, to inform strategy; adopting the principles and values; influencing workforce engagement process and as a reference guide. Comments included: I have sent out the link [to the Principles of Workforce Redesign pages on the Skills for Care website] to our Provider Managers in our local authority area. I am a Social Care Consultant (Domiciliary) and have been able to pass this information on to providers who are making organisational change so that they can look at their business from within. - Consultant To support my role in providing guidance to provider organisations that need to improve their services or want to redesign their services to better meet the needs of service users. - Local authority commissioner - Local authority training department 15

18 [I used the principles of workforce redesign] to inform and support service currently in transformation. I am able to develop and deliver supportive learning and development solutions to support teams and individuals prepare for change and engage in the change. To support managers to identify [the] skills/knowledge gaps and competencies the workforce of the future may require. - Local authority [I used the principles of workforce redesign] to underpin [our] workforce strategy as well as [our] recruitment and retention strategy. - Strategic/ representative organisation. Adapting the Principles of Workforce Redesign Respondents struggled to describe how they had adapted the principles of workforce redesign, since most in the sample had only relatively recently downloaded it. The most common response given in relation to this question (Table 13 below) was that adaptation was a work in progress ). Table 13: How have you adapted the tools for your own purpose? summary of responses. N=54 (27 responses coded from open question) Other respondents had redesigned and incorporated materials into their own; or extracted key points to support the changes proposed. One respondent mentioned that they had adapted the materials to ensure there was equal awareness of other models; and one said they had not changed the materials at all. 16

19 Summary and conclusions This was a light-touch evaluation of the principles of workforce redesign. In order to undertake an evaluation that was cost effective and proportional to the scale of the project in expenditure terms, we opted to conduct the evaluation in-house and limit the research method to a concise, post-test beneficiary survey, imposing the minimum burden on employers to respond. The design is therefore limited, particularly in relation to what we can say about additionality. It has however yielded valuable feedback which can be used to inform future Skills for Care practice. The key observations that can be made from the data include: Take-up of the Principles of Workforce Redesign has been reasonable, with penetration rates of up to nine percent of the target market. A large majority of respondents felt that the principles of workforce redesign work meets its objectives. Employers were especially positive in their views on this topic. Reported confidence in relation to workforce remodelling grew significantly following the use of the principles of workforce redesign materials. Competence levels in relation to workforce remodelling also grew slightly. The mean distance travelled (using fourpoint Likert scales) was a moderate of a point on the confidence scale, and on the competence scale. Reported learning outcomes appeared greater, for those who started out less confident and competent in relation to workforce remodelling. 76% of respondents felt that the principles of workforce redesign materials had helped inform their practice; and 11% felt that they had been able to directly apply learning from the principles of workforce redesign in their workplace. The materials had been used by employers to support their workforce in transformation, and by commissioners to promote good practice to provider organisations. Overall, it appears that the principles of workforce redesign have been welcomed by employers, who feel that the materials have fulfilled their objectives of: setting out a workforce developmentfocused model for transforming services supporting employers with remodelling the way people work setting out a framework for workforce redesign that is practical enabling employers to adopt the principles when remodelling the workforce. The principles of workforce redesign materials have assisted in boosting confidence (and to a lesser extent, competence) in relation to undertaking workforce remodelling, and have helped inform practice in a range of settings. In order to further increase take-up and widen the observed benefits, there is therefore a case to be made for further promotion of the principles of workforce redesign materials. Given the need to reduce unit costs, which are relatively high, this should be undertaken costeffectively, e.g. within Skills for Care s existing engagement work with employers. There is a case to be made for a targeted approach, identifying employers who may be more likely to lack confidence and skills in this area, or who are likely to be considering service redesign. 17