Canadian Occupational Performance Measure Report 2015

Transcription

1 Canadian Occupational Performance Measure Report 2015 Noah s Ark Inc Malvern Rd, Malvern VIC Australia 3144 T (03) F (03) Published by Noah s Ark Inc 2015

2 Page 2 INTRODUCTION Noah s Ark has used the Canadian Occupational Performance Measure (COPM) to measure performance and quality of services provided to families since This data provides information about parents perception of changes in performance and satisfaction over time across a range of priority areas. The COPM, along with the Family Survey and Family Feedback exit interviews, are used to gather feedback and inform Noah s Ark s Quality System. The COPM is a valid and reliable outcome measure which is administered using a semistructured interview and structured scoring framework. Participants rate their performance and satisfaction against each priority area using a scale of These scores are then generated into an overall average score for performance and satisfaction. The score changes between initial assessment and reassessment can be used to measure change over time. It is intended that the COPM scoring process is completed with each family during the development of their Family Service and Support Plan (FSSP). FSSPs are typically reviewed every six months. At this point the second COPM ratings are completed. This report presents results from 2015 COPM data, with scores collected between October 2014 and September RESULTS COPM data was collected from the 18 early childhood intervention service (ECIS) teams across Noah s Ark. It should be noted that there is a variance in time periods in which data was collected between teams. Some teams provided data from September 2014 to September Other teams provided data from January 2015 to September This should be considered when interpreting the results. Variations in data collection periods may mean that the sample of scores is not totally representative of trends across the organisation. Although comparisons have been made across teams for the purposes of this report, it is important to note the discrepancies in data collection periods in analysing the results. The completion rate of COPMs continues to be low. It appears that only 44% of families had their COPM scores recorded. However, this data may not be completely accurate as some of the families counted as part of caseload numbers may not have been with Noah s Ark for more than six months which would mean their plans had not yet been reviewed and second set of data not yet collected. Refining data collection methods, including introduction of a system of unique identifiers, will assist in completing a more accurate analysis of completion rates in future years.

3 Page 3 The number of COPMs completed is similar to This year 716 COPM scores were recorded, compared to 771 in 2014, 307 in 2013 and 827 in 2012 (see table below). There was a significant drop in scores recorded in 2013, with numbers reasonably similar across all other years in which data has been collected. There was a slight drop in 2015, despite there being a new team in the organisation and an overall increase in client numbers as a result. Year Number of COPMs Caseload numbers % COPM completed % % % % Table 1. Comparison of COPMs completed STATE-WIDE RESULTS The frequency of performance and satisfaction score changes across all 18 teams has been analysed as represented in the graph below. The score changes in 2015 are largely clustered between -1 and +5 for performance and -1 and +7 for satisfaction. The majority of score changes (75% of performance score changes and 70% of satisfaction score changes) fell in the range of 0 to +4.

4 Page Performance score change Sa<sfac<on score change < Figure 1. Frequency of performance and satisfaction score changes across Noah s Ark teams Almost all families (96.3%) reported improvement in the performance of their child against the goals they identified in their FSSP. The rate of improvement in satisfaction was also high, with 93.8% of families reporting improved satisfaction in their child s performance. The performance and satisfaction score changes were evenly spread between the 1 2 and 2 3 scoring intervals. Of the data provided, 22.8% of performance score changes fell in the 1 2 range and 22.3% fell in the 2 3 intervals. Satisfaction scores were similar with 22.3% falling in the 1 2 intervals and 22% falling in the 2 3 score range. Across the whole organisation the average change in both performance and satisfaction between time one and time two was 2.9. It should be noted that the COPM Manual specifies that a difference of two points or greater is required to differentiate between clients who improve and those who do not. All Noah s Ark teams recorded a positive average change in performance and satisfaction between times one and two.

5 Page 5 YEARLY COMPARISONS COPM data has been collected across Noah s Ark since The number of COPM scores completed was similar to the number received in The low number of scores received in 2013 (approximately 22% of families) should be noted as this sample may not be truly representative of general trends across the organisation. There continues to be a trend of low numbers of COPMs completed in general, when compared to the number of families serviced by Noah s Ark. The frequency of scores across 2013, 2014 and 2015 can be compared through the use of percentage calculations to account for variance in the total number of COPM scores recorded. When looking at the 2013, 2014 and 2015 performance scores it is evident there is a shift to more scores falling in the 2 3 intervals in There is a shift in frequency of performance scores toward higher intervals than in previous years, as represented by the orange line in Figure 2 below. The 2015 data was reasonably consistent with 2013 and 2014 in terms of frequency of performance scores falling below 0 and beyond the +3 range. This shift toward more scores falling in a higher interval range is positive, as it indicates a general trend of more families reporting greater overall improvements in their child s performance Performance 2014 Performance 2015 Performance < >10

6 Page 6 Figure 2. Comparison of frequency of performances scores for 2013, 2014 and 2015 Similar trends are evident when comparing frequency of satisfaction scores as evident in Figure 3 below. In 2015 more scores fell in both the 1 2 range and 2 3 range than previous years, indicating more people had higher levels of satisfaction with the changes in their child s performance. The 2015 data was reasonably consistent with 2014 in terms of frequency of satisfaction scores below the -2 and beyond the +3 range Sa<sfac<on 2014 Sa<sfac<on 2015 Sa<sfac<on < >10 Figure 3. Comparison of frequency of satisfaction scores for 2013, 2014 and 2015 The organisation wide average scores for performance and satisfaction in 2015 were consistent with the 2014 averages. There was a slight improvement in the average satisfaction score change, increasing from 2.8 to 2.9. Performance stayed consistent at 2.9. This indicates the overall trend of improvement from 2012 may be slowing. It is interesting to note that although the distribution of improvement in performance and satisfaction has moved more to the 2 3 interval in 2015 the overall average scores have remained stable.

7 Page Average change performance Average change sa<sfac<on Figure 4. Comparison of average performance and satisfaction over time KEY FINDINGS Almost all families reported an improvement in performance of their child against the goals they identified in their FSSP. Almost all families reported an improvement in their satisfaction with their child s performance. The average scores for performance and satisfaction across the organisation remained consistent with 2014 data. From an upward trend was observed, indicating improvement in outcomes and quality of service provided to families. This has been maintained in Although the most common score change still fell in the 1 2 intervals there was a general trend of more scores falling in the 2 3 intervals than in previous years. This indicates an upward trend in the number of families experiencing higher levels of improvement in performance and satisfaction than in There were discrepancies in the time periods for data collection between teams.

8 Page 8 RECOMMENDATIONS Implement processes to ensure all teams consistently record and submit COPM data. This will lead to more reliable information on outcomes for families and assist in accurately measuring quality of service provided. Introduce unique identifiers to assist in data analysis (noting when a child has two sets of COPM scores in a 12 month period and accurately checking the number of COPMs completed against team numbers. A system of unique identifiers may also help to cross reference information across different projects).