APPENDIX H KEY PERFORMANCE INDICATORS AND SERVICE LEVEL AGREEMENTS

Transcription

1 APPENDIX H KEY PERFORMANCE INDICATORS AND SERVICE LEVEL AGREEMENTS 1

2 Key Indicators Instructions This document defines the Service Level Agreements (SLAs) and Key Indicators (KPIs) that the vendor is required to meet. SLAs are intended to document clear accountability for the contract by defining the service level metrics, target goals for the metrics and the penalties for not meeting the goals. KPIs monitor business activities and show management the efficiency and effectiveness of the vendor s efforts on this contract. In the event the vendor fails to meet one or more KPIs, PennDOT may require the vendor to complete a corrective action plan. Depending on the frequency of non-compliance with a KPI and / or the degree to which the vendor falls short of the target, PennDOT reserves the right to convert the KPI to an SLA with an associated penalty for future non-compliance. SLA and KPI performance will be actively monitored by the vendor. The vendor will include a quarterly summary KPI and SLA performance as part of the Quarter Status Report (see Task 10h Meetings) and should also show trends for each KPI and SLA since the project inception. SLA performance and any associated service credits should be reflected on each invoice the vendor submits. Note: Italicized words and phrases are defined at the end of this appendix. Page 2 of 11

3 Table 1 - Key Indicators (KPIs) No. Target Definition Calculation KPI-1 Deliverables Submitted On Time All deliverable(s) completed by the agreed upon due date. This KPI measure the vendors adherence to the agreed upon schedule. Received by 4:00 pm EST on the deliverable due date (the KPI should be evaluated for each deliverable submitted in the quarter) Page 3 of 11

4 No. Target Definition Calculation KPI-2 Deliverable Quality Each deliverable should have no more than 3 grammatical errors per page. This KPI is intended to measure the effectiveness of the vendor s quality assurance practices associated with the preparation and submission of deliverables. Total number of grammatical errors / total number of pages in the deliverable (the KPI should be calculated separately for each deliverable submitted in the quarter) Page 4 of 11

5 No. Target Definition Calculation KPI-3 Cumulative Unplanned Downtime Statewide Less than 4 hours of Unplanned Downtime per month. This does not include Scheduled Downtime. The KPI assesses the stability of the vendor s system and the vendor s ability to quickly recover from a system outage. Cumulative Unplanned Downtime in a month should reflect the total agency hours the system was down. For example, if two agencies are down simultaneously for 30 minutes, this should be counted as 1 hour of Unplanned Downtime. The assessment of this KPI should be based on data from the Ticket Tracking System. (the KPI should be evaluated for each month in the quarter) KPI-4 User Satisfaction with Help Desk 90% satisfaction with help desk support This KPI assesses agency satisfaction with the vendor s help desk support. Number of users who indicated they were satisfied or very satisfied with the support they received/total number of support satisfaction surveys returned. The assessment will be based on the satisfaction survey results stored in the Ticket Tracking System. (the KPI should be evaluated for each month in the quarter) Page 5 of 11

6 No. Target Definition Calculation KPI-5 Minimal Equipment Failure <1% of agency vehicles removed from service due to equipment failure or malfunction each month For the purposes of this KPI, the reliability of FR- ITS hardware will be based on the number of times a vehicle needs to be taken out of service as a result of equipment failure or malfunction. This statewide assessment considers all vehicles for agencies using the FR-ITS. Number of times a vehicle is taken out of service each month for any period of time/total number of vehicles for agencies using FR-ITS. The assessment will be based on the data in the Ticket Tracking System. (the KPI should be evaluated for each month in the quarter) KPI-6 Releases On time 100% of agency upgrades performed on time Measures adherence to the schedule provided in the release plan. Compare actual agency deployment dates to dates in the release plan. Page 6 of 11

7 No. Target Definition Calculation KPI-7 Release Quality < 5 Minor Defects and <1 Major Defect identified following the implementation of a planned software release. This KPI measures the quality of software releases. It reflects the effectiveness of the vendor s testing protocols. The total number of Minor Defects and Major Defects identified following a planned software release. This applies to all software components of the system including the CAD / AVL software and scheduling software. The assessment will be made based on the data in the defect tracking system. (the number of defects is cumulative following each release but should be updated and reported quarterly) KPI-8 Staff Turnover <1 project team member replacement or reassignment per quarter The vendor should seek to retain qualified staff and not reassign staff who have been committed to the project. The number of project team members who leave the project for any reason. Page 7 of 11

8 Table 2 - Service Level Agreements (SLAs) No. Target Definition Calculation Service Credit SLA-1 Implementations Completed On Time Agency implementations will be completed in accordance with the schedule in the preliminary implementation plan or statewide implementation plan For the purposes of this SLA, determining whether an agency implementation was completed on time will be based on the submittal date for the agency close out report. Agency close out report submitted by the due date specified in the preliminary implementation plan or the statewide implementation plan. Monthly 10% total agency implementation costs SLA-2 Downtime Per System Outage No more than 2 hours downtime per System Outage For the purposes of this SLA, the downtime is any time one of the agency s system is completely down or critical system functions are unavailable Length of time an agency s system is down for each system outage based on data from the ticket tracking system Monthly See Table 3 SLA-3 Support Ticket Response and Resolution >90% conformance to the response and resolution targets for each of the 4 ticket priority levels (see Table 4) Measures the length of time to respond to support requests and the length of time to resolve support requests. This SLA will apply separately to each of the 4 ticket priority levels. For each priority level, the number of tickets which were within the windows defined in Table 4 / Total number of support requests. The assessment of this SLA will be based on the data in the Ticket Tracking System. Monthly 5% of the statewide monthly maintenance and support charges Page 8 of 11

9 Table 3 Incident Duration Service Credit Incident Duration Service Credit (% of monthly support and maintenance costs for the agency) < 2 hours 0% 2 hours and < 4 hours 5% 4 hours and < 8 hours 10% 8 hours 20% Table 4 - Service Request Response and Resolution Times Priority Level Normal Business Hours Outside Business Hours Response Time Resolution Time Response Time Resolution Time Critical 15 minutes 2 hours 1 hour 4 hours High 2 hours 1 business day 4 hours 1 business day Medium 1 business day 5 business days 1 business day 5 business days Low 1 business day 10 business days 1 business day 10 business days Page 9 of 11

10 Definitions Business Hours Monday through Friday, 6:00AM to 6:00 PM EST Excluding Federal Holidays. Critical Priority Support Ticket A ticket associated with a System Outage. Downtime A period of time during which (1) the system or a key component of the system is completely down, or (2) a critical system function(s) is unavailable to one or more agencies or PennDOT users or (3) the system performance is so slow that users cannot effectively complete critical tasks. Downtime Period A period of ten consecutive minutes of constant or intermittent Downtime. Intermittent Downtime for a period of less than ten minutes will not be counted towards any Downtime Period. High Priority Support Ticket An issue which has a significant impact on system performance and / or which has a significant impact on the work of one or more agency or PennDOT staff. Low Priority Support Ticket A nuisance issue which is not causing a significant impact on system performance or interfering with the work of agency or PennDOT staff. Medium Priority Support Ticket A support ticket which does not have a significant impact on system performance but which has a modest impact on the work of a single agency staff member. Major Defect A defect with only significant impacts to system functionality and does not have a viable work around. A major defect generally requires an unplanned release in order to resolve the issue as soon as possible. Minor Defect A defect with only minor impacts to system functionality and / or one which has a viable work around. The impacts caused by a minor defect are minor enough that it does not need to be handled with an unplanned release but can wait until a future scheduled release. Scheduled Downtime - Periods of downtime related to network, hardware or service maintenance or upgrades. To be considered scheduled downtime, these periods must be scheduled at least 3 days in advance. System Outage An incident which results in Downtime. Unplanned Downtime Downtime that is not scheduled at least 3 days in advance. Page 10 of 11

11 A minor defect is improper behavior in the system which does not require an unplanned release to resolve. The defect is minor enough in its impacts that resolution of the defect can wait until the next planned release. A major defect is a defect which is significant enough that it warrants an unplanned release to correct it. Page 11 of 11