Metrics That Matter Integration Process Overview An overview of the standard timing, resources, and tasks required to establish an integration from an LMS to Metrics That Matter in a typical organization Metrics That Matter
Contents Integration Options... 3 File Delivery and Conversion... 4 File Processing Times... 5 Project Teams and Roles... 5 Configuration and Testing... 7 Live Test and Go Live... 8 Post Go Live Ongoing Support... 8 Standard Integration Timelines... 8 Frequently Asked Questions: MTM Integration... 9 About Gartner... 10 About Metrics That Matter... 10 Contact Us to Learn More... 10 2017 Gartner. All rights reserved. 2 gartner.com/ceb
Overview and Purpose Many organizations choose to integrate their Learning Management (LMS) system with Metrics that Matter (MTM) in order to reduce administration and further automate the evaluation delivery process. Integration is typically accomplished by establishing an automated data extract from the LMS that sends class information to MTM on a nightly basis. As a result of the nightly data extract, classes are scheduled and survey links are created in MTM without any manual intervention. Integration Options Metrics That Matter offers two integration options based on the types of data elements included in the file feed from the LMS to MTM: Basic and Comprehensive. A Basic integration focuses upon the core elements of the class configuration (Instructor, Location, Course, etc). The Basic integration is best suited for organizations which are interested in providing MTM with only the most essential learning event data or those whose LMS does not contain the additional metadata which is best suited to a more in-depth data feed into MTM. A Comprehensive integration adds additional information such as student language preference, business unit, and course hierarchy associations for organizations that desire additional flexibility, customization, and can send the required information from the LMS into MTM.. Below are the fields which are to be present in both integration types. For a description of each field please click this link. Field Basic Comprehensive Unique Class XID Learning Provider XID Learning Provider Name Class Start Date Class End Date Instructor XID Instructor First Name Instructor Last Name Instructor Email Course XID Course Name Course Type Couse Description Location XID Location Name Time offset Learning Method ID Student XID Student First Name Student Last Name Student Email 2017 Gartner. All rights reserved. 3 gartner.com/ceb
Manager Full Name Manager Email DemographicValue1-7 Measurement Plan Name MTM Post Event ID MTM Follow Up ID MTM Instructor ID MTM Manager Survey ID Student Client XID Student Client Name Student Business Unit XID Student Business Unit Name Cancel Student Language Region Hierarchy XID Region Hierarchy Name Course Hierarchy XID Course Hierarchy Name Both integration options require that the data be provided in a format that supports the MTM import schema. Metrics That Matter will provide the Client with an extract template to be used to convert, test, validate and finalize as valid and reliable data to import into MTM. Should the Client LMS or Client requirements extend beyond the ability to provide information conforming to the extract template, a Custom integration will be scoped and additional charges may apply. Additionally, should the Client need/request services from the LMS provider to fulfill its obligations during the integration, the LMS provider may charge a fee for such services which would be the responsibility of the Client. Integration Source File Options Basic and Comprehensive integrations are designed to accept source data file from the LMS to MTM in an extract file in a delimited CSV format. This data should be delivered in UTF-8 format. Custom integrations will support standard reports from the LMS, extract files in CSV format which cannot adhere to the MTM required column ordering, or an extract file in XML format. Your MTM support team will discuss these options with you in order to determine the best approach for your organization based on business requirements, availability of technical resources, and security requirements. File Delivery and Conversion Files are delivered to a dedicated SFTP site for processing. Files may also be delivered via a web service for an additional configuration fee. PGP Encryption is supported if required. Specific file posting times are needed to automate the decryption process prior to processing files. 2017 Gartner. All rights reserved. 4 gartner.com/ceb
Metrics That Matter's native file format is XML; however, CSV is the most universally accessible extract format from LMS systems. As a result each CSV file must be converted to XML prior to MTM processing. XML conversion is often referred to as processing framework conversion or simply "executing the framework". This task must be completed prior to an organization's scheduled MTM processing times and is scheduled to occur at a set time each day for an organization based upon pre-defined business rules. File Processing Times The Metrics That Matter file importer processes XML files three times daily: 3 AM, 8 AM, and 10 PM (US Central time). Your MTM project team will recommend a file processing time based on your organization s scheduled CSV delivery time/ conversion to XML and desired survey email distribution time. Project Teams and Roles In order to facilitate a successful integration, you will require the coordination of several resources across your organization. Our most successful clients assemble the appropriate resources and then clearly define individual accountabilities. Here is a sample team organization. Your team and time allocations may vary based on the type of integration selected, the source file format selected, and other project needs. Client Project Team Members: Resource Accountabilities Typical Time Required Project Manager Overall project coordination Coordinate Client team support as needed Project Sponsor Champion Project & validate decisions 2-5 hours per week throughout project duration 2-4 hours per month Provide Resources to support project Additional Client Functional/Business Resources (as appropriate) Subject matter experts on current evaluation process and desired future state 1-3 hours per week throughout project duration Define/redesign process 2017 Gartner. All rights reserved. 5 gartner.com/ceb
and workflow as needed Test/utilize product/processes during and post implementation Client Technical Resource Write CSV extract to meet requirements, support testing and go live Depends on type of extract provided, technical expertise of resource, and complexity of requirements Metrics That Matter Project Team Members: Resource Implementation Consultant Accountabilities Ensure appropriate documentation of process and technical requirements, based on established MTM business rules Provide subject matter expertise on best practices Technical Consultant Manages the integration project Responsible for technical configuration and testing Provide subject matter expertise on technical processes Operations Director Sales Representative Oversees project from a high level to ensure high quality and Client satisfaction Responsible for overall Client relationship Manages contractual items Process and Key Activities Your Metrics That Matter Integration project will be divided into four key milestones: 2017 Gartner. All rights reserved. 6 gartner.com/ceb
Requirements Finalization Business rules review and technical requirement finalization to determine integration approach Receipt of Output File from LMS Project officially kicks off. Client provides output file based on technical requirements Configuration and Testing MTM confirms file format and completes configuration to ensure accurate nightly upload Live Test and Go Live Live deployment of the automated nightly process Requirements Finalization The requirements finalization phase includes activities related to the review and finalization of business rules, which drive the final technical requirements for the integration file. MTM will provide guidance regarding required fields, based the needs of the Client, and will confirm which integration approach is most appropriate. This process normally occurs prior to the purchase of an LMS integration and is generally 60 minutes in duration. In some cases, MTM may incorporate additional logic to convert data available in the LMS into a useful format. When applicable, this will be documented in the requirements documentation. Ongoing update and maintenance of unique tables required to support this logic is not included in the standard integration fee and will be scoped on an as-needed basis. Receipt of Output File from LMS The project is formally launched once technical requirements have been finalized, the solution confirmed in a statement of work, and the Client provides an output file that meets the agreed upon requirements. MTM then conducts testing to ensure that the file contains all fields required for integration and provides written feedback regarding the pass / fail status of each field and / or file. Configuration and Testing may not begin until the Client has supplied this initial test file. Configuration and Testing MTM confirms that the file format is viable and meets all requirements, and completes the necessary configuration to ensure accurate upload of the data file into MTM by building a utility that converts the data to a standard XML schema for upload. Finally, testing is then managed internally with MTM ensuring that default surveys and email forms are configured properly to support the integration after go live. Credentials to support secure file delivery to a dedicated sftp location are provided in preparation for the live test and go live. 2017 Gartner. All rights reserved. 7 gartner.com/ceb
Live Test and Go Live After successful testing by the MTM team, a go live date is set and the first live file is generated from the Client LMS for upload to MTM as a live test of the process. MTM recommends that the first live file includes a subset of classes. After successful processing of the first live file, the Client activates the automated process to generate nightly files from the LMS. The process is monitored closely until all parties are confident that the integration is functioning as expected - typically through the first week. Post Go Live Ongoing Support A daily import log will be generated and sent via email to Client contacts as determined by the Client. The daily import log will contain a summary of the status of the file (import successful, import successful with errors, import failed) and detail regarding the items included in the file and their status. In the event of a file failure, the daily import log will provide instructions regarding what element(s) of the file caused the failure and steps to correct the issue. In some cases, the Metrics That Matter team may need to perform additional research on the cause of a file failure; in these cases every effort will be made to communicate root cause and next steps within 1 business day. Once a failed file is corrected by the Client and re-posted, it will be imported during the next automatic import cycle. Requests to manually import files may be billed at $225 per hour (one hour minimum). Upgrades to the LMS that impact the extract or the addition of new groups/business units to the MTM solution should be communicated to MTM and may be subject to additional testing fees; this also applied to upgrades that materially impact the file structure such as changing to a different LMS which will require a new integration. Please notify Metrics That Matter of any planned upgrades or new LMS installations as soon as possible in order to reduce the risk of gaps in the survey data collection process. Standard Integration Timelines Metrics That Matter is committed to providing you with a high quality integration project experience that results in an automated process that can run virtually untouched for years to come. The typical project timeline is included below, and may vary based on the technical specifications of your project / organization. Integration Timeline: Timing Task Responsible Pre-Sale Requirements Gathering and Business Both Rules Definition Week 1 Initial Test File Provided to MTM Client Week 2 Configuration and Setup MTM Week 3 Live Test/End-to-End Pilot Both Week 4 Go Live Both 2017 Gartner. All rights reserved. 8 gartner.com/ceb
*Note: For comprehensive integrations, 2 additional weeks are required for testing (3 weeks of testing total) Frequently Asked Questions: MTM Integration What kind of information will be sent from the LMS? The minimum requirements include class information such as the course, location, instructor, start date, and end date. Metrics That Matter will provide a requirements document that includes all of the information that needs to be included in the output file in order to meet your business requirements and will review each item to confirm availability and location in the LMS. What types of file transfer are available? Can we send encrypted files to MTM? MTM accepts delivered files to our MTM-hosted SFTP. Web Services are supported for certain LMS providers only and may incur an additional charge. PGP encryption is supported with additional scoping/ configuration. What are the key risk factors or pitfalls that can negatively impact the project timeline? There are many potential risk factors that may impact the project timeline; the most typical and disruptive are listed below. Client s ability to complete their responsibilities within proposed project timeframe particularly the ability to deliver the first test file from the Client LMS. Lack of Client dedicated resources Complex rules or business processes Business process change requiring significant organizational change management It s important to note that deviation from the defined scope (including significant timing delays) may necessitate additional discovery to validate scope, assumptions and resource requirements and will incur additional costs. 2017 Gartner. All rights reserved. 9 gartner.com/ceb
About Gartner Gartner (NYSE: IT) is the world's leading information technology research and advisory company. It is known for analyst-driven, syndicated research and advisory services in information technology (IT), marketing and supply chain. Like CEB, Gartner has a rich history and culture that is both collaborative and client-centric. Gartner was founded in 1979 and is headquartered in Stamford, Connecticut, USA, with almost 9,000 associates worldwide. Its clients are located in more than 90 countries and are highly diversified across geography, industry and size. For more information, visit gartner.com. About Metrics That Matter The standard in learning analytics, Metrics That Matter from Gartner enables more precision in strategic talent decisions, moving beyond big data to optimizing your workforce learning investments against the most businesscritical skills and competencies. Our cloud-based talent analytics software and proven methodology enable organizations to run learning and talent like a business by establishing the right metrics, comparing against external benchmarks, and leveraging automated insights. Over 500 organizations utilize Metrics that Matter, including the most widely recognized organizations for talent development. Contact Us to Learn More Phone: +1-800-561-3341 or +1-312-676-4400 Email: MTMsolutions@executiveboard.com Web: gartner.com/ceb 2017 Gartner. All rights reserved. 10 gartner.com/ceb