WHY ARE SKILLS ASSESSMENTS REQUIRED IN TODAY S IT ORGANIZATIONS? Carol Gravel, EdD, MEd, SHRM-CP

Transcription

1 WHY ARE SKILLS ASSESSMENTS REQUIRED IN TODAY S IT ORGANIZATIONS? Carol Gravel, EdD, MEd, SHRM-CP

2 WHY ARE SKILLS ASSESSMENTS REQUIRED IN TODAY S IT ORGANIZATIONS? By: Carol Gravel, EdD, MEd, SHRM-CP Over the past few years, we have seen the mainstreaming of cloud computing as well as the increased use of disruptive technology such as artificial intelligence (AI). For example, a 2017 study by Forbes showed that hybrid cloud adoption has increased threefold since 2016, and an organization is using, on average, 29 different cloud solutions. These technologies make our lives easier and more productive while also contributing to the globalization of trade and commerce. Going forward, we will continue to see continuous changes in technology that will drive the need to change how we develop and assess skills. Gartner s Top 10 Strategic Predictions for 2018 and Beyond shows that the digital transformation will be even more disruptive as the relationship between people and machines will be blurred. Gartner suggests that to cope, organizations must develop practices to that will allow them to keep pace. For IT organizations, coping requires creating a New IT Organization that has new approaches to skills development and leverages new organizational models. In essence, keeping pace requires a workforce that is nimble and can quickly define the right skills and the right people to quickly respond to change on top of ever-changing technologies, especially those hosted in the cloud. These new approaches will provide IT the ability to do rapid skilling or reskilling of the IT workforce as well as the ability to quickly respond to business needs. The need to skill or reskill the IT workforce is urgent. A 2018 McKinsey Global Institute report revealed that by 2030 as many as 375 million workers, approximately 14 percent of the global workforce, will change because of digital transformation, automation and advancements in artificial intelligence (AI). Sixty-six percent stated that addressing potential skills gaps related to automation/digitization is a top-ten priority while almost 30 percent stated it was in the top-five. (See graphic on next page.) 2

3 Many innovative IT organizations are using agile teams as a new organizational model. These teams are dedicated, small, cross-functional groups of people who have many or all the skills the team needs to produce a working, tested increment of a product or service and subsequently a fully functioning product or service. It is important to note that there is no one model of an agile organization. Each organization must define what is appropriate based on their internal and external needs. However, the agile workforce must be deliberately designed to gather and deploy quickly and efficiently in response to requirements. There are two key definitions of an agile organization: Agile: Demonstrating the ready ability to move with quick, easy grace and having a quick, resourceful and adaptable character. (Source: Webster s Dictionary) Agility: The ability of an organization to sense environmental change and respond efficiently and effectively to that change. (Source: Gartner) There are two key attributes of great agile teams; 1) continuous mentoring and 2) continuous skills development. One of the largest benefits in working on a true agile team is that members learn from and mentor each other. As a result, the agile team is greater than the impact made by individual members. However, it is still vital that members have formal structured learning because it is critical these teams have the scope and depth of skills needed to support each other s work. It also guards against groupthink that can lead the team down the wrong path. 3

4 Skill assessments provide an inventory of validated skills Finding or building the skills needed in the New IT Organization can be challenging. Recent research from CompTIA suggests that 8 in 10 US businesses are being negatively impacted by the lack of technology talent. In addition, an EU Commission report revealed a shortage of 900,000 IT professionals by Many organizations are creating skills inventory systems to track and analyze individual skills to not only properly staff agile teams, but also to identify skills gaps. Building a skills inventory can be broken down into three stages. Stage 1: Define the competencies (knowledge, skills and abilities) needed for each role and/or job family. It is vital that you look beyond the job description to define competencies. Consider using focus groups of high performing agile teams as input as to what competencies are needed. Then, for example, define competency levels from novice to expert. Stage 2: Define a skills inventory system. Once you have the competencies defined, you will need to organize them to search and analyze the data to determine staff and skills gaps. It is important to define standard terminology and organize the data to be able to produce effective queries. Several enterprise LMS s provide a competency/skills inventory module or you can build your own. For example, PwC has the Digital Fitness Assessment app. Employees assess their own technology skills and create personal learning paths to increase the scope and depth of their skills. Stage 3: Assess the skills. There are two common approaches to assessing skills. The first is self-assessment, and the second is formal assessment. Self-assessment involves having an employee or their manager assess their level of proficiency in each competency. With self-assessment, it is important to communicate the assessment s objective to staff because if employees are thinking this will be a performance assessment instead of a competency assessment for staffing, there may be challenges with getting accurate data. A formal assessment is a systematic preplanned method of testing to see if the employee has the competency or skills. Formal assessments can be tests using a set of questions or real-world situations or challenges. Two common types of assessments, activity-based and performance- based, are described in greater detail below. It is important to note that each stage is not a one-time activity. Technology is always changing, so it is important your skills inventory also is adaptive to change. I recommend learning leaders use an iterative approach to maintaining the skills inventory. For example, in tandem with performance review cycles, learning leaders collaborate with IT to review competencies to ensure they are up to date. Then, adjust the skill assessments as needed. 4

5 Skill assessments allow organizations the ability to: Assign individuals to teams or projects based on knowledge and/or necessary skills With the skills inventory in place, you can perform simple or complex analyses of the skills data. From a simple query of who has Java skills to a more complex query of who has Java skills that are available to start in one week on a project that requires them to work in Northern Ireland. Beyond queries of who has what skills you can also see where there are skills gaps in terms of current or future technology. For example, IBM has a tool called Talent Match that allows leaders to identify employees with specific verified skills. Employees register to be part of the Talent Match system. Their verified credentials/assessment results are then available, so a leader can search the system to locate individuals with specific skills. Create a learning culture It is important to note that IT professionals in the New IT Organization must possess the willingness and desire to continuously learn new skills. With the skills inventory in place, you can support a continuous learning culture in your organization. In a learning culture, the employee, mangers and teams are responsible for seeking out the knowledge or skills they need to best perform their work. In addition, knowledge and skills are shared among teams and, thus, creates a more agile organization. By having a skills inventory, individuals, managers and teams can see who in the organization they can reach out to for support and/or coaching. HOW TO GET STARTED Define a skills framework Having an inventory of skills is vital data for the New IT Organization. However, the key first step is to define a skills framework that you can use to classify and organize those skills. As a former Director of IT Training, it was important for me to answer two key questions associated with assessing skills to determine what training may be needed: 1. How can we objectively measure achievement of non-technical skills? 2. How can we effectively assess the quality and output of IT technical skills? 5

6 The T-Shaped skills framework is an ideal framework. The vertical line on the T represents the depth of skills needed. The horizontal line on the T represents the breath of skills needed. The T-Skills Framework suggested includes 3 skill domains. 1. Technical 2. Interpersonal 3. Process 6

7 IT professionals in the New IT Organization must still have an advanced level of technical or IT skills. However, they also need foundational skills in processes like Agile or DevOps. In addition, they must also possess at least a foundational level of interpersonal skills such a teamwork. Assessing technical and non-technical skills require different assessment criteria. Interpersonal skills are assessed based on feedback on attitudes and behaviors. IT technical skills are is generally assessed in terms of the quality and output of the task for example, the quality of the lines of code for software developers. Typically, IT organizations pay little attention to teaching IT professional process or interpersonal skills as it is often assumed technical skills are the most important. However, in the New IT Organization IT professionals need to be able to work collaboratively in teams. For example, a software developer requires a depth of technical skills, such as full-stack development (e.g. Java,.Net, and Node.js). However, writing code requires more than just proficiency with a specific language; it requires a breadth of interpersonal skills, such as problem solving and communications, as well as a breadth of knowledge of process skills, such as Agile. Define a skills assessment strategy There are two steps in defining a skills assessment strategy; what types and when. First, define what types of assessments are important to your organization. 1. Knowledge-based assessments Knowledge-based assessments are a traditional part of learning design. When designing learning, we begin by creating learning objectives that are typically based on the six levels of Bloom s Taxonomy. Level 1: Knowledge Ability to recall information Level 2: Comprehension Ability to interpret information Level 3: Application Ability to apply information in a new way Level 4: Analysis Ability to break down information and show relationships Level 5: Synthesis Ability to bring together information to solve a problem Level 6: Evaluation Ability to make effective judgments based on information We then create knowledge-based assessment questions on those objectives. For example, A: Knowledge Question: Define B: Comprehension Question: Explain the relationship between C: Application Question: Could the methods be used to? Explain your answer. D: Analysis Question: Compare and contrast. E: Synthesis Question: Create a. that could. F: Evaluation Question: Evaluate in terms of 7

8 2) Performance-based assessments In general, a performance-based assessment measures a person s ability to apply the skills and knowledge learned. Typically, a performance-based assessment is an activity or challenge to complete. Performance-based assessments can range in complexity level from a simple response, such as a short answer, to a scenario where they need to provide a sample solution. The scenario requires a higher-level of thinking because students must create or do something to achieve success. The most effective performance-based assessments are ones that closely mirror real world situations. Key characteristics of performance-based assessments: 1. Complex 2. Authentic 3. Process/product-oriented 4. Open ended 5. Time-bound Example of an Assessment Activity: Activity: Create a few storage accounts in a Cloud Subscription that will be used later in the lab. The instructional designer creates a PowerShell script (see below) that will check if the storage accounts were created correctly. This script is then used to assess if the student has created a storage account correctly: $storaccount = Get-AzureRmStorageAccount ` -ResourceGroupName CSST@lab.LabInstance.GlobalId ` -Name sa@lab.labinstance.globalid ` -ErrorAction Ignore if ($storaccount -eq $null){ $result = $false The Storage Account has not been created. } else { $result = $true You successfully created the storage account. } $result 8

9 This is what the student will see in the lab: The student clicks the Score button, and the scripts will begin executing: If the student created the storage accounts correctly, they will receive a message that says You successfully created the storage account. If the student did not create the storage accounts correctly, they will receive a message that says The Storage Account has not been created. Example of an Assessment Challenge: Challenge Scenario: Can you Provision Data Storage for a Web App? You will be asked to complete a large configuration task to showcase your mastery of the topic, which could span multiple products, technologies, and platforms. You will not receive task guidance, and an automated score will be provided to you upon completion. In this challenge, you will provision all data components for a line of business application. This includes Azure storage, Azure SQL database, Cosmos DB, and search. Tasks: 1. Create and populate a storage account with blob containers. 2. Provision an Azure SQL database 3. Provision an Azure Database for MySQL server 4. Provision an Azure Cosmos DB database For a live demo see: 9

10 It is important to note that the traditional lecture/lab/test approach to training design will not help the New IT Organization quickly adapt to rapid changes. IT training must adopt new instructional design models that include challenge-based or project-based scenarios where learners learn by doing and receive near real-time feedback and assessment on their proficiency in completing the scenario. 10

11 Second, define when you want to use assessments. The most common times for conducting assessments are prior to training (pre-assessment), during training (formative assessment) and after training (post-assessment). Pre-Assessment: An initial assessment to gather data on current knowledge and/or skills. This is important because it provides you with baseline data that you can use to: a) Determine if there is a gap in skills. If there is a gap, identify specific training to fit their needs. b) Have an inventory of what skills an individual already has so you can assign them to projects or teams. Formative Assessment: An assessment to gather data during training to validate understanding of the material. If they do not understand, then re-teaching using a difference approach or repeating the lesson may be needed. Post-Assessment: A post training assessment to validate the new knowledge and/or skills acquired from training. This information is then available in your skills inventory. For example, an assessment strategy to validate that someone can define the elements of the Agile Manifesto would be to have a pre-assessment, a formative assessment and a post assessment. KNOWLEDGE/SKILL WHEN TO ASSESS WHAT TYPE OF ASSESSMENT Pre-Assessment Activity-Based: Level 1 Multiple Choice Define the elements of the Agile Manifesto Formative Assessment Post-Assessment Activity-Based: Level 2 Matching Performance-Based: Scenario as a project manager explaining Agile Manifesto to their team 11

12 CLOSING IT organizations must develop organizational practices, such as skills assessments and the use of agile teams, to keep pace with the inevitable digital transformation. In the next five years, the demand for IT talent will significantly outstrip supply. For agile skills, the demand is expected to be four times the supply. For big-data talent, it is projected to be 50 to 60 percent greater than anticipated supply. It will require IT organizations to look at new organizational models and approaches to skills development. Then, use that data to create agile teams to be able to scale up and down to respond quickly. To address skill gaps, IT organizations are also facing the requirement to reexamine the traditional lecture/lab/test approach and adopt new instructional design models like challenge-based or project-based scenarios where learning by doing incorporates near real-time feedback with limited guidance to encourage applied learning by experimenting. In other words, rapid changes in technology require the New IT Organization to take a different approach on talent management. Focusing on skills assessment and learning by doing will be key tools in meeting the challenges of the digital transformation. 12