Alexander Tolstoy, Natalia Miloslavskaya The National Research Nuclear University MEPhI (Moscow Engineering Physics Institute)

Transcription

1 Professional Competencies Level Assessment for Training of Masters in Information Security Alexander Tolstoy, Natalia Miloslavskaya The National Research Nuclear University MEPhI (Moscow Engineering Physics Institute) WISE9, Hamburg, 2015

2 Outline Introduction 1. Information Security Competencies (ISCs) and their Levels 2. ISCs Formation 3. Metrics for the Levels of Mastery of the Discipline 4. ISCs Level Assessment Model 5. Model Usage Example Conclusion

3 Introduction (1) Any professional must have specific qualifying characteristics (QCs). QCs for IS specialists: a set of their professional competencies (PCs) in the field of IS. PC: a professional s capacity to solve given problems and to perform specific work within his sphere of activity. PCs support definitions of job classifications/ occupational group profiles, roles and responsibilities, position descriptions, duty statements, etc. QCs can be worked out on the basis of some unified set of PCs by joint efforts of: 1) organizations personnel departments (including the role identification and assignment, the personnel recruitment and placement); 2) educational institutions (including the definition of requirements to the graduates and the educational process as a whole).

4 Introduction (2) ISO/IEC Information technology -- Security techniques -- Competence requirements for information security management systems professionals TC: ISO/IEC JTC 1/SC 27 IT Security techniques Edition 1. Stage: (New project (NP) has been approved on ) (access date )

5 Introduction (3) Issue of the day: assessment of the level of PCs (PCL) formed in the implementation of a specific curriculum in the field of IS. Two possible assessments: at the training completion (via one single examination on all aspects related to a specific IS PC) of the levels of mastery of all specific disciplines forming one IS

6 1. ISCs and their Levels (1) A competency: a combination of observable and measurable knowledge, abilities and skills that contribute to enhanced employee performance and ultimately result in organization s success. (+individual attributes & work experience) Knowledge (K): the cognizance of facts, truths and principles gained from formal training and/or experience. Ability (A): the power or aptitude to perform physical or mental activities that are often affiliated with a particular profession. Skill (S): a developed proficiency or dexterity in mental operations or physical processes that is often acquired through specialized training; using these skills results in successful performance. K+A+S levels are three integral parameters (attributes) of a specific PC. Problem to formulate all the parameters & to determine their necessary levels for that PC.

7 1. ISCs and their Levels (2) US

8 1. ISCs and their Levels (3) US Government IS Workforce Development Model : IS Proficiency Levels Not Not required to apply or demonstrate this competency. This competency is not applicable to the position. Applicable Limited Experience, basic knowledge. Level of experience has been gained in a classroom and/or experimental scenarios Entry-Level or as a trainee on-the-job. Help or mentoring will be needed in performing this skill. Demonstrates basic understanding (Basic of IS concepts, common knowledge and its application to computer systems architecture. Ability to understand and Knowledge) discuss terminology, concepts and principles, and issues related to this competency. Focus is on learning and development of on-the-job experience. Intermediate (Practical Application) Advanced (Applied Theory, Recognized Expert) Practical Application. Demonstrates solid IS knowledge and possesses the ability to apply the competency with minimal or no guidance in the full range of typical situations. Would require guidance to handle novel or more complex situations. Able to successfully complete tasks in this competency as requested. Competency has been applied to situations occasionally while needing minimal guidance to perform successfully. Help or mentoring from an expert may be required from time-to-time, but the skills can be performed independently. Focus is on applying and enhancing knowledge and skill sets. Ability to understand and discuss competency context and implications of changes to processes, policies, and procedures in this area. Applied Theory. Recognized Authority. Demonstrates expert understanding, knowledge and a broad advanced understanding, knowledge and ability in multiple information security subjects. Can apply the competency in new or complex situations. Demonstrates expert knowledge of law, regulation, policies and procedures, and IS standards including the ability to interpret and translate subject matter to various audiences. Actions can be performed associated with this skill without assistance. Recognized within organization as knowledgeable and a person to ask when difficult questions arise regarding this competency. Focus in on broad organizational/professional issues. Provide consistent, practical, and relevant ideas and perspectives on process or practice improvements which may easily be implemented. Capable of coaching and mentoring others in the application of this competency by translating complex nuances relating to this competency into easy to understand terms in a business and/or technology context. Participate in senior and executive level discussions regarding this competency. Assists in development of reference and resource materials related to this competency.

9 1. ISCs and their Levels (4) IS COMPETENCIES 1 M - Manage D - Design I - Implement E - Evaluate Data (Information) Security 2 Digital Forensics 3 Enterprise Architecture 4 Enterprise Continuity (Disaster Recovery) 5 Incident Management 6 IS Training Awareness and IT Systems Operations and 7 Maintenance Network and 8 Telecommunications Security Physical and Personnel 9 Security Policies, Standards and 10 Compliance (Information Assurance) 11 Privacy 12 Procurement 13 Security Risk Management 14 Strategic Security Management 15 System and Application Security Chief IS Officer Executive (Managerial) Privacy Officer IS Officer or Manager STATE GOVERNMENT IS ROLES Compliance Officer (Information Assurance) IS Engineer Functional (Technical) IS Professional IS Operations & Maintenance Professional IS System Administration Professional M M D M D D D D E E E E I E I I E I E M D E E I E E I E I I M D E E I E E I E I E M D D D D E E E I E I E I I E M M D D D D D D I E I E E I E I E I E I E M D D D D E I E E I E I E I E I E D M D M D E I E E I E I E M D M D M D E E I E E I E I E M D M D D D I E E E I E I E E I E M D M D M D E I E I E I E I I E I E I E M D M D D D E I E I E E I E I E E M D D D E E E E I E E E M D M D D D D D I E I E I E I E I E I E I I E M D I E I E I I E I I M D D D D E E E E I E I E I E I E

10 2. ISCs Formation (1) To train IS professionals in a university = to develop the students IS PCs. The IS PCs existence when training is complete = the graduate has the ability to perform his job successfully (to solve practical tasks assigned by his employer). Traditional PCs formation: mastery of a specific number of educational disciplines: training courses; various practices (practical works); completion of a final qualifying work (FQW). Any university can combine them to define specific features of an educational program leading to a Bachelor s or Master s degree, or a Specialist diploma.

11 2. ISCs Formation (2) Diagram of IS PCs formation Aim of a specific IS Master programme to develop a set of students' PCs CJ A predetermined number M of disciplines [D m ] form each IS PC A predetermined number N of disciplines [D mn ] provides K+A+S for the discipline D m A few disciplines create the required levels of IS K+A+S, necessary for mastering of disciplines, directly forming all necessary IS PCs A few disciplines directly form a specific PC C j

12 2. ISCs Formation (3) => the levels of attributes of a specific IS PC are formed directly and indirectly by IS K+A+S levels obtained by mastering of a set of disciplines. For that it is required: 1) to measure the levels of mastery of disciplines (DML) making up that PC; 2) to assess the contribution of each discipline to achieving IS PC.

13 3. Metrics for the Levels of Mastery of the Discipline (1) DML assessment (DA m ) scales: DA m = α km KD m + α am AD m + α sm SD m α km, α am, α sm the weights of the contribution of K+A+S levels into DML; α km + α am + α sm = 1. The levels KD m, AD m, SD m for mastering specific IS disciplines are defined during their progress testing (e.g. examination). The control questions should be formulated in such a way that these levels can be clearly assessed based on responses to them.

14 3. Metrics for the Levels of Mastery of the Discipline (2)! The peculiarities of a specific IS discipline should be considered. e.g. For a training course with lectures, seminars & labs. Lectures mainly form KD m, seminars AD m ; labs SD m. Only lectures (how to write IS policies) => α km = 1, α am = 0, α sm = 0. + seminars (an assignment to write a specific private IS policy) => α km α am, α km α sm, α am α sm. + labs (an assignment to detect network intrusions using IDS) => α km α sm, α km α am, α am α sm. e.g. Practices and FQW => α km α am, α km α sm, α am α sm. The values of α km, α am, α sm can be determined by experts (professor/ ). e.g. based on the work involved in a discipline (= laboriousness) in credits or time of classes duration in academic hours T 0 in view of duration of lectures T k, seminars T a and labs T s : α km = T k T 0 ; α am = T a T 0 ; α sm = T s T 0 ; T 0 = T k + T a + T s.

15 4. ISCs Level Assessment Model (1) Additional assumptions: the contributions of disciplines into the formation of different PCs are equal; to weight the contributions of D m to the formation of a specific IS PC. For D m : βk jm, βa jm, βs jm, defined by experts, 11-points range (0-10); scales for PCs levels:

16 4. ISCs Level Assessment Model (2) Initial data: a set of IS disciplines D m, directly forming the selected IS PC C j ; DML assessments DA m and the levels KD m, AD m and SD m ; the coefficients for each discipline α km, α am, α sm ; the weights βk jm, βa jm and βs jm for attributes assessment for C j. The main expressions for the model: Assessment of attribute value for C j, determining the IS level 1) knowledge: KC j = βk jm KD m M m=1 2) abilities level: AC j = βa jm AD m M m=1 3) skills level: SC j = βs jm SD m M m=1 M / βk jm m=1 M / βa jm m=1 M / βs jm m=1...

17 4. ISCs Level Assessment Model (3) Assessment of IS PC level CA j : 1. α km = α am = α sm : CA j = (KC j + AC j + SC j )/3. 2. The contributions are not equal => it is necessary to introduce λk j, λa j, λs j (λk j + λa j + λs j = 1) : CA j = λk j KC j + λa j AC j + λs j SC j. 3 (called frontier and based on the transition from points to levels). CA j : «0», if at least one attribute is at this level; «1», if at least one attribute is at this level and the rest are at the higher levels (2, 3); «2», if at least one attribute is at this level and the rest are at the higher level (3); «3», if the values of all competency attributes are at the same level (3). 4. DMLs DA m (forming C j ) are known and the contributions of these disciplines into PC formation µ jm are defined: CA j = M m=1 µ jm DA m If βk jm, βa jm, βs jm are defined, then µ jm = βk jm + βa jm + βs jm. M / µ jm m=1.

18 5. Model Usage Example (1) NRNU MEPhI: the "Business Continuity and Information Security Maintenance" (BC&ISM) program for Masters At the completion of training they must be able to fulfill the following: to analyze and to investigate models of systems ensuring business continuity (BC) and models of IS maintenance (C 1 ); to practically implement the standards relating to BC&ISM (C 2 ); to conduct IS risk assessment for the purpose of IS maintenance (C 3 ); to carry out the synthesis and analysis of design projects on BC&ISM for their organization where they work (O s) (C 4 ); to ensure the effective usage of O s IT resources to meet BC&ISM requirements (C 5 ); to participate in the design and operation of O s IS incident management system (C 6 ); to participate in the design and operation of O s BC&ISM system (C 7 ); to conduct technical auditing (monitoring) of O s information protection (C 8 ); to develop proposals for improving O s BC&ISM system (C 9 ); to establish and effectively implement a set of measures (rules, procedures, practical methods, guidelines, methods, tools) for BC&ISM (C 10 ).

19 5. Model Usage Example (2) 10 IS disciplines D m form this IS PC: 6 training courses: Protected IT (abbreviation PIT), IS Management (ISM), IS Risk Management Basics (ISRMB), IS Incident Management Basics (ISIMB), IT Security Assessment (ITSA), Business Continuity Management (BCM); Practical Works during Semesters 1, 2 and 3 (P1, P2, P3); Completion of a FQW (FQW). KC 7 = 68; AC 7 = 76; SC 7 = 84. D m T 0 T k T a T s α km α am α sm KD m AD m SD m DA m βk jm βa jm βs jm PIT ,58 0 0, ISM ,5 0, ISRMB ,5 0, ISIMB ,5 0, ITSA ,5 0, BCM ,5 0, P ,3 0,5 0, P ,2 0,4 0, P ,1 0,3 0, FQW ,1 0,2 0,

20 5. Model Usage Example (3) Options 1. The contributions of PC attributes are equal: CA 7 = 76. The PC level assessment will refer to the second (intermediate) level. 2. The contributions are unequal => we need λk 7, λa 7, λs 7. If the knowledge level is more important (e.g., λk 7 = 0,8; λa 7 = 0,1; λs 7 = 0,1): CA 7 = 71 the second (intermediate) level. If the abilities and skills levels are more important (e.g., λk 7 = 0,1; λa 7 = 0,45; λs 7 = 0,45): CA 7 = 79 the second (intermediate) level. 3. For frontier approach CA 7 = 68 the first (entry-level) level. 4. From the Table & µ jm = βk jm + βa jm + βs jm : CA 7 = 77 the second (intermediate) level.

21 Conclusion 1. The model allows to assess any IS PC level and the values of the PC's attributes according to the assessment characteristics of the disciplines forming it. 2. The model changes the way one assesses mastery of a specific discipline: assessment in general + separate assessments of the K+A+S levels developed by the students. 3. But the values of some of the model's parameters are determined by experts. 4. The given example confirms the usefulness and efficiency of the model. Future research (Further results - at WISE10!): approaches used can be applied to construct an inverse model: if the required IS PC level, its attributes and their assessment are known, then: to determine the list of disciplines forming the specific PC and their characteristics + the list and the level of K+A+S to be developed at the training completion; to formulate the initial data for a particular IS professional training curriculum and training courses. More complicated problem based on different synthesis methods and requiring solving a few optimization problems with predefined criteria!

22 Annex

23 Thank you for your attention!