Validating Academic Training Verses Organizational Training An Analysis in the Enterprise. Farmingdale State College

Transcription

1 Fourth Annual National Conference on Enterprise Computing 2012 Marist College Validating Academic Training Verses Organizational Training An Analysis in the Enterprise Resource Planning (ERP) Field Dr. Jill O Sullivan Farmingdale State College

2 Table of Contents Introduction What is Enterprise Resource Planning Questionnaires, Interviews, Case Studies and Surveys Researched Literature ERP Employer Survey Questions and Results Comparison of Training Methodologies Key Results Conclusions

3 Presentation Intention What this researcher did. How this researcher did this. What results are, what results say and what results mean. How the researcher can support what this means. Why this information is important.

4 This research disclosed ERP in the workforce, ERP academic history, Business drivers impacting ERP strategy, ERP importance to organizations, benefits, Critical success factors in ERP training and education.

5 Introduction

6 Problem and Solution Strategy New hires not meeting the needs of companies regarding their ERP education and skills. The intent of this research was to determine a correlation between colleges prepared students and employees new to ERP that have gone through their companies ERP training.

7 Significance Training to use tools required in business is Critical. Lack of proper skills--and especially the latest skills--can prevent a company from being a leader, as well as affecting the bottom line profits. Need in academia to effectively teach ERP Using ERP concepts usingtraining a hands on real world ERP tool

8 Purpose of Study Explored what industry uses in their ERP training, as compared to college prepared students. Exposed a correlation between ERP prepared students and employees new to ERP that have gone through ERP training.

9 ERP Enterprise Resource Planning

10 What Is Enterprise Resource Planning Packaged software applications that support most of an organization information needs within and across functional areas to coordinate information in every area of business. Helps to manage companywide business processes, using a common database and shared management reporting tools.[56].

11 Typical ERP Components

12 APICS Definition 12th edition Framework for organizing, defining, and standardizing the business processes necessary to effectively plan and control an organization so the organization can use its internal knowledge to seek external advantage. [4].

13 Infor Visual The use of the Infor Visual ERP is the focus of this study since it is what the students featured in the interviews (IDI) and Case Studies have learned and mastered. Those companies solicited for responses to the ERP Employer Survey are all Infor Visual Users except for some of the APICS ERP Forum Survey.

14 Questionnaires, Interviews, and Surveys

15 What this Researcher Did How This Was Done What The Results Were

16 Student Self Reported Knowledge Questionnaire

17 Student Self Reported Knowledge Questionnaire Created and used a questionnaire to assess FSC students proficiency/skills at the beginning and at the end of the ERP class experience for four semesters. Nine Questions, likert Scale 1,2,3. A total of 72 students responded from four classes.[48]

18 How Student Self Reported Knowledge Questionnaire Was Completed ERP Students were given the questionnaire with an envelope in class at the beginning of the semester and then at the end of the semester. Students completed this questionnaire in the class then put this in a sealed envelope.

19 Student Self Reported Knowledge Questionnaire Results Students learned how to use this real world ERP Infor Visual Tool because of this class experience. Students better understand business functions per specific applications and the importance of their cross functionality.

20 Interviews

21 Students (c) Jill O Sullivan 2011

22 Interviews -Students Conducted Interviews of ERP Students now in full time positions in industry about their ERP responsibilities and performance. Interviews were conducted: In the researchers office at FSC, in the Students offices at their companies, in their Managers offices and in the President s Conference room of the company the student worked for. Validating Academic Training Verses and Organization Interviews lasted between one hour twenty five minutes to three hours (c) Jill O Sullivan 2011 including a class

23 Interviews Students Results Students revealed that they are more prepared, knowledgeable, and skilled in using their companies ERP because of the experience in this ERP class. They were able to start faster than others with less training and more accuracy. Therefore, what this tells us is that this type of ERP preparation is beneficial to ERP user companies as Training we saw Using in ERPthe many areas these (c) Jill O Sullivan 2011 job functional students described in their

24 Student s Managers

25 Interviews Managers Conducted Interviews of ERP Student s Managers acquiring information about these students now in full time positions to determine how they performed ERP tasks. Interviews conducted; In Managers offices, on teleconference, via responses. Interviews were based on an Interview script. Interviews Trainingapproximately Using ERP lasted forty five

26 Interviews Managers Results Students Managers revealed that these students are more prepared, knowledgeable, and skilled in using their companies ERP because of the experience in this ERP class. They were able to start faster than others with less training and more accuracy. ERP preparation is beneficial to ERP user companies per students managers.

27 Professors Teaching SAP ERP Validating Academic Training Verses Organization

28 Interviewed SAP Professors Conducted five telephone interviews of University Professors who use the SAP ERP in their Classrooms since there are No Infor Visual Course available except for the featured classes at FSC.

29 How Interviewed SAP Professors Contacted five SAP Professors via telephone. Sent Script Questionnaire via . Followed up with telephone call to discuss. Lasted around thirty minutes to one hour. Recorded

30 Results Interviewed SAP Professors SAP Professors stated they did not have an assessment or way to measure if what was taught to students was successfully meeting the needs of industry. They did not have information on follow up with their students now in industry. Many recommended this study as an important contribution.

31 Case Studies

32 Case Studies Created a case study script to guide students into describing their transition experience from the ERP classroom into the business world. Sent this script via to these students. Went over some in person face to face and some returned the script completed. Based on the featured Students input; Highlighted Students ERP skills, (c) Jill O Sullivan 2012 proficiency and

33 Case Studies The case studies include In Depth Interviewing (IDI) of students from this ERP class that have transitioned over into industry. Students that have taken the ERP class and are now in industry using ERP as a part of their daily functions exposes their significant distinct advantage because of this ERP hands on learning experience.

34 Case Studies Results Students conveyed how the class ERP experience made a difference in their ability to perform their responsibilities using ERP. The hands on ERP experience preparation in the ERP Class at FSC prepared students very well for industry ERP positions.

35 ERP Employer Survey

36 ERP Employer Survey Created an ERP employer survey in many iterations until we concluded with a 31 question survey, the ERP Employer Survey. The survey was put on a tool called Lime Survey due to its open source, user friendly and fantastic report bearing capabilities. Survey was open for two weeks with one reminder after the eight day. Solicited responses to an ERP Employer Survey

37 ERP Employer Survey The survey had thirty one questions and was segregated into six different groups including; ERP experience which had four questions. ERP Training and Education which had thirteen different questions. ERP Skills which had two questions. New to ERP Capabilities which had four questions. Performance Validating Measurements which had two Academic Training Verses Organization questions.

38 ERP Employer Survey Results There were seven hundred seventy one Infor Visual Customers in the United States and Canada solicited. Three hundred thirty one opened the first eblast; and of those, 41 clicked on the survey link, out of those 39 proceeded to complete the survey. Actual Results in section ahead, slide 51.

39 APICS NYC/LI ERP Survey

40 APICS ERP Survey in ERP Forum Conducted an APICS ERP Survey in ERP Forum. Sent using Survey Monkey Soliciting APICS NYC/LI Membership Survey had 12 questions Combination fill-in blank and multiple choice 19 responders

41 APICS ERP Survey Results Most important information revealed from this survey that correlates with this study is in response to this question: Cross-functional collaboration is a critical factor in the success of an ERP system. Select the functions that are actively involved in the development or continuous improvement of the ERP system in your organization: Because we saw cross functional collaboration Training Using ERP of ; sales, identified in the answers

42 Researched Literature

43 Researched Literature ERP In Academia ERP Curriculum Development Collaboration Efforts-Industry and Academia Training Profiles

44 Academic ERP Program The research on the integration of ERP into university curricula loosely falls into four general themes: (SAP ONLY Represented here) (a) ERP curriculum development (b) adoption levels of ERP software in curriculum (c) integrating ERP concepts into a single course or project (d) lessons learned from implementing ERP into university curricula. [21] Training Using ERP

45 Collaboration Industry and Academia Companies are eager to consult with faculty on developing realistic hands on exercises because they are aware that students in these classes will then the be better prepared as potential employees. [6]

46 Training

47 Generic Training Profile Training, instruction, or educational courses can be provided through a variety of media: Personal instruction Classroom instruction Seminars On-the-job (OJT) training Written instruction Text books

48 Industry ERP Training A Full Coverage Support System for Business The primary focus of ERP is to improve and increase the flow of accurate information within an organization (Norris, Hutley, Hartley, Dunleavy, & Balls, 2000). ERP software improves cooperation, collaboration and interaction between all business functional areas or units in an organization including accounts payable, (c) Jill O Sullivan receivable, payroll, HR, 2012 purchasing, inventory

49 Academic ERP Program The research on the integration of ERP into university curricula loosely falls into four general themes: (a) ERP curriculum development (b) adoption levels of ERP software in curriculum (c) integrating ERP concepts into a single course or project (d) lessons learned from implementing ERP into university curricula. [21]{SAP}

50 FSC ERP Program The ERP courses at FSC utilize a variety of media: Personal instruction Classroom instruction Seminars (Visiting Specialists) On-the-job (OJT) In Class apprenticeship type training Written instruction Text books Manuals

51 ERP Employer Survey Questions

52 ERP Employer Survey The survey had thirty one questions and was segregated into six different groups including; ERP experience which had four questions. ERP Training and Education which had thirteen different questions. ERP Skills which had two questions. New to ERP Capabilities which had four questions Performance Validating Measurements which had two Academic Training Verses Organization questions.

53 ERP Survey Elements Nine Tasks for Comparison Six Training Methodologies Four Employee Categories

54 Nine Tasks 1. Search to find data/understand impact on functions 2. Drill down in different module 3. Perform basic activities per modules 4. Obtain reports and use for decision making 5. Track activities per operation flows 6. Display existing data and create new data 7. Create reports to perform analysis 8. View and manipulate data 9. Inquire from any(c)module to2012 any other for data Jill O Sullivan

55 Four Training Methodologies Training Methodologies: 1. On the Job Apprentice; includes working closely with functional manager in the position hired for. 2. On the Job Training Program includes; Inside trainers (responsible for ERP training Inside company) 3. Outside trainers includes; Outside Consultant. 4. Outside trainers includes; Third party Validating Academic Training VersesTrainer. Organization

56 Employee Categories NO ERP EXPERIENCE; Includes; a) 1st time job out of college, b) new employee with work experience c) current employee ERP EXPERIENCE BUT NOT CURRENT ERP TOOL; Includes; a) academic trained b) work trained

57 ERP Class Equivalent Activities To Nine Tasks A description of what these ERP students due in each area that equates to these tasks follows. All required exercises, assignments, homework's, quizzes, tests and presentations are put in to the SUNY Learning Network SLN Angel system that controls all class materials per chapter modules. All grades are derivedvalidating per aacademic rubrictraining andverses Organization automatically go into the gradebook once

58 1. Search to find data/understand impact on functions Students required to search per exercises, assignments, homework, quizzes tests and projects. Required to prove that they have found certain items per each module when searching; customers, addresses, PO s, part numbers, revisions, ECN s, work orders, BOM s, quotes, suppliers, vendors, employees, reports, invoices, pricing, receivables, customer orders, RFQ s, Training Using ERP employee pay rates, category, delivery dates,

59 2. Drill down in different module. Students are required in this hands on class to go into their ERP database and drill down into every module to find, gather and input data. This is required for exercise, assignments, quizzes, tests and projects all which are graded. Each module has a drop down selection of

60

61 3. Perform basic activities per modules Students are required to perform tasks per exercise, assignments initially giving them an overview of the system. Then students perform more detailed tasks per exercises, assignments, homework, projects, quizzes and tests.

62 5. Track activities per operation flows The student activities build on each other as if they were a company building and selling a product. As a manufacturer of the parts (solar panels) they are responsible for all aspects of the companies information flow. They have to input data that affects other areas and track that information, specifically when they run MRP for the solar Training Using ERP panels.

63 6. Display existing data and create new data Students are required to prove that they can find data and create new data whether in the form of quotes, purchase orders, invoices, bill of materials, work order, shop floor resources, receivable and any other real life transaction a company does. They prove they can do this buy submitting screen shots per assignments.

64 7. Create reports to perform analysis Students must create specific reports and explain the analysis of these reports. As with all other assignments these are graded.

65 8. View and manipulate data Students prove they can view information in their system and be able to manipulate that data for example they will have to find or enter a quote, they might have to change the ship to, terms, unit of measure or part revisions or anything else that real companies have to deal with. They then have to either find existing information or create it.

66 9. Inquire from any module to any other for data Students often do this in their assignments, they must be able to understand where to get info and where info goes when entering. They must be able to pull from different screens in completing their assignments. Examples on exercise and activities slide.

67 Students Exercises and Activities Includes all Nine Tasks En t e r Qu o t a t io n Or d e r o n t h e S h o p F lo o r M a t e r ia l t o a Wo r k Or d e r Re vie w in g t h e Co s t o f t h e Wo r k Or d e r En t e r Co m p le t io n o f Cu s t o m e r t h e Wo r k Or d e r Or d e r fr o m P r o c e s s in g a Qu o t a t io n S h ip m e n t Eva l u a t in g M e a s u ri n g B a c k lo g P ro Training Using ERPfit a n d Lo s s 2012 Eva l u a t in g o u r(c) Jill O Sullivan

68 All Nine Tasks are found in Student Requirements and were derived from student activities, literature and company information

69 Survey Questions Results

70 Questions 1-5 Results Majority of company responses 1. IT departments manage ERP training. 2. Department managers decided who gets trained. 3. Companies use an On the job apprentice training. 4. Companies create their ERP training database in house. 5. Academic Training Verses Organization ERP Experience ButValidating Not with Current ERP was (c) Jill O Sullivan 2012 the employee category chosen when hiring for

71 Questions 6-7 Results 6. Training methodology used was the on the job apprentice which includes working closely with functional manager in the position hired. Validates the method used in the classroom as apprentice type training. We also saw evidence of in the interviews and featured case studies. 7. ERP revealed that the most important per training methodology came out to be Improve production / process for on the job apprentice.

72 Questions 8-10 Results 8. Revealed that those with No ERP use a training database while those with current system or not use a live ERP system. *conclude that the system in the classroom is validated. 9. Those with no ERP and or without the current tool receive apprentice on the job training while those with ERP experience received training from inside trainer. Trainingan Using ERP

73 Questions Results Time and money are the driving forces in ERP Training. The time of training ranges from one week to eight weeks while the class is fifteen weeks twice a week. Therefore we see more actual training with the ERP class resulting in a student who is more prepared which correlates with the interview and case studies.

74 Questions Results The modules new hires exposed to in training are inventory, sales, purchasing, engineering and manufacturing. This correlates directly with the course material subject ERP offering per the case studies herein and is highlighted by the students interviews. [27] Because we saw this we induce a direct validation.

75 Questions 18 Results Critical skills required are; How to interpret demands & supplies Have knowledge and experience with Manufacturing Work orders. Therefore and because of this information the case study and interviews validate that this is taught in the class and required by industry.

76 Question 19 Results Companies would include in training: ERP Knowledge of a general overview of standard flow of organization General system navigation. [27] Because we saw this in these results and recognize industries need for these skills we therefore confirm that the class utilized these and validates the correlation.

77 Questions These four questions resulted in 576 answer which are reviewed here in graph form for a quick view of the conclusions. Slides The first to be viewed are NO ERP experience starting with On the Job Apprentice; Inside trainers, Outside Consultant and Outside Third party Trainer. The second to be viewed are ERP experience Academic Training Verses Organization starting with On thevalidating Job Apprentice; Inside (c) Jill O Sullivanand 2012 Outside Third trainers, Outside Consultant

78 New to ERP Capabilities Each of these questions asked the company to rank their responses with Ranking Choices; 1 Least Competent, 2 Competent and 3 Most Competent. The Results follow in the form of graphs. 20. When you put an employee through the company's ERP education and training program (in a Job Training Apprentice), how do they rate in being competent after completion per employee category? Validating Academic Training Verses Organization 21. When you put an employee through the company's ERP education and Training Using how ERP do they rate in being training program (in a Job Training Inside),

79 When you put an employee through the company's ERP education and training program how do they rate in being competent after completion? Comparisons Question 20 To ERP Class Per Task NO ERP Knowledge; Training Method -On Job Apprentice

80 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category NO ERP Knowledge; Training Method -On Job Apprentice

81 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category NO ERP Knowledge; Training Method -On Job Apprentice

82 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category NO ERP Knowledge; Training Method -On Job Apprentice

83 Graphs Task Comparisons Question 21 NO ERP Class Per Task ; Category -NO-ERP -Knowledge; Training Method - Inside Training (c) Jill O Sullivan 2011

84 Graphs Task Comparisons Question 21 NO ERP Class Per Task ; Category -NO-ERP -Knowledge; Training Method - Inside Training

85 Graphs Task Comparisons Question 21 NO ERP Class Per Task ; Category -NO-ERP -Knowledge; Training Method - Inside Training

86 Graphs Task Comparisons Question 21 NO ERP Class Per Task ; Category -NO-ERP -Knowledge; Training Method - Inside Training

87 Graph Task Comparisons Question 22 To ERP Class Per Task; Category -NO-ERP Knowledge ; Training Method -Outside Consultant Training

88 Graph Task Comparisons Question 22 To ERP Class Per Task; Category -NO-ERP Knowledge ; Training Method -Outside Consultant Training

89 Graph Task Comparisons Question 22 To ERP Class Per Task; Category -NOERP Knowledge ; Training Method -Outside Consultant Training

90 Graph Task Comparisons Question 22 To ERP Class Per Task; Category -NOERP Knowledge ; Training Method -Outside Consultant Training

91 Graphs Task Comparisons Question 23 To ERP Class Per Task; Category -NO ERP Knowledge; Training Method-Outside Third Party Training

92 Graphs Task Comparisons Question 23 To ERP Class Per Task; Category -NO ERP Knowledge; Training Method-Outside Third Party Training

93 Graphs Task Comparisons Question 23 To ERP Class Per Task; Category -NO ERP Knowledge; Training Method-Outside Third Party Training

94 Graphs Task Comparisons Question 23 To ERP Class Per Task; Category -NO ERP Knowledge; Training Method-Outside Third Party Training

95 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category ERP Knowledge; Training Method -On Job Apprentice

96 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category ERP Knowledge; Training Method -On Job Apprentice

97 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category ERP Knowledge; Training Method -On Job Apprentice

98 Graphs Task Comparisons Question 20 To ERP Class Per Task; Category ERP Knowledge; Training Method -On Job Apprentice

99 Graphs Task Comparisons Question 21 To ERP Class Per Task; Category ERP -Knowledge; Training Method- Inside Trainer

100 Graphs Task Comparisons Question 21 To ERP Class Per Task; Category ERP -Knowledge; Training Method- Inside Trainer

101 Graphs Task Comparisons Question 21 To ERP Class Per Task; Category ERP -Knowledge; Training Method- Inside Trainer

102 Graphs Task Comparisons Question 21 To ERP Class Per Task; Category ERP -Knowledge; Training Method- Inside Trainer

103 Graph Task Comparisons Question 22 To ERP Class Per Task Category -ERP ; Training Method-Outside Consultant Training

104 Graph Task Comparisons Question 22 To ERP Class Per Task Category -ERP ; Training Method-Outside Consultant Training

105 Graph Task Comparisons Question 22 To ERP Class Per Task Category -ERP ; Training Method-Outside Consultant Training

106 Graph Task Comparisons Question 22 To ERP Class Per Task Category -ERP ; Training Method-Outside Consultant Training

107 Graph Task Comparisons Question 23 To ERP Class Per Task Category -ERP Knowledge; Training Method-Outside Third Party Training

108 Graph Task Comparisons Question 23 To ERP Class Per Task Category -ERP Knowledge; Training Method-Outside Third Party Training

109 Graph Task Comparisons Question 23 To ERP Class Per Task Category -ERP Knowledge; Training Method-Outside Third Party Training

110 Question 24 Results Performance Measurements Majority of companies surveyed measure the results of their ERP system training and skills with On the job performance reviews.

111 Question 25 Results Critical to this study, is the answers herein because we see that the following has resulted from ERP training which related directly to literature about ERP implementation, interviews and case study findings; Ease of use of the system Lack of supervision required, Increased productivity

112 Demographic Information Questions 26,27,28,29 and 30 Reveal the demographic information about these companies. Those responding included: Presidents, CEO S, Vice Presidents, Directors, IT Managers, Engineering Managers, Operation Managers, Finance Managers, and ERP Managers. Only one response for Public Industry Sector reported and that was as an Assembler, no other categories we inputted. The results Validating Academic Training Verses Organization indicate for the Private Industry (c) Jill O Sullivan 2012 selections

113 Comparing Results of Training Methodologies For NO ERP Experience Apprentice to Inside Company Training Because the data revealed that the apprentice on the job type training was used more therefore this tells us that there is more confidence in this type of training. Apprentice to Outside Consultant Training Because the data revealed that the apprentice on the job type training was used more therefore this tells us that there is more

114 Comparing Results of Training Methodologies For ERP Experience Apprentice to Inside Company Training Because the data revealed that the apprentice on the job type training was used more therefore this tells us that there is more confidence in this type of training. Apprentice to Outside Consultant Training* Because the data revealed that the apprentice on the job type training was used more for those new to the company those with

115 Comparing Results of Training Methodologies For No ERP Inside Company to Outside Consultant Experience Training Because the data revealed that the outside consultant type training was used more therefore this tells us that there is more confidence in this type of training. Inside Company to Third Party Training Because the data revealed that the third party type training was used more for those new to

116 Comparing Results of Training Methodologies For ERP Experience Inside Company to Outside Consultant Training Because the data revealed that the outside consultant type training was used more therefore this tells us that there is more confidence in this type of training. Inside Company to Third Party Training Because the data revealed that the third party type training was used more for those new to

117 Comparing Results of Training Methodologies For NO ERP Experience Outside Consultant to Outside Third Party Training Because the data revealed that the third party type training was used more therefore this tells us that there is more confidence in this type of training.

118 Comparing Results of Training Methodologies For ERP Experience Outside Consultant to Outside Third Party Training Because the data revealed that the third party type training was used more therefore this tells us that there is more confidence in this type of training.

119 Key Results (c) Jill O Sullivan 2011

120 Key Results Because of the data received in Interviews, Questionnaires, Case Studies and the ERP Employer Survey within this study the key results are as follows; We saw proficiency in the use of ERP on the job by these students and conveyed by their management. We conclude that these ERP Students are

121 Key Results People new to ERP that go through an academic program like the one featured here specifically can acquire skills and ERP knowledge that new employees new to ERP might not acquire even after their companies ERP training.

122 Key Results Apprentice on the job training is validated as a preferred method of training. Those with NO ERP experience but with work experience are equivalent to those who are in a different functional area with in the company and do about the same in competence after training.

123 Key Results The academic preparation of students for industry positions is validated as worthy since based on this information and because of these results, the researcher can conclude that the academic training meets or exceeds that which is done in industry according to this research.

124 Key Results The results revealed when comparing the types of training for both No ERP experience and ERP experience revealed new knowledge at least to this researcher. Results indicate that; Apprentice On the Job type training was the preferred training type when compared to all others. When comparing Insider training to outside consulting and third consulting, both the (c) party Jill O Sullivan 2012

125 Conclusion As technology becomes more advanced in the industry, the need for a better trained and prepared workforce becomes more critical to building a sustainable talent pipeline. Lockhead Martin, VP Manufacturing Operation, Mike Packard

126 Conclusion Today s business people (and tomorrow's) should understand what an ERP system can do for a company. The intent, as in any educational environment, is not only to help students attain an understanding of the material, but to also master that material in the environment in which they will be required to use it.

127 Conclusion The areas of concentration for the training, the training methodology is very similar if not the same as college prepared per the entity in this study. The training system mostly used for new hires is a training system, which is what is used in the studied cases.

128 Comments Preparing students for entry and success in the workforce is paramount. There is a demand currently for students and employees with this experience. PM100 Award

129 Questions? Dr. Jill O Sullivan Farmingdale State College osullija@farmingdale.edu

130 Dr. Jill O Sullivan Professor Farmingdale State College Computer System Department President APICS NYC/LI Chapter Manufacturing Executive Leadership Council Workforce Task Force

131 References References A. Shtub, A Framework for teaching and training in the Enterprise Resource Planning (ERP) era, 2000, INT.J.Prod. RES., Vol.39, No.3, A.A. Chawdhry, M. J. Donohoe, Delivering ERP Learning To Students In A University Business Program. (c) Jill O Sullivan 2011 Adnan A. Chawdhry, California University of

132 References J. C. Stewart, C. Sher DeCusatis, K. Kidder, J. R. Massi, and K. M. Anne, Evaluating Agile Principles in Active and Cooperative Learning, Proceedings of Student-Faculty Research Day, CSIS, Pace University, J. Esteves, J. Pastor, Enterprise Resource Planning Systems Research: An Annotated Bibliography, Communications of The Association forvalidating Learning Information System, Academic Training Verses Organization Vol 7, Article 8, August (c) Jill O Sullivan 2011