HEA 2017 Conference: Exploring Innovations in Assessment with Statistical and Data Analytical Software Packages Kathy Maitland Peter Samuels

Transcription

1 HEA 2017 Conference: Exploring Innovations in Assessment with Statistical and Data Analytical Software Packages Kathy Maitland Peter Samuels 6 th July 2017

2 Background Increasing importance of data analysis in the workplace (Cheng, 2014; Manyinka et al., 2011) Increasing use of powerful tools statistical analysis, data analytics and data mining Students are not being properly prepared for the workplace: Using analytical tools is still seen as cheating Lack of conceptual understanding when using tools Students often thrown in the deep end with their dissertations and projects No benchmarking of skill development

3 Aim To give participants the opportunity to create teaching and assessment plans including benchmarking and online assessment for different subject areas which include developing analytical skills with popular software packages.

4 Objectives 1. To gain practical experience in combining and embedding software skills into academic modules and associated assessments 2. To benchmark skills assessments (both subjectbased and software-based) 3. To explore the feasibility of online assessment 4. To share best practice in incorporating key software competency skills into taught academic programmes

5 Statistical analysis, data analytics and data mining Subject Initial research question Systematic pattern recognition Statistical analysis Yes - specific Data analytics Yes - general Data mining No No No Yes Descriptives Yes Yes Yes Hypothesis testing Yes Yes Yes Prediction Possibly Yes Yes Example software Excel and SPSS SAS and JMP SAS Enterprise Miner

6 Online testing for data analysis? Online testing of data analysis capability is in its infancy (Czaplewski, 2014) One promising system is DEWIS from UWE ( This has the capability of generating randomised data sets using R, numerical input and feedback. Example: question/stage_two.pdf Another system is Numbas from Newcastle University ( Example questions are available from Both systems are free and allow formative and summative assessment

7

8

9 Sample Dataset: CensusAtSchool Questionnaire Completed by 673 school children No specific research question but one could be imagined Hence suitable for both statistical analysis and data mining

10 Example generic activities Descriptive statistics using Excel: Tables, bar charts, pie charts, percentage frequency bar charts, histograms, summary statistics and scatter plots Inferential statistics using Excel: Correlation and linear regression Descriptive statistics using SPSS: Cross-tabulation, box and whisker plots, error bar charts, histograms with fitted normal curves Inferential statistics using SPSS: Assumption checking, independent t-test, Mann- Whitney U test

11 Data analytics SAS JMP example The Task Using descriptive statistics and graphical displays, explore claim payment amounts, and identify factors that appear to influence the amount of the payment. The Data (MedicalMalpractice.jmp) The data set contains information about the last 118 claim payments made, covering a six month period. With eight variables: Amount: Amount of the claim payment in dollars Severity: The severity rating of damage to the patient, from 1 (emotional trauma) to 9 (death) Age: Age of the claimant in years Private Attorney: Whether the claimant was represented by a private attorney Marital Status: Marital status of the claimant Specialty: Speciality of the physician involved in the lawsuit Insurance: Type of medical insurance carried by the patient Gender: Patient gender

12 Data analytics SAS JMP example Analysis: Begin by looking at the key variable of interest, the amount of claim payment: histogram and summary statistics for Amount. (Analyze > Distribution; Select Amount as Y, Columns, and click OK. For a horizontal layout select Stack under the top red)

13 Benchmarking Subject benchmarking a commonly practice in course development possibly driven by professional bodies Data analysis software benchmarking less common often at a lower level than subject benchmarking Benchmarking assessments using data analysis software requires both

14 Benchmarking example Computer Science Sources: BCS core requirements for accreditation of honours programmes Additional requirements for CITP BCS Computing-related practical abilities Royal Statistical Society Core Knowledge and Skills SAS Certified Base Programmer for SAS 9 Quality Assurance Agency (QAA) levels (2014)

15 Examples of blended assessments: Practical Exam Questions on academic theory Multiple choice of similar style to vendor certification that represents academic theory in practice

16 Examples contd.: Case Study Academic theory in practice through the use of a case study Students provide an industrial strength report using the case study as the vehicle for assessment Competency of tool demonstrated in the production of statistics and charts Reflection of real world tasks at the appropriate QAA level

17 Example: Exploratory Analysis Students are required to explore a particular area of interest through data analytics Locate or obtain appropriate data set through experiment or open source data Students are required to demonstrate a variety of competencies through the use of data analytics tools Results are presented through a poster or report

18 30 minutes: Activity Break into small groups of your choosing Choose a subject area Choose appropriate software: e.g. data analytics using SAS / JMP, or statistical modelling using Excel / SPSS, or data mining using Enterprise Miner / Watson Analytics Based on the generic examples provided, develop an approach to teaching and assessment using software for your subject area for Levels 4 to 6 Explore the use of benchmarking standards and online assessment Report back

19 Conclusions Future developments of effective teaching and assessment techniques with data analysis software packages in different STEM subjects

20 Thank you for your participation. If you would like to contact us, especially if you are interested in working on a joint paper on this subject, please us at: kathleen.maitland@bcu.ac.uk peter.samuels@bcu.ac.uk

21 References Chen, J. (2014) Big data for development in China. New York: UNDP. P%20Working%20Paper_Big%20Data%20for%20Development%2 0in%20China_Nov% pdf. Czaplewski, J. R. (2014) An evaluation of blended instruction in terms of knowledge acquisition and attitude in an introductory mathematics course. Doctoral dissertation. University of Minnesota. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C. and Byers, A. (2011) Big data: The next frontier for innovation, competition, and productivity. McKinsey & Company. _Innovation/Big_data_The_next_frontier_for_innovation. The Quality Assurance Agency for Higher Education (QAA) (2014) The UK Quality Code for Higher Education.