CHIA CENTER FOR HEALTH INFORMATION AND ANALYTICS RESEARCH, SOFTWARE, AND SERVICES

Transcription

1 CHIA CENTER FOR HEALTH INFORMATION AND ANALYTICS RESEARCH, SOFTWARE, AND SERVICES Created by Mitch Sturge

2 TPMI TRANSLATIONAL AND PERSONALIZED MEDICINE INITIATIVE NL SUPPORT / \ CHIA

3

4 CHIA ENVIRONMENT 96 - NeXtScale nx360 M4 Compute Servers - 64GB RAM 4 - x3750 M4 Large Memory Compute Servers - 256GB RAM 1692 Intel Xeon E5-2650v2 (Ivy Bridge) 8-Core 2.6GHz 20MB Cache Six (6) POWER8 822 Linux/AIX Compute Servers 160, 3.42 GHz POWER8 cores OpenStack Juno Private Cloud Environment IBM Cognos and SPSS environments 6 PB of GPFS (GSS) Storage IBM TS3500 Enterprise Tape Library with ~13 PB of LTO6 tape Additional power and cooling capacity to add an additional 10,000 cores

5 SECURITY

6

7 DATA SECURITY All data within the CHIA data center is controlled by a hierarchy of security measures: 1. Access to systems filtered through dedicated firewalls and advanced malware protection on an asset by asset basis. 2. Seperated file systems for research, clinical, database, cloud and user home directories. 3. All files are assigned groups associated with the specific research project they belong to. Typically associated with the PI. 4. Users who have ethics/pi approval to access data are then assigned a role specific to that data, allowing them access.

8 SECURITY POLICIES We have worked with Electronic Warefare Associates (EWA) to create the following policies to guide the operation of CHIA services: Systems Security Management Policy User Access Management Policy Privacy Policy Incident Response Plan A threat risk assessment was completed on the CHIA infrastructure A privacy impact assessment was created to handle concerns and requirements around holding clinical data

9 HOW TO SUBMIT JOBS Currently there are three types of workloads being used within the CHIA environment: 1. Command line job submission with IBM LSF. 2. Web based job submission with IBM PAC. 3. Complex job workflow creation with IBM PPM. 4. Dedicated OpenStack environments

10 COMMAND LINE SUBMISSION bsub [options] command [cmdargs] bjobs [ a][ J jobname][ u usergroup u all][...] jobid bhist [ a][ J jobname][ u usergroup u all][...] jobid bbot/btop [jobid "jobid[index_list] ] [position] bkill [ J jobname] [ m] [ u ] [ q] [ s signalvalue] bmod [bsub_options] jobid bpeek [ f] jobid bstop/bresume jobid bswitch destination_queue jobid Using this method gives the user full control of how their job is submitted and handled by the cluster. Documentation is readily available.

11 SAMPLE JOB SUBMISSION bsub J `pwd` q normal n 4 cwd `pwd`./my_job_launcher.sh There are many additional options that can be specified for a job. Job <4>, Job Name </gpfs/home/msturge/regression/chip1>, User <mst urge> Project <default>, Status <RUN>, Queue <normal>, Command <./my_job_launcher.sh> Thu Apr 17 17:48:43: Submitted from host <chia-sched1>, CWD </gpfs/home/msturge/regression/chip1>; Thu Apr 17 17:51:55: Started on <chia-sched1>, Execution Home </home/msturge>, Execution CWD </gpfs/home/msturge/regression/chip1> ; Thu Apr 17 17:52:12: Resource usage collected. MEM: 2 Mbytes; SWAP: 5 Mbyt

12 SUBMITTING A JOB: IBM PAC Common jobs can be submitted through the browser.

13 JOB STATUS

14 AUTOMATION: IBM PLATFORM PROCESS MANAGER

15 PPM EXAMPLE Takes in a datafile for the program SHAPEIT. After it succeeds, the output is converted using another workflow. If all conversion jobs succeed then they are run through finestructure. All workflows are reusable components.

16 Used to provide dedicated IBM Cognos environments for each reseach project held in our environment. By providing separate environments we ensure that data in a research project can only be accessed by authorized users.

17 RESEARCH EFFICIENCY A research project worked on by Dr. Zhai required the analysis of a large dataset, taking approxametly 2 months to complete on his desktop system. After publication, it was identified that he would need to run this analysis in 5 seperate ways to meet reviewer recommendations By migrating this work into the CHIA HPC cluster, all 5 jobs were completed in 8 days

18 RESEARCH EFFICIENCY The lab utilization project has the requirement to analyze and report on administrative data relating to lab tests requested within Eastern Health The question is simply to identify where efficiencies can be gained in this system. But the data set to analyze just one section of tests is over 4 million records. Traditional tools are not capable of handling Big Data of this size. By employing a combination of Apache Spark, Hadoop, HPC, IBM Cognos and SPSS we can now interrogate this data set in ~3 seconds

19 COLLABORATIONS

20 CURENT CHIA COLLABORATIONS Memorial Faculty of Business Administration Evaluation of predictive abilities of different statistical methods for estimating structural equation models (a type of statistical model popular in the social sciences). C-CORE Retrieval of sea-ice parameters from fusion of satellite data and a high resolution sea-ice model. University of Alberta Studying/building atom-scale switches. Running a series of 512 core jobs that each last about 3 days

21 CURENT CHIA COLLABORATIONS Potential analysis of thousands of whole genome datasets, requiring extensive HPC resources Memorial & Sick Kids Hospital - In Planning Deep Sequencing of Autism Spectrum Disorder Postmortem Brain Exome sequencing of Autism Spectrum Disorder (ASD) twins

22 CURENT CHIA COLLABORATIONS Partnership with IBM Canada - Predictive Analytics Jointly develop validated solutions that help translate knowledge and data into action Support both clinical and administrative scenarios Not intended to host production systems

23 COMPREHENSIVE APPROACH ENABLES CARE FROM BENCH TO BEDSIDE

24 CHIA - INNOVATION SPACE The CHIA offices are capable of holding a number of additional students or staff that are interested in accessing CHIA services Working from within the CHIA office can provide users direct access to IT and administrative staff to help streamline project deployment

25 QUESTIONS?

26 ACCESS

27 HOW TO ACCESS SERVICES CHIA Privacy Presentation Signed CHIA Oath of Confidentialty A valid MUN ITS/Labnet account If requesting access to research data then ethics approval docuements are required

28 ACCESS REQUESTS In order to access CHIA services you will also need to submit a request to allow your systems access. Assets must be MUN (HSIMS/ITS) provided system Updated MUN McAfee virus scan/epo Simply provide us with the following: Your affiliation with MUN (Staff, Student, Researcher,etc) Identify what research group you belong to System Hostname IP Address System/Service you wish to access