CFD Workflows + OS CLOUD with OW2 ProActive

Transcription

1 CFD Workflows + OS CLOUD with OW2 ProActive Denis Caromel Agenda 1. Background: INRIA, ActiveEon 2. ProActive Parallel Suite 3. Use Cases & Demos Renault OMD2 Use Cases Workflow Acceleration -- Parallelism Scheduling, Resource & Data Management, Deployment 1 1

2 1. Background 2 2

3 OASIS Team was started in the team 3 3

4 Technology Transfer in 2007 Located in Sophia Antipolis, between Nice and Cannes, France Visitors Welcome! 4 4

5 ActiveEon Overview ActiveEon, a software company born of INRIA, founded in 2007, HQ in the French scientific park Sophia Antipolis Developping, with INRIA contributions, ProActive Parallel Suite, a Professional Open Source middleware for parallel, distributed, multicore computing Core mission: Scale Beyond Limits Providing a full range of services for ProActive Parallel Suite Worldwide production customers and users: 5

6 A Wide Range of Services Training and Certification Accelerate learning process Consulting Optimize your infrastructure and maximize ROI Technical Support - Subscription The guarantee of a quick and efficient assistance Integration-Development Get ActiveEon s products fine tuned to your specific needs Partnerships With OEMs and ISVs 6

7 2. ProActive Parallel Suite 7 7

8 HPC Workflow & Parallelization Scheduling & Orchestration Cloud & Grid IaaS 8

9 3. Use Case and Demo OMD2 Renault Distributed Multi-Disciplinary Optimizations 9 9

10 Coupling Mechanics, Aerodynamics Conduit d'admission 350 P2 P min CPU P4 P6 P3 P5 P7 P8 P10 P9 <1min CPU D Air Conditionning P12 P D Air Conditionning 100h CPU 1000h CPU Cylinder Head External Aerodynamic 10

11 ProActive Renault Use Case 1000 Cores Production Cloud Portal 11 11

12 Remote Visualization Directly from Portal 12

13 Demo: on ProActive PACA Grid Platform Production Platform operated by: Total: Cores 480 GPU CUDA Cores 150TB Storage Publically Available 13

14 Workflow Studio 14

15 Application/ISV Ready with APIs: REST, Java, C/C++, CLI 15

16 Portal with Graphical Visu of Workflow Execution 16

17 Workflow Studio 17

18 VMs from PMs 18

19 VMs from PMs 19

20 Renault Navier-Stokes Flow Simulation with STARCCM on 64 nodes Objective: Minimize Air Depression at the back of the car 20

21 Renault Navier-Stokes Flow Simulation with STARCCM on 64 nodes 21

22 Resulting Air Penetration Cofficient 22

23 Monitoring Simulation Time per Iteration 23

24 Integration with Applications 24 24

25 Integration: Scilab and Matlab, Applications Static Policy LSF Timing Policy 12/24 Desktops Dynamic Workload Policy EC2 Dedicated resources Desktops Amazon EC2 25

26 Mines St Etienne Use Case: Distributed turbulent flow simulations Use within Scilab environment Deployment of 500 independent tasks in parallel Each task takes 1 to 10 hours to execute We have applied ProActive Scheduler Server with its built in client Scilab extension. We were able to run about 500 independent tasks in parallel to test our numerical optimization algorithms, a single task would take from one to ten hours to execute, it would not have been possible to achieve the same results when running the experiments in a serial mode on a local computer. We have used the grid to run distributed turbulent flow simulations designed jointly by Renault and many French universities. The use of PA technology greatly simplifies parallel programming with futures paradigm as it allows for quick prototyping in dynamically interpreted environments such as Scilab or Matlab. There is no clutter created by static type systems, explicit memory management or archaic library dependency management, Rodolphe Le Riche and Ramunas Girdziusas, École des Mines de Saint-Etienne 26

27 Use Case: Hydrodynamic with K-Epsilon and FineMarine 27 27

28 Hydrodynamic Optimization: Workflow generated from a GUI ProActive Studio Graphical Workflow Editor 28 28

29 Hydrodynamic Optimization: Execution 29 29

30 Hydrodynamic: Remote Steering during execution 30 30

31 IFP Energies Nouvelles Production User (Press Release) 31 31

32 IFP Energies Nouvelles Use Case (2) Machine Types: Workstations PC, Laptop, Virtual Machines (vmplayer Windows sur PC Linux), LSF Cluster Various OS : Windows XP et 7, Linux Centos Application Types: Internal Software CO2 Analysis Model for Petroleum Molecular Dynamics Integration with Proprietary software Matlab, for instance for Engine Combustion 32

33 IFP Energies Nouvelles Use Case Deployment in production for all sites of IFP EN On over 600 computers Demanding applications: business workflows, numerical and financial simulations, Matlab and Scilab, data analysis ( Map / Reduce) Web-based portal as well as RCP and APIs : from within Application 33

34 IFP Energies Nouvelles Use Case Deployment in production for all sites of IFP EN On over 600 computers Demanding applications: business workflows, numerical and financial simulations, Matlab and Scilab, data analysis ( Map / Reduce) Web-based portal as well as RCP and APIs With the adoption of ProActive [ ] IFP EN enters in the era of Cloud Computing. We are going to cut on our hardware and software costs, to strengthen our business workflows, to use these globalized resources directly in our business software in order to accelerate them, Frédéric Gauluet, IFP EN 34

35 4. Conclusion 35 35

36 Open the Missing Links Application Acceleration, Workflows: Script, GUI Advanced Scheduling and Mapping Resource Control Core, CPU, Host Heterogeneity Support Physical Machines: Linux, Windows, Mac Desktop, Clusters, Grids, Clouds 36

37 Request from CFD experts From Jean-Marie Le Gouez: Use of various resources (Cluster, Desktop) Description of resources Use of GPUs On the fly post-processing (No intermediate storage), for Model Coupling and Visu. From Michel Ravachol: Collaborative Vizualisation 37

38 Thank You! Extra Material and Use Cases Below