Migrating to Rose & Cylc ISENES2 Workshop on Workflow Solutions in Earth System Modelling

Transcription

1 Migrating to Rose & Cylc ISENES2 Workshop on Workflow Solutions in Earth System Modelling David Matthews, June 2014

2 Contents This presentation covers the following areas Why? How? When? Challenges Future?

3 Why? April 2010: Completed analysis of issues with our software infrastructure for running scientific suites and applications Overly complex. Fragmented (lots of duplication). Hard to maintain. Hard for collaborators to install. Jobs (suites) were not repeatable & hard to exchange. No version control. References to external files (e.g. UMUI hand edits). Jobs required correct UI version to run. Forced to use GUI

4 How? June 2010: Strategy agreed Complete replacement of existing infrastructure (rather than incremental improvement to what we had) A common framework to run all types of experiments as suites (from the simplest to the most complex) Use existing scheduler (SMS or Cylc) Suites are a programming task (don t constrain via a GUI) Experiments defined using simple self contained text files (GUI optional) kept under version control An user interface driven by metadata Keep it simple!

5 Choice of scheduler Nov 2011: Chose cylc (rather than SMS/ECFlow) (More?) powerful Designed around the concept of cycling tasks Technology choices closely aligned with Rose (INI file, Python, PyGTK) Ability to influence and contribute to its development Open source license

6 Further milestones Oct 2012: Rose released under GPL license & development moved onto github Nov 2012: System ready for general use Nov 2012: Work started on operational implementation Dec 2012: First training course run Jan 2014: Operational implementation Jun 2014: First UM release fully configurable via Rose GUI

7 Current status All NWP trials and operations now using Rose/Cylc Migration of climate users about to begin with UM 9.1 release

8 Operational implementation Massive complexity! Likely to only get worse Some necessary, some not - aim is to remove unnecessary complexity Total of 16 suites currently global suite (most complex) runs global deterministic + ensemble + sea-ice analysis ~1800 tasks per day Benefits? Much improved change control Full complexity of suite exposed

9 UM Migration Preparing the UM to use Rose has been a lot of work UMUI extremely complex & hard to replace Hides some of the complexity of the inputs New UI release for each model release Testing requires hand-edits Major review of UM inputs required for use with Rose Lots of suite functionality built into UM system Post processing Job re-submission Archiving Benefits? Already benefitting from much improved test framework

10 Challenges: Installation Simple for individuals to set up More complicated setup required to support full functionality for a large group of users running tasks across lots of different servers Note: Can use parts of Rose without Cylc e.g. laptop users running simple suites, SMS users

11 installation Multiple cylc servers used to spread the load Also need server(s) for: rose bush rosie repository & web service Running cylc gui remotely difficult without shared $HOME

12 Challenges: Communication cylc server needs to be able to ssh to server running task (non-interactive) Task needs to be able to communicate back to server running cylc (via ssh or Pyro port) Alternatively cylc can poll for task status but this does not scale to large suites or lots of users No common solution?!

13 Challenges: Portable suites How to design suites which can easily be configured to be used on different platforms? Directives + other system specific configuration e.g. polling svn access need to split build process if system doing the build cannot access the repositories Changes to resources (e.g. number of nodes) which impact on application configurations (performance tuning, etc) are a more general problem, not just a portability issue

14 Future challenges Complete the migration & reap the benefits e.g. UM users, esp. climate Enhance Rose & Cylc to cope with ever more complex suites (e.g. larger ensembles) Previous systems would not have coped! Improve suite portability

15 Summary Change is hard Change is slow in our environment Life is getting more complex Rose / Cylc arriving just in time to help with this Lots of work still to do in order to reap the benefits

16 Questions and answers