State of XSEDE John Towns PI and Project Director, XSEDE Director, Collaborative Cyberinfrastructure Programs, NCSA

Transcription

1 September 4, 2012 State of XSEDE John Towns PI and Project Director, XSEDE Director, Collaborative Cyberinfrastructure Programs, NCSA

2 XSEDE Vision The extreme Science and Engineering Discovery Environment (XSEDE): enhances the produc;vity of scien;sts and engineers by providing them with new and innova;ve capabili;es and thus facilitates scien;fic discovery while enabling transforma;onal science/engineering and innova;ve educa;onal programs 2

3 Motivation for XSEDE Scien;fic advancement across mul;ple disciplines requires a variety of resources and services and thus the availability of comprehensive cyberinfrastructure composed of heterogeneous digital resources. High- end computa;onal science is befer served if these capabili;es leverage the aggregate exper;se of a small number of leading ins6tu6ons rather than being fully centralized at a single ins;tu;on or being fully decentralized. 3

4 Science requires diverse digital capabilities XSEDE is a comprehensive, expertly managed and evolving set of advanced heterogeneous high- end digital services, integrated into a general- purpose infrastructure. XSEDE is about increased produc;vity of community and expanded capability leads to more science is some;mes the difference between a feasible project and an imprac;cal one lowers barriers to adop;on 4

5 Boundary Conditions and Principles for XSEDE XSEDE inherited TeraGrid environment XSEDE inherited TG community and their expecta;ons Point of view has changed not an HPC/CS/tech play about produc;vity and crea;ng the environment necessary to be produc;ve Finally figured out that the project must define a solu;on that is designed to evolve! aqer 25 years of not recognizing this Iden;fy the greatest needs and start there don t forget what you have learned both good and bad! 5

6 XSEDE Factoids: high order bits 5 year, $121M project op;on for addi;onal 5 years of funding upon major review aqer PY3 No funding for major hardware coordina;on, support and crea;ng a na;onal/ interna;onal cyberinfrastructure ~120 FTE funded across 17 partner ins;tu;ons 6

7 7 XSEDE Org Chart

8 What do you mean by Advanced Digital Services? OQen use the terms resources and services these should be interpreted very broadly most are likely not operated by XSEDE Examples of resources compute engines: HPC, HTC (high throughput compu;ng), campus, departmental, research group, project, data: simula;on output, input files, instrument data, repositories, public databases, private databases, instruments: telescopes, beam lines, sensor nets, shake tables, microscopes, infrastructure: local networks, wide- area networks, Examples of services collabora;on: wikis, forums, telepresence, data: data transport, data management, sharing, cura;on, provenance, access/used: authen;ca;on, authoriza;on, accoun;ng, coordina;on: meta- queuing, support: helpdesk, consul;ng, ECSS, training, And many more: educa;on, outreach, community building, 8

9 Highest Level Highlights for Program Year 1 All areas of project have been spun up and delivered value Significant science impact! to date: 1,978 projects, 8,695 dis;nct users, 32 NSF divisions, 1,833 publica;ons Seamless transi;on from TeraGrid to XSEDE Extended Collabora;ve Support Service (ECSS) supports new architectures and new disciplinary fields XSEDE User Portal solidly established as interface to the community Ini;ated new workforce development efforts including formalized cer;ficate and degree programs Fostering stronger community around campuses 9

10 Transparency: publicly available project information Rolling out documents at: documents Project Defini;on documents original Project Summary original Science Case (ge:ng approvals from science teams) Policy documents Service Provider Defini;on and applica;on template SPs as members of the XSEDE Federa;on Project Reports quarterly reports program plans and annual reports as they are available Architecture and Design documents Level 1 and Level 2 Decomposi;on Campus Bridging use cases and Quality AFribute Scenarios Level 3 decomposi6ons (first in review now) Engineering documents System Requirements Specifica;on (Requirements Baseline) Produc;on Environment and Opera;ons documents SoQware and Service Baseline Technical Security Baseline 10

11 XSEDE Advisory Functions XSEDE Advisory Board guidance to help XSEDE achieve the maximum impact assist with requirements gathering and priori;za;on assist with annual planning process review the annual report and plans for NSF recommend strategic direc;ons User Advisory CommiFee represent the user s voice to XSEDE management review quarterly and annual reports review ASCC proposals review annual User Survey Results Service Provider Forum forum for Service Providers that deliver services via XSEDE means by which Service Providers have input into XSEDE s management venue via which XSEDE management may communicate effec;vely with SPs 11

12 Approach to new disciplines XSEDE is making a proac;ve effort to iden;fy and reach out to subfields that are not tradi;onal XSEDE users that may be enabled to make transforma;ve breakthroughs with resources brought about by XSEDE s scale and availability of resources. Examples- genomics, computa6onal finance, machine learning, language processing expect to see results in the coming year 12

13 Novel and Innovative Projects Mission: provide proac6ve, sustained efforts to jump- start XSEDE projects by non- tradi6onal (to HPC/CI) users Major addi;on to past efforts genomics, economics, digital humani;es, machine learning successful XSEDE users as bridgeheads into their communi;es connect us to colleagues, point/invite us to mee;ngs, advise us on what does (not) work. get advice from NSF program officers: SBE, BIO workshop Extending High- Performance Compu;ng Beyond its Tradi;onal User Communi;es at the IEEE escience 2012 conference (Chicago, October) 13

14 XSEDE Outreach GOAL: Recruit a larger and more diverse scien6fic, academic, and industrial workforce capable of advancing scien6fic discovery using XSEDE services. Student Engagement provide meaningful experiences for undergraduate and graduate students to become engaged in extreme digital environments Underrepresented Engagement engage under- represented faculty, staff and students in u;lizing XSEDE resources and services Campus Champions build broader and deeper community engagement via XSEDE trained representa;ves on campuses across the country 14

15 Campus Bridging Mission/Purpose for XSEDE: to provide virtual proximity seamless integra;on for researcher and student from their worksta;on to campus CI to XSEDE un;l seamlessness is achieved reduce barriers to integra;on of campus and XSEDE cyberinfrastructure Currently 7 specific use cases defined, cons;tute basis for Campus Bridging work plans focus on produc;on deployments of soqware tools and documenta;on resources 15

16 Innovation: proactively looking to expand scope of capabilities Striking a balance between providing stable, reliable services that foster innova;on both in what we are doing and how we do it Campus Bridging use cases are mostly for capabili;es we have not tradi;onally supported NIP seeking out new communi;es and iden;fying new capabili;es necessary to support them Architecture design process specifically addresses XUAS capabili;es for all Level 3 decomposi;ons explicitly support innova;on by the project more importantly, facilitate innova;on by the community Our goal is to deliver new capabili6es faster and faster 16

17 Defining an Inclusive Architecture: Key Areas Campus Bridging (Craig Stewart & Rich Knepper) Use Cases, Quality AFribute Scenarios Level 3 decomposi;on in progress Science Gateways Lead stakeholders: Suresh Marru and Nancy Wilkins- Diehr CompuAng High Performance High Throughput Scien;fic Workflows Big Data Data Analy;cs Data Movement, Storage, Backup & Archival Visualiza;on ConnecAng InstrumentaAon 17

18 Approach to Other Cyberinfrastructures Increased interac;on with Open Science Grid (OSG) the na;on s premier high- throughput compu;ng infrastructure has become an XSEDE Service Provider Joint ac;vi;es with PRACE Summer Schools annually planning joint alloca;on call for collabora;ng teams Soon, users will be able to move rather seamlessly between these infrastructures (joint alloca;ons, some shared user support). 18

19 Working with other CI Providers Realizing a na;onal/interna;onal CI ecosystem requires federa;on of CI providers OSG: the na;on s premier high- throughput compu;ng infrastructure ;es to CI (escience infrastructure) providers interna;onally is a Level 1 Service Provider in XSEDE PRACE (including DEISA) is a significant CI in Europe joint Summer School series working toward joint alloca;ons call later this calendar year Soon, users will be able to move rather seamlessly between these infrastructures 19

20 Big Plans for the Coming Year Transi6oning from a project in start- up mode to regular delivery of value Facilitate broad range of ground- breaking research Introduce Standard alloca;ons requests streamlining the process of getng larger than startup awards Deliver training on new resources, with new technologies, to be deployed in XSEDE Pursue new disciplinary areas with Novel and Innova;ve Projects Formal adop;on of undergraduate and graduate cer;ficate and degree programs working with 12 ins;tu;ons Deliver new or improved soqware products to XSEDE opera;ons and to XSEDE users on a regular basis Wide Area filesystem deployment Ini;ate joint ac;vi;es with industry 20

21 XSEDE14 Site: Atlanta Contact ScoF Lathrop to volunteer! 21

22 XSEDE 13: Gateway to Discovery San Diego, CA July 22-25, 2013

23 See you in San Diego next year! Key leads: Amit Majumdar, Warren Froelich, Susan McKenna, Jeff Pummill, Jeff Gaede, Tom HuFon, ScoF Lathrop Track Chairs: Amit Majumdar, Jan Odegard, Suresh Marru, Renato Figueireda, Raj Ketmuthu, Tom Hacker, Galen Arnold Logis;cs & Local Arrangements Chairs: Warren Froelich, Diane Baxter, Ange Mason Possible biotech focus day Looking for program committee members and chairs Contact anyone listed or 23

24