Coming to you from Austin, TX and Logan, UT December 1 st, 2006 To begin at 3:05 ET

Transcription

1 CUAHSI Fall 2006 Cyberseminar Series David Maidment, University of Texas, Austin David Tarboton, Utah State University Coming to you from Austin, TX and Logan, UT December 1 st, 2006 To begin at 3:05 ET The CUAHSI Hydrologic Information System Version 1.0.

2 Welcome to the 7 th Semester of CUAHSI Education and Outreach Cyberseminars Host: Jon Duncan CUAHSI Project Manager Organizer: Dave Tarboton Utah State Univ. Problems? Send a chat to Host Feedback? Send an to commgr@cuahsi.org Seminar Ideas to: dtarb@cc.usu.edu This presentation can be downloaded from

3 Fall 2006 Lineup To be rescheduled in early 2007: Dave Goodrich, ARS Lessons learned from the USDA-ARS ARS Walnut Gulch Experimental Watershed and expansion into the Upper San Pedro Basin

4 CUAHSI Hydrologic Information System Update David R. Maidment (PI) David G Tarboton Ilya Zaslavsky Michael Piasecki Jon Goodall With support from collaborators, postdocs and graduate students: Rick Hooper, Jon Duncan, David Valentine, Tom Whitenack, Jeff Horsburgh, Bora Beran, Tim Whiteaker, Ernest To, Cedric David

5 Definition The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.

6 CUAHSI HIS Update Current Status of the Project HIS 1.0 Components WaterOneFlow web services Observations data model Digital Watershed

7 CUAHSI HIS Update Current Status of the Project HIS 1.0 Components WaterOneFlow web services Observations data model Digital Watershed

8 HIS Progress Phase 1 grant, 2.5 year grant completed Phase 2 grant, 5-yr, $4.5M to Maidment (Texas) Co-PI s Ilya Zaslavsky (San Diego Supercomputing Center) David Tarboton (Utah State) Michael Piasecki (Drexel) Jon Goodall (Duke) WATERS Testbeds serve as beta-testers for software First community-wide distribution in months (dependant on establishing support services)

9 CUAHSI HIS Components (in development) Project co-pi in Phase 2 Collaborator in Phase I

10 HIS Team and its Cyberinfrastructure Partners Government: USGS, EPA, NCDC, USDA CUAHSI HIS Industry: ESRI, Kisters, Microsoft Domain Sciences: Unidata, NCAR LTER, GEON HIS Team: Texas, SDSC, Utah, Drexel, Duke Super Computer Centers: NCSA, TACC

11 WATERS Network Information System HIS Team WATERS Testbed

12 HIS, WATERS and the CUAHSI Community Government: USGS, EPA, NCDC, USDA CUAHSI HIS WATERS Network Information System Industry: ESRI, Kisters, OpenMI HIS Team Domain Sciences: Unidata, NCAR LTER, GEON WATERS Testbed Super computer Centers: NCSA, TACC

13 International Partners European Commission Water database design and model integration (HarmonIT and OpenMI) Government: USGS, EPA, NCDC, USDA CUAHSI HIS WATERS Network Information System Industry: ESRI, Kisters, OpenMI Domain Sciences: Unidata, NCAR LTER, GEON HIS Team WATERS Testbed Super computer Centers: NCSA, TACC CSIRO Land and Water Resources Water Resources Observations Network (WRON)

14 CUAHSI HIS Update Current Status of the Project HIS 1.0 Components WaterOneFlow web services Observations data model Digital Watershed

15 Water quantity and quality Water Data Soil water Rainfall & Snow Remote sensing Meteorology Modeling

16 Water Data Web Sites

17 NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS NEUSE RIVER NEAR CLAYTON, NC # agency_cd site_no dv_dt dv_va dv_cd USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI s GetValues function

18 Observation Stations Map for the US Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System USGS National Water Information System NOAA Climate Reference Network

19 Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

20 Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

21 Geographic Integration of Storet and TCEQ Data in HIS

22 Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

23 CUAHSI Hydrologic Data Access System (being built using HIS Server in collaboration with ESRI) EPA NCDC NASA NWS USGS Observatory Data A common data window for accessing, viewing and downloading hydrologic information

24 Supports data discovery, delivery and publication Data discovery how do I find the data I want? Map interface and observations catalogs Metadata based Search Data delivery how do I acquire the data I want? Use web services or retrieve from local database Data Publication how do I publish my observation data? Use Observations Data Model HIS Server

25 HIS Server and Analyst HIS Server HIS Analyst Implemented at San Diego Supercomputer Center and at academic departments and research centers Sustainable industrial strength technology Web Services Details of HIS Analyst are here Implemented by individual hydrologic scientists using their own analysis environments Flexible any operating system, model, programming language or application

26 Point Observations Information Model USGS Data Source Streamflow gages Network Neuse River near Clayton, NC Sites Discharge, stage (Daily or instantaneous) Variables 206 cfs, 13 August 2006 Values {Value, Time, Qualifier} A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value

27 Example: Matlab use of CUAHSI Web Services % create NWIS Class and an instance of the class createclassfromwsdl(' w/nwis/dailyvalues.asmx?wsdl'); WS = NWISDailyValues; % Site Info for Site of Interest siteid='nwis: '; strsite=getsiteinfoobject(ws,siteid,''); strsite.site.siteinfo.sitename ans = NEUSE RIVER NEAR CLAYTON, NC lat=strsite.site.siteinfo.geolocation.geoglocation.lat itude long=strsite.site.siteinfo.geolocation.geoglocation.lo ngitude lat = long =

28 Variable and variabletimeinterval strsite.site.seriescatalog(1).series(:).variable ans = variablecode: '00065' variablename: 'Gage height, feet' units: 'international foot' ans = variablecode: '00060' variablename: 'Discharge, cubic feet per second' units: 'cubic feet per second' strsite.site.seriescatalog(1).series(:).variabletimeint erval ans = begindatetime: ' T00:00:00' enddatetime: ' T00:00:00' ans = begindatetime: ' T00:00:00' enddatetime: ' T00:00:00'

29 getvariableinfo varcode='nwis:00060'; varinfo=getvariableinfoobject(ws,varcode,'') varinfo = variables: [1x1 struct] varinfo.variables.variable ans = variablecode: '00060' variablename: 'Discharge, cubic feet per second' units: 'cubic feet per second'

30 GetValues % GetValues to get the data siteid='nwis: '; bdate=' t00:00:00'; edate=' t00:00:00'; variable='nwis:00060'; valuesxml=getvalues(ws,siteid,variable,bdate,edate,'');

31 Parse XML and Analyze % Parse the XML into a Matlab object to work with valuesobj=xml_parseany(valuesxml);... plot(date,flowval);datetick; 2.5 x 104 Daily Discharge NEUSE RIVER NEAR CLAYTON, NC cfs

32 Data Sources Storet NASA Ameriflux Extract NCDC Unidata NWIS NCAR Transform CUAHSI Web Services Excel ArcGIS Visual Basic C/C++ Load Applications Matlab Access Java Fortran Some operational services

33 Objective Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them What we are doing now.. NWIS request return request return request return NAWQA request return request return NAM-12 request return request return request return Michael Piasecki Drexel University NARR

34 Semantic Mediator GetValues What we would like to do.. GetValues NWIS GetValues generic request GetValues GetValues GetValues NAWQA Michael Piasecki Drexel University GetValues GetValues NARR HODM

35 CUAHSI HIS Update Current Status of the Project HIS 1.0 Components WaterOneFlow web services Observations data model Digital Watershed

36 Hydrologic Science It is as important to represent hydrologic environments precisely with data as it is to represent hydrologic processes with equations Hydrologic Process Science (Equations, simulation models, prediction) Physical laws and principles (Mass, momentum, energy, chemistry) Hydrologic conditions (Fluxes, flows, concentrations) Hydrologic Information Science (Observations, data models, visualization Hydrologic environment (Dynamic earth)

37 Data Cube A simple data model Time, T When D Where Space, L Variables, V What

38 Continuous Space-Time Model NetCDF (Unidata) Time, T D Coordinate dimensions {X} Space, L Variables, V Variable dimensions {Y}

39 Discrete Space-Time Data Model ArcHydro Time, TSDateTime TSValue Space, FeatureID Variables, TSTypeID

40 Terrain Data Models Grid TIN Contour and flowline

41 CUAHSI Observations Data Model A relational database at the single observation level (atomic model) Stores observation data made at points Metadata for unambiguous interpretation Traceable heritage from raw measurements to usable information Precipitation & Climate Streamflow Water Quality Groundwater levels Soil moisture data Flux tower data

42 Hydrologic Observations Data Model What are the basic attributes to be associated with each single observation and how can these best be organized? Data Source and Network Sites Variables Values Metadata Controlled Vocabulary Tables e.g. mg/kg, cfs e.g. depth Streamflow Landuse, Vegetation Depth of snow pack e.g. Non-detect,Estimated, Windspeed, Precipitation A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured Data Discovery A variable is a property describing the flow or quality of water See A value is an observation of a variable at a particular time Metadata provide information about the context of the observation. Data Delivery Ernest To Center for Research in Water Resources University of Texas at Austin

43 Independent of, but coupled to Geographic Representation HODM Arc Hydro Hydrologic Observations Data Model Feature MonitoringPoint SiteID SiteCode SiteName Latitude Longitude 1 1 OR CouplingTable SiteID (GUID) HydroID (Integer) 1 1 HydroPoint HydroID HydroCode FType Name JunctionID ComplexEdgeFeature HydroEdge HydroID HydroCode ReachCode Name LengthKm LengthDown FlowDir FType EdgeType Enabled * Waterbody HydroID HydroCode FType Name AreaSqKm JunctionID * HydroNetwork SimpleJunctionFeature 1 HydroJunction HydroID HydroCode NextDownID LengthDown DrainArea FType Enabled AncillaryRole Watershed HydroID HydroCode DrainID AreaSqKm JunctionID NextDownID 1 * EdgeType Flowline Shoreline

44 Variable attributes Cubic meters per second L 3 /T m 3 /s VariableName, e.g. discharge VariableCode, e.g SampleMedium, e.g. water Valuetype, e.g. field observation, laboratory sample IsRegular, e.g. Yes for regular or No for intermittent TimeSupport (averaging interval for observation) DataType, e.g. Continuous, Instantaneous, Categorical GeneralCategory, e.g. Climate, Water Quality NoDataValue, e.g

45 Stage and Streamflow Example

46 Daily Average Discharge Example Daily Average Discharge Derived from 15 Minute Discharge Data

47 Water Chemistry from a profile in a lake

48 ODM and HIS in an Observatory Setting e.g.

49 HDAS Website Portal and Map Viewer Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses 3 rd party data servers e.g. USGS, NCDC Downloads Uploads HTML -XML WaterOneFlow Web Services WSDL - SOAP Data access through web services Data storage through web services Web services interface GIS Matlab IDL Observatory data servers CUAHSI HIS data servers Splus, R Excel Programming (Fortran, C, VB)

50 CUAHSI HIS Update Current Status of the Project HIS 1.0 Components WaterOneFlow web services Observations data model Digital Watershed

51 Digital Watershed How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

52 Digital Watershed Hydrologic Observation Data (Relational database) Digital Watershed Geospatial Data (GIS) Weather and Climate Data (NetCDF) Remote Sensing Data (EOS-HDF) A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

53 NHDPlus for Region 17E

54 NHDPlus Reach Catchments ~ 3km 2 Average reach length = 2km 2.3 million reaches for continental US About 1000 reach catchments in each 8-digit HUC

55 Reach Attributes Slope Elevation Mean annual flow Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

56

57 Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive Software standards for model linking Uses model core as an engine

58 OpenMI Conceptual Framework All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data Time, T D Variables, V Space, L VALUES An application of the data cube to integrate simulation models Jon Goodall, Duke University

59 HIS as OpenMI Components NWIS Streamflow water balance model To calculate storage, the model needs inflow and outflow Trigger: Calculate storage Daymet Precipitation To calculate storage, the model needs precipitation Goal: Link the National HIS web services with a simple water balance model using OpenMI as the mediator

60 Watershed Hydrovolumes Hydrovolume USGS Gaging stations Geovolume is the portion of a hydrovolume that contains solid earth materials

61 Stream channel Hydrovolumes Residence time distributions Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa)

62 Integration of surface water and groundwater data Describe the relationship between surface water features ( e.g. streams and waterbodies) with groundwater features (aquifers, wells). Enable the connection with the surface water data model Hydro network Aquifers In the future go to 3D...

63 Water OneFlow We need a Water OneFlow a common window for water data and models Federal State Local Academic Flow Precip Model Advancement of water science is critically dependent on integration of water information