ESIP Federation Interoperability Outreach Package January 29, 2011

ESIP Federation Interoperability Outreach Package January 29, 2011 Ali Rezaiyan, INNOVIM, LLC Christine Whalen, INNOVIM, LLC Acknowledgements The team would like to acknowledge the contributions of: Chris Lynnes, NASA GSFC, for technical review and input Rahul Ramachandran, ORNL, for technical guidance and recommendations James Marshall, INNOVIM, LLC, for advising on layout and editing materials Roger Gill, INNOVIM,LLC, for technical input Mary Hunter and Neal Most, INNOVIM, LLC, for staff and resource coordination and proposal material development Outline

Background Interoperability by Functionality Introduction Data Access Catalog Search and Discovery Usage and Formats Semantic Web

Case Studies Distributed Computing Platforms References 3 Background 4 About the ESIP and IT&I Cluster The ESIP Federation is a consortium of more than 110 organizations that collect, interpret, and develop applications for Earth observation information. Partners include NASA and NOAA as strategic funding partners (Type IV), USGS data centers, and research universities, as well as many other organizations involved in Earth science.

ESIPs Information Technology and Interoperability cluster was created, in part, to ensure that data, information and services can be readily discovered, exchanged and integrated through the use of interoperability standards and protocols. 5 Outreach Goal The ESIP Federation initiative coupled with community standards, such as those of the Open Geospatial Consortium (OGC) and existing legacy systems, will allow different scientific communities to work together to achieve a bigger vision. 6 About Interoperability The capability to communicate, execute programs, or

transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units. According to ISO/IEC 2382-01, Information Technology Vocabulary, Fundamental Terms The IEEE (Institute of Electrical and Electronics Engineers) defines interoperability as: the ability of two or more systems or components to exchange information and to use the information that has been exchanged. 7 Types of Interoperability Syntactic interoperability Semantic interoperability If two or more systems are capable of communicating and exchanging data, they are

exhibiting syntactic interoperability. Specified data formats, communication protocols and the like are fundamental. In general, XML or SQL standards provide syntactic interoperability. This is also true for lower-level data formats, such as ensuring alphabetical characters are stored in ASCII format in both of the communicating systems. Syntactical interoperability is a necessary condition for further interoperability. Beyond the ability of two or more computer systems to exchange information, semantic interoperability is the ability to automatically interpret the information exchanged meaningfully and accurately in order to produce useful results as defined by the end users of both systems. To achieve semantic interoperability, both sides must defer to a

common information exchange reference model. The content of the information exchange requests are unambiguously defined: what is sent is the same as what is understood. 8 Approaches to Interoperability Regardless of which path is selected, both require: There are two approaches: Build a new system which may incur higher cost

Utilize the pre-existing legacy system and bring it up to speed A service-oriented approach that underlies much of todays World Wide Web New systems shall use a serviceoriented approach to making their data available to distributed communities of scientists 9 Transaction-oriented and conceptual aspects of Web 2.0 Interoperability Standards

Open Geospatial Consortium (OGC) Standards ESIP Federation Open Search Open-source Project for a Network Data Access Protocol (OPeNDAP) Metadata File Formats 10 Interoperability by Functionality 11 Interoperability by Functionality (1/2) This Outreach package focuses on interoperability based on three main areas of functionality: Data Access Catalog Search and Discovery Usage and File Formats

12 Interoperability by Functionality (2/2) Catalog Catalog Search Search And Discovery And Discovery Data Access ESIP Open Search CS-W ebRIM GEOSS Search & Discovery Federated Search

OGC Standards WMS, WCS, WFS SWE, SSW, GSW OPeNDAP Meta Data File Formats Usage and File Formats FDGC NetCDF and CF ISO 19115 and 19119 NASA ECS and GCMD DIF 13 OGC Participants/Communities There are three primary groups of participants: 1. Data providers who should be able to download an API to implement OGC web services to distribute their data. 2. The community of users who can utilize a web-based visualization tool or client tool that is capable of communicating with data providers in

a form such as REST, SOAP, or XML. 3. Developers who implement, test, and integrate these systems/software and provide them to the user community. This developer community is formed by a group of experts in software engineering, standards, and database managers. 14 Interoperability Standards Interoperability standards include ones developed by the Open Geospatial Consortium (OGC), Federated Search, and OPeNDAP. Open Search Federated Search Data Access and

Catalog Search Standards 15 OGC OPeNDAP OGC Standards The Open Geospatial Consortium (OGC), via their standards process, has developed protocol standards implemented operationally at various level such as WMS, WFS, WCS, and CSW. Catalogue Service for the Web (CS-W) WMS Web Map Service (WMS) Flat Images

Web Future Service (WFS) Geospatial data Support Transaction Registers for Services Ontology Visualization Description GML Application CS-W WFS OGC SSW Semantic Sensor Web (SSW)

Sensor Web Enablement (SWE) SWE SOS SPS SAS WNS GSW O&M TML SensorML Geospatial Semantic Web (GSW) WCS Web Coverage Service (WCS) Geospatial types

Gridded data with GML Metadata 16 Data Access OGC Open Geospatial Consortium 17 Some OGC Standards Web Map Service (WMS)

Web Feature Service (WFS) Web Coverage Service (WCS) Sensor Web Enablement (SWE) Catalog Services - Web (CS-W) Geospatial Semantic Web (GSW) Semantic Sensor Web (SSW) 18 Web Map Service (WMS) Maps Client sends a request to the server using HTTP WMS Features OGC WMS standards serve visualizations of geographical data in flat map-like layered images. HTTP queries a WMS server and returns pre-rendered images from different

sources. The returned images from different sources can be layered on top of each other. 19 Client Server 1. GetCapabilities() XML Schema Capability Response 2. GetMap() Map Return WMS Examples (1/3) The following links are two XML examples for

GetCapabilities Request: Northern Hemisphere: http://nsidc.org/cgi-bin/atlas_north?service=WMS&reque st=GetCapabilities&version=1.1.1 Southern Hemisphere: http://nsidc.org/cgi-bin/atlas_south?service=WMS&reque st=GetCapabilities&version=1.1.1 20 WMS Examples (2/3) GetMap Northern Hemisphere request using the known parameters from GetCapabilities: http://nsidc.org/cgi-bin/atlas_north?service=WMS&version =1.1.1&request =GetMap&srs=EPSG:32661&f ormat=image/gif&width=1000&height=1000&bbox=-2700000,-2700000,6700000,6700000&layers=sea_ic e_extent_01,land,snow_extent_01,permafrost_extent,country_borders,treeline,north_pole_geographic,ar ctic_circle,country_labels,geographic_features_sea 21

WMS Examples (3/3) GetMap Response from Server: 22 Web Feature Service (WFS) Features Client sends a request to the server using HTTP WFS Features Client Publishes feature-level geospatial data to the web and supports INSERT, UPDATE, DELETE, LOCK, QUERY, and DISCOVERY operations on these data using HTTP WFS Provides three main Requests:

GetCapabilities: Describes its which feature types it can service and what operations are supported on each feature type. DescribeFeatureType: Describes the structure of any feature type it can service GetFeature: Allows retrieval of feature instances based on client specified request to constrain the query spatially and non-spatially. WFS Optional Requests: GetGmlObject: Provides retrieval of element instances by traversing XLinks that refer to their XML IDs. Transaction : Allows create, update, and delete operations on geographic features LockFeature: Provides lock request on one or more instances of a

feature type for the duration of a transaction to ensure support of serializable transactions 23 Server 1. GetCapabilities() Ask data provider what is available and how I can request them XML Schema Capability Response 2. DescribeFeature(param) Describing

feature and what will be expected if request is made Request for desired GML feature XML Schema Response 3. GetFeature(param) GML Features Insert/Update/Delete WFS Benefits Instead of returning an image as WMS does, WFS provides

detailed information about specific geospatial features of the underlying data, at geometry and attribute levels. Unlike the WMS getFeatureInfo request that only returns information about the feature, WFS provides the geometry itself. Whereas WMS offers imaging services, WFS provides geographical features as the source code behind the map. WFS has optional features that allow inserting and modifying geospatial objects such as weather observation readings. 24 WFS Examples (1/3) The following links are two examples for GetCapabilities Request: Northern Hemisphere: http://nsidc.org/cgi-bin/atlas_north?service=WFS&reques

t=GetCapabilities&version=1.1.0 Southern Hemisphere: http://nsidc.org/cgi-bin/atlas_south?service=WFS&reques t=GetCapabilities&version=1.1.0 25 WFS Examples (2/3) GetFeature Request: Elevation contours for the Greenland ice sheet: http://nsidc.org/cgi-bin/atlas_north?service=WFS&version=1.1.0&request=GetFeature &typename=greenland_elevation_contours 26 WFS Examples (3/3) WFS GetFeature XML Response from Server: -406699.951843 -456661.863393 1677349.428068 1589388.039689

Client OPeNDAP will result in following three different requests for data to OPeNDAP Server: Request for the Data Descriptor Structure (DDS): is a data structure to describe datasets and subsets of those datasets Shape of Data Syntactic Metadata - Rigid Request for the Data Attribute Structure (DAS): is a set of name-value pairs used to describe the data in a particular dataset Semantic Metadata - Flexible Request for actual Data (DODS): is data structure used by the DODS software to describe datasets

47 OPeNDAP Client Server Architecture (1/3) OPeNDAP Servers: CODAR netCDF HDF4/5 Matlab

DSP Tables (JGOFS) SQL (JDBC) FITS

CDF 48 OPeNDAP Client Server Architecture (2/3) OPeNDAP Clients : Ferret and GrADS (netCDF C) IDV, VisAD, and ncBrowse (netCDF Java) Matlab IDL Access Excel

49 OPeNDAP Client Server Architecture (3/3) OPeNDAP Servers OPeNDAP Clients 50 OPeNDAP Service at GSFC Currently, the GES DISC offers the datasets listed at http://disc.gsfc.nasa.gov/services/opendap/ through OPeNDAP. For example, AIRS data are made available through http://disc.sci.gsfc.nasa.gov/gesNews/opendap_AIRS_dat a_access 51

Uses of OPeNDAP OPeNDAP can be used to: Make netCDF data files available over the Internet Adapt existing software that use the netCDF API to read data served by an OPeNDAP data server In general, any program that uses netCDF can become a client in the OPeNDAP client-sever system. 52 OPeNDAP Support of IDV, GrADS, and Panoply There are a variety of tools and file formats used with Earth science data, including IDV, GrADS, and Panoply. In general, IDV and Panoply have the easiest learning curves while GrADS is somewhat harder. OPeNDAP allows for remote access of data at a sub-file

level, making the data easier to use. 53 IDV Example Carbon Monoxide Plume from California wildfires, seen in AIRS Daily Level 3 (AIRX3STD) on 29 Aug 2009: http://acdisc.gsfc.nasa.gov/opendap/Aqua_ AIRS_Level3/AIRX3STD.005/2009/AIRS.2 009.08.29.L3.RetStd001.v5.2.2.0.G092431 31454.hdf 54 Panoply Example Tropospheric NO2 from OMI for July 7, 2010: http://acdisc.gsfc.nasa.gov/opendap/HDF-EOS

5/Aura_OMI_Level3/OMNO2e.003//2010/OMIAura_L3-OMNO2e_2010m0707_v003-2010m 0708t161138.he5 54 Other Examples Additional examples for Panoply and IDV, as well as examples for GrADS can be found at: http://wiki.esipfed.org/index.php/Making_Science_Data_ Easier_to_Use_with_OPeNDAP This site also provides links to related tools. 56 Catalog Search and Discovery 57 Catalog Services Web (CS-W)

According to OGC specification, catalog services support the ability to publish and search: Collections of metadata (representing resource characteristics) Catalog services (required to support the discovery and binding to registered information resources) Other related information objects 58 CS-W Abilities CS-W Features Client Provides four main Requests:

GetCapabilities: Allows CS-W clients to retrieve service metadata from a server DescribeRecord: Allows a client to discover all data model supported by the catalogue service GetRecordById: Retrieves the default representation of catalogue records using their identifier. GetRecords: Allows record filtering using a single filter element that can contain a potentially complex filter expression 1. GetCapabilities() Ask data provider what is available and how I can request them

XML Schema Capability Response 2. DescribeRecord(param) Describing some or all of the information model to be described Returns detailed information selected object of service 59 Server XML Schema Response 3. GetRecordById (param)

Coverage in pre-defined format OpenSearch OpenSearch is a collection of simple format for sharing of search results OpenSerach formats allows clients to discover and use your search engine Allows search engines and search clients to communicate using the common set of formats Was developed and created by Amazon.com and A9.com 60 How to Use OpenSearch OpenSearch allows you to direct clients to your search engine or use

your search engine. To direct clients to your web site: Write a simple OpenSearch Description document to describe your search. See OpenSearch description XML document example in this link: http://www.opensearch.org/Specifications/OpenSearch/1.1#OpenSear ch_description_document If you wish clients to use your search engine: Syndicate your search results by formatting them with extended existing syndication formats, such as RSS or Atom formats, augmented with OpenSearch elements. See example provided in this link: http://www.opensearch.org/Specifications/OpenSearch/1.1#OpenSear ch_response_elements 61 OpenSearch Clients

OpenSearch search clients include: Search Aggregator Websites A9.com OSfeed TagJag Keywop Web browsers Internet Explorer 7: user instructions to install an opensearch provideralso supports OpenSearch Referrer Extension Firefox 2.0 - also support OpenSearch Suggestions Extension Arora - also supports OpenSearch Suggestions Extension and OpenSearch Referrer Extension Google Chrome 62 Writing OpenSearch (1/2) Writing OpenSearch results with various types of software

Alfresco Drupal OpenSearch Results by Robert Douglass Kwiki by Tatsuhiko Miyagawa Lucene Lucene by Apache.org Nutch PyOpenSearch example Python Whoosh application with jQuery client 63 Writing OpenSearch (2/2)

Additional types of software for writing OpenSearch OpenLink Data Spaces by OpenLink Software MediaWiki by Gregory Szorc Moveable Type by Alf Eaton PLOS (Plone OpenSearch) SearchGenerator (Ruby on Rails) Wordpress by Chris Fairbanks GeoNetwork opensource geospatial catalog 64

Reading OpenSearch Description Documents Reading OpenSearch Drupal client library by Robert Douglass Drupal OpenSearch Aggregator by Steven Wittens Perl library by Tatsuhiko Miyagawa and Brian Cassidy PHP Hirose Masaaki

Python library by Ed Summers ROME plugin OpenLink Virtuoso by OpenLink Software Apache Abdera OJAX LibraryFind open source metasearch application qopensearch - a set of Qt classes 65 ESIP Federation OpenSearch The ESIP Federation Open Search utilizes OpenSearch description XML document for Two Step Search: 1) Search for datasets and then granules within the selected dataset 2) Space-time granule query for the selected dataset 66

Usage and File Formats 67 Metadata and Related Standards (1/2) Federal Geographic Data Committee (FDGC) Metadata standards FGDC Content Standard for Digital Geospatial Metadata (CSDGM) and its extension of remote sensing data to design and implement catalogue service. NetCDF refers to a data model for array-oriented scientific data. Climate and Forecast (CF) conventions for netCDF serve as a interoperability standard. 68 Metadata and Related Standards (2/2)

Geospatial datasets (ISO 19115 parts I and II metadata) Geospatial services (ISO 19119) NASA ECS (EOS Core System) and remote sensing datasets NASA Global Change Master Directory (GCMD) DIF (Directory Interchange Format) see: http://gcmd.nasa.gov/User/difguide/difman.html 69 FGDC Metadata Standard The Content Standard for Digital Geospatial Metadata (CSDGM), Vers. 2 (FGDC-STD-001-1998) is the current US Federal Metadata standard. The FGDC originally adopted the CSDGM in 1994 and revised it in 1998. According to

Executive Order 12096, all Federal agencies are ordered to use this standard to document geospatial data created as of January 1995. The standard is often referred to as the 'FGDC Metadata Standard' and has been implemented beyond the federal level with State and local governments adopting the metadata standard as well. http://www.fgdc.gov/metadata/geospatial-metadata-standards 70 FGDC Content Standard for Digital Geospatial Metadata (CSDGM) The objectives of the standard are to provide a common set of terminology and definitions for the documentation of digital geospatial data. The standard establishes the names of data elements and compound elements (groups of data elements) to be used for these purposes, the definitions of these compound elements and data elements, and information about the values that are to be

provided for the data elements. 71 NetCDF NetCDF (network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of arrayoriented scientific data. Downloads are available at http://www.unidata.ucar.edu/downloads/index.jsp 72 Climate and Forecast (CF) Metadata The conventions for climate and forecast (CF) metadata are designed to promote the processing and sharing of files created with the NetCDF API. The conventions define metadata that provide a definitive description of what the data in each variable

represents, and the spatial and temporal properties of the data. This enables users of data from different sources to decide which quantities are comparable, and facilitates building applications with powerful extraction, regridding, and display capabilities. 73 Geospatial datasets (ISO 19115 part I and II metadata) ISO 19115:2003 defines the schema required for describing geographic information and services. It provides information about the identification, the extent, the quality, the spatial and temporal schema, spatial reference, and distribution of digital geographic data. This is revised by ISO 19115-2:2009 (Extensions for imagery and gridded data) and ISO/NP 19115-1, which is currently in development with a target publication date

around mid-2012. 74 Geospatial services (ISO 19119) ISO 19119:2005 identifies and defines the architecture patterns for service interfaces used for geographic information, defines its relationship to the Open Systems Environment model, presents a geographic services taxonomy and a list of example geographic services placed in the services taxonomy. It also prescribes how to create a platform-neutral service specification, how to derive conformant platform-specific service specifications, and provides guidelines for the selection and specification of geographic services from both platform-neutral and platform-specific perspectives. 75

NASA GCMD DIF (Directory Interchange Format) Originated in 1987 as the product of an Earth Science and Applications Data Systems workshop, as a step towards data system interoperability. GCMD DIF is a metadata file format. As the container for the metadata in the Committee on Earth Observation Satellites (CEOS) International Directory Network (IDN), it does not compete with other metadata standards. The metadata used by the GCMD is considered to be that set of attributes that are instrumental in helping users to determine if a data set meets their qualifications. The set of attributes (fields) and its associated syntax is known as the Directory Interchange Format (DIF). It has evolved over the years and serves the user community in the discovery of Earth science data.

76 Relational Database Management System (RDBMS) A relational database management system (RDBMS) is a database management system (DBMS) that is based on the relational model as introduced by E.F. Codd. Most popular commercial and open source databases currently in use are based on the relational database model. 77 Semantic Web 78 Semantics Semantics (from Greek smantiks) is the study of meaning. It typically focuses on the relation between

signifiers, such as words, phrases, signs and symbols, and what they stand for. 79 Semantic Web Semantic Web was coined by Tim Berners-Lee the inventor of WWW, HTTP, URLs, HTML and director of the World Wide Web Consortium (W3C) The Semantic Web is an extension of the World Wide Web through the embedding of additional semantic metadata, using semantic data modeling techniques such as Resource Description Framework (RDF) and OWL Web Ontology Language (OWL). 80 Syntactic Web vs. Semantic Web Syntactic Web

Semantic Web Interchange data is controlled by application Interpretation and identification of the data are done by human beings Utilizes common formats of data and combines them together from different sources

Provides capabilities to record how data are related to real world objects Allows humans or machines to start searching one database and move on to an unending set of databases Databases are not connected via wires, but by the same concepts. With the increase in volume of data and

complexity, it is virtually impossible to manage the data (Information Overload) Information Overload can pose a serious threat to Syntactic usefulness. 81 Semantic Web Example Purchasing a new computer Search criteria are: 17-inch screen, 4 GB of RAM, bonus software and games, lowest available price, new, free shipping or shipping cost less than $10 With Syntactic Web all you can do is search through different web pages and compare the conditions listed above, or use pages that compare dealers with available prices

With a Semantic Web Agent: User enters preferences listed above into computerized agent Agent will initiate a complex search through invisible metadata only visible to computers. Agent will display the best option, let you place an order, open your credit card payments, and mark the date of arrival 82 Geospatial Semantic Web (GSW) The idea behind GSW is to have discovery, query, and consumption of geospatial content based on formal semantic specification. An OGC Interoperability Experiment which aims to augment WFS/FE with a semantic query capability, through the definition of an ontology for the geospatial intelligence community. 83

GSW Interoperability The GSW will enable the meaning of geographic queries to be easily shared among different software systems and online services. 84 Semantic Sensor Web (SSW) The SSW is a framework for providing enhanced meaning for sensor observations by adding semantic annotation to existing SWE to: Provide more meaningful descriptions Provide more access to sensor data

Provide a mechanism to bridge the gap between syntactic XML-base metadata standards of SWE and Resource Description Framework (RDF) and Web Ontology Language (OWL) based metadata standards of Semantic Web 85 SOS and SWE Architecture Using SSW Technology SOS Client Sensor DB HTTP

Request SOS GetCapabilities DescribeSensor GetObservation 86 Semantic annotation of SWE SWE Semantic annotation of SWE SensorML Response

Ontology and Rules Case Studies 87 Case Studies Case studies illustrate the benefits of using interoperability standards. The following case studies are presented: MODIS Adaptive Processing Systems (MODAPS) WCS Web Services The NOAA-led Integrated Ocean Observing System (IOOS) NASAs sensor webs 88

MODIS Adaptive Processing System (MODAPS) WCS Web Services (1/3) Provides a SOA machine-to-machine API to level 1 and atmospheric archived MODIS (Moderate Resolution Imaging Spectroradiometer) data Utilizes standard Web Service interfaces through both Simple Object Access Protocol (SOAP) and Representational State Transfer (REST) protocols Allows OpenSearch and the Open Geospatial Consortiums Web Coverage Service (WCS) interfaces 89 MODAPS WCS Web Services (2/3) Provide synchronous web services that allows users to utilize WCS capabilities to perform: Exchange of information with OpenSearch and Open Geospatial Consortium

Gridded search Post processing 90 MODAPS WCS Web Services (3/3) MODAPS WCS Web Services Implementation Issues and Concerns WCS Issues and Concerns Implementation issues using AXIS2 MODAPS data volume and WCS capability WCS Schema validation and Java binding dilemma Our Approach Considering other Web Services Framework Using automated schema binding technologies 91 MODAPS WCS Web Services

Issues and Concerns SOA and AXIS2 Security AXIS2 Software Support Contract last vs. Contract first 92 MODAPS WCS Web Services WCS Schema Issues OGC WCS Schema WCS Schema fails to support MODAPSs large number of products XMLSpy fails to validate OGC WCS Schema version 1.0.0 Castor binding tools fail to create Java classes to support this Schema WCS Schema many based objects which are restricted This schema design causes failures within Castor binding

93 MODAPS WCS Web Services Lessons Learned Use experience with other technologies such as Spring Web Services instead of AXIS2 to eliminate: Dependencies to AXIS2 OMElement AXIS2 Security issues AXIS2 lack of software and security support Use XML binding COTS products such as Castor to eliminate manual implementation of Java API to handle OGC WCS Schema. Difficulty with validating WCS Schema, modifying the schema to be readable by the Castor binding tool, and at the same time protect the integrity of WCS Schema standards. 94

The NOAA-led Integrated Ocean Observing System (IOOS) The Integrated Ocean Observing System (IOOS) is a coordinated network of people and technology that work together to generate and disseminate continuous data on our coastal waters, Great Lakes, and oceans. By collecting and bringing data together in a way that ensures the information can be used with other data sets, IOOS will make a broader suite of data available to scientists, allowing them to develop a more complete characterization of our oceans and coasts. IOOS is a major shift in our approach to ocean observing, drawing together many networks of disparate, Federal and non-Federal observing systems to produce data, information, and products at the scales needed to support decision making. 95

About IOOS This figure shows an IOOS federated, service-oriented architecture. The Integrated Ocean Observing System (IOOS) provides information about open oceans and US coastal waters and Great Lakes to scientists, managers, businesses, governments, and the public in order to support research, to inform decision-making, and to enable new applications and derived products beyond the original intent of the data gathering. The US National Oceanic and Atmospheric Administration (NOAA) has been assigned the role of lead

federal agency in this endeavor. Technically, IOOS includes or interfaces with existing observing systems, data providers, and archives, and IOOS collaborates in developing additional capabilities in observations, data management and data use. The IOOS uses mostly OGC Sensor Observation Service (SOS) and Unidata's Data Access Protocol (DAP), and to a somewhat lesser extent (so far) OGC Web Coverage Service (WCS) and Web Map Service (WMS). 96 IOOS Data Access The NOAA IOOS program initiated development of a Data Integration Framework (DIF) to improve management and delivery of an initial subset of ocean observations.

The following services are the first to be established by the NOAA IOOS program and its partners to provide access to data: National Data Buoy Center (NDBC) Sensor Observation Service (SOS) CO-OPS SOS NDBC THREDDS Data Server SECOORA SOS 97 Sample National Data Buoy Center SOS Output GetObservation for WaterLevel

station_id,sensor_id,"latitu (sea_floor_depth_below_sea_surface) de (degree)","longitude (degree)",date_time,"sea_ Single point, Station 46403, floor_depth_below_sea_sur observation for a specific time, CSV face (m)","averaging format interval (s)" http://sdf.ndbc.noaa.gov/sos/server.ph p?request=GetObservation&service=S urn:ioos:station:wmo:46403,u OS&offering=urn:ioos:station:wmo:464 rn:ioos:sensor:wmo:46403: 03&observedproperty=sea_floor_dept :tsunameter0,52.65,h_below_sea_surface&responseforma 156.94,2008-07t=text/csv&eventtime=2008-07-17T00: 17T00:00:00Z,4509.488,900 00Z

98 Sample CO-OPS SOS Output Currents Data (GetObservation Service) Profile Bin Data (Latest), Station db0301 Result format text/xml 40 199.0 39.1

99 IOOS Software The IOOS also provides some software on an as-is basis with no support or warranty, including Server Code NDBC SOS software version 1 GCOOS SOS software beta version 0.6.1 THREDDS Data Server (TDS) Format Converters IOOS SOS to CSV v0.6.1 netCDF to BUFR converter beta v0.1 100 NASAs Sensor Webs (1/4)

All of these satellite and airborne sensors are, at least some of the time, using SensorML for geolocation and other purposes. See NASAs JPL and GSFC Sensorweb/EO-1 pages. 101 A number of NASA projects have adopted the OGC SWE suite of standards. Collaboration between JPL and GSFC with the Earth Observing 1 (EO-1) satellite is an important part. EO-1 was used in a wildfire sensor web scenario, and GSFC with partners prototyped a

transformation to an SWE framework using GeoBliki NASAs Sensor Webs (2/4) 102 The main objective of the sensorweb activity is to create an interoperable environment for a diverse set of satellite sensors via the use of software and the Internet to better understand physical phenomena [and] it facilitates science investigation. The end goal is to make

discovery and access to sensors as easy as finding and using websites on the Internet. NASAs Sensor Webs (3/4) 103 The basic components are: Reference architecture A set of web services Cross-domain generic scripting language Campaign Manager 1.0 Campaign Manager 1.0 API Identity Management Service

OGC Publish/Subscribe GeoTorrent The five components outlined in red in the figure at left were created by NASA to augment the international standards. NASAs Sensor Webs (4/4) Nearly 100 papers, over 30 presentations, and 16 articles on NASAs sensor web efforts are listed at http://eo1.gsfc.nasa.gov/new/sensorWebExp/Papers.html Subject areas include: An interoperable sensor architecture Autonomous sciencecraft experiments On-board diagnostics tools Adaptive phased array ground antennas 104 Additional Resources

A document on Practical Data Interoperability for Earth Sciences is available at http://www.esdswg.org/techinfusion/downloads/pdies 105 OpenGIS Members of the OGC have developed OpenGIS specifications for SQL, CORBA, and OLE/COM. These form the basis of the Spatial Web in that just as web sites implement HTML, sites serving spatial data or processing services will need to implement OpenGIS specifications in order to be part of the open spatial web. 106 OpenGIS Standards Some OGC OpenGIS standards have been

discussed already: Other OpenGIS standards include: WMS WFS WCS C-SW SensorML

107 GeoXACML OGC KML OpenGIS CityGML OpenGIS Location Service Computing Platforms 108 Computing Platforms The essential keys of meeting the complex challenges of sharing information among multiple sources are the abilities to:

Discover data Access data Integrate data Share information effectively 109 What are the Driving Factors? Internet Interoperable Web Services Client Interface Wireless Tools Thin Clients with limited bandwidth

Spatial Data Satellite imagery/data Variety of data providers Standards Roles Integration of Enterprises databases/ Components Infrastructure for Automated Distributed Computing Services 110 Frameworks Web Services SOA Distributed Objects

111 Web Services Web services represent a new architecture for creating applications that can be accessed from a different system. Web Services are self-describing, modular applications that can be published, located, and invoked dynamically over the web. Enables interoperability across: Distributed computing platforms Operating systems Programming languages Geographic boundaries 112 Web Services Framework Web Services Framework: Enables interoperable services over the standard

interfaces Supports publishing, discovery, and binding services Separates data instances from service instances Enables one providers services to be used by another providers data 113 Web Services Core Standards (1/3) Web Services Core Standards: HTTP

HTTPS SOAP XML XML Schema WSDL UDDI 114 Web Services Core Standards (2/3) Web Services builds on five key protocols: HTTP, XML, UDDI, WSDL, and SOAP HTTP and HTTPS are transport protocols. XML is a standard data format for implementing service interface and data exchange formats. UDDI, WSDL, and SOAP are used for discovering, describing, and invoking data, respectively. 115

Web Services Core Standards (3/3) UDDI WSDL SOAP XML XML XML HTTP/HTTPS HTTP/HTTPS HTTP/HTTPS

Discovery Description Invocation 116 Functionality Interface and data exchange format Protocol Service Oriented Architecture (SOA) The SOA provides a theoretical model for all Web Services. The SOA model contains three entities and three operations. 3. Service Bind()

Requester 4. Invoke/Utilize Services Registry/Broker 117 ) h( is bl Pu e ion ic t 1. rv rip Se esc D

2. To Find Se Di () Lo rvic sco an cati e ver d D on es cri p ti on Providers SOA Entities and operations The Service Requester is an application that is looking for services. To locate desired services, it turns to Registry and requests a Find() operation. The Registry is a well-known application that returns information regarding registers in response to search

criteria submitted by Find() operations. The Provider Publish() these details as well as information about how to access provider and connection details. Requester uses connection details to Bind() to the Provider. 118 Geospatial SOA Infrastructure (1/2) Consumers 3-D 2-D Clients Mobile Web SOA Infrastructure Applications

Mobile Web Subscription Tier Supporting Services Security Rich Clients Management Directory Semantics Services OGC Services

Internal Database/Files Open Search MAP Services Government Agencies Service Providers 119 Catalog Services Metadata Services

GEO Processing Services Geospatial SOA Infrastructure (2/2) Applications SOA Infrastructure Mobile -Mobile .NET -J2ME -PDA Apps Web -.Net -J2EE

-Spring Database/Data Files -ODBC -JDBC -RPC -Geo DB Data Access -WMS -WFS -WCS -SWE Supporting Services Rich Clients Security

-Heavily and Light Client -Highly Interactive -LDAP -Certificate -WS Security Semantics -OWL/RDF Management Directory -UDDI -OGC CS-W

with ebXML -JMX Services Open Search -CS-W -Federated Open Search -ESIP Open Search Metadata Services -FGDC -NetCDF 120 Geographic Information Services (GIS) -3D Globe Maps -GPS -Weather Data

-Google Map -GIS Analysis -GIS Modeling and Projections SOA vs. Distributed Objects SOA is opposite of the concept of distributed objects. With SOA: Data are exchanged between different systems where each system operates on its own local copy. Each system operates using its own local methods and procedure. Each system is decoupled from others. Not like distributed objects, each connected system has its own business object model. Therefore, the systems are allowed to scale. 121 Other Alternatives

Remote procedure call architectures such as Java RMI, DCOM, and CORBA HTTP-like transactional architectures like Servlet/JSP, ASP, PHP, and CGI Screen Scraper client-side program. It uses the existing interface by pushing data into the interface, and then scrapes the returned data off the interface and converts them into something the client application needs. 122 References 123 References citations listed in order of referenced in this presentation

http://wiki.esipfed.org/index.php/IT%26I_Chair New World Encyclopedia, (http://www.newworldencyclopedia.org/entry/Interoperability/Definition.) October 12, 2008 New World Encyclopedia, http://www.newworldencyclopedia.org/entry/Interoperability, October 10, 2008http://en.wikipedia.org/wiki/Interoperability http://nsidc.org/data/atlas/ogc_services.html, The National Snow and Ice Data Center, University of Colorado Boulder, CO 80309-0449. http://nsidc.org/data/atlas/ogc_services.html, The National Snow and Ice Data Center, University of Colorado Boulder, CO 80309-0449. The development of this map server application was supported by NASA's Earth Observing System (EOS) Program under contract NAS5-03099. Author: John Maurer.

http://nsidc.org/data/atlas/ogc_services.html, The National Snow and Ice Data Center, University of Colorado Boulder, CO 80309-0449. The development of this map server application was supported by NASA's Earth Observing System (EOS) Program under contract NAS5-03099. Author: John Maurer. http://nsidc.org/cgi-bin/atlas_north?service=WFS&version=1.1.0&request=GetFeature&typename= greenland_elevation_contours The National Snow and Ice Data Center, University of Colorado Boulder, CO 80309-0449. The development of this map server application was supported by NASA's Earth Observing System (EOS) Program under contract NAS5-03099. Author: John Maurer. http://docs.geoserver.org/latest/en/user/services/wcs/reference.html, Geoserver, Open Geospatial Consortium (OGC), December 17, 2010. http://docs.geoserver.org/latest/en/user/services/wcs/reference.html, Open Geospatial Consortium, copyright 2009 GeoServer.

http://nsidc.org/cgi-bin/atlas_north?service=WCS&request=GetCapabilities&version=1.1.1 The National Snow and Ice Data Center, University of Colorado Boulder, CO 80309-0449. The development of this map server application was supported by NASA's Earth Observing System (EOS) Program under contract NAS5-03099. Author: John Maurer 124 References (cont.) Botts, Mike; Robin, Alex; Davidson, John; Simonis, Ingo. Open GIS Discussion Paper. Open Geospatial Consortium Inc., version 1.0. 06021r1, March 4, 2006. Botts, Mike; Percivall, George; Reed, Carl; Davidson, John. OGC White Paper, OGC Sensor Web Enablement: Overview and High Level Architecture. Open Geospatial Consortium Inc., version 3., page 4 and 10, OGC 7-165, December 28, 2007.

Botts, Mike., University of Alabama-Huntsville, and McKee Lance, Open Geospatial Consortium Inc. , 2010. http://www.opengeospatial.org/standards/sensorml Sensors . A Sensor Model Language: Moving Sensor Data onto the Internet. April 1, 2003. http://www.sensorsmag.com/networking-communications/a-sensor-model-language-moving-sensor-data-internet-967. Gallagher, James. Accessing the DDS object, revision 1.10 . OPeNDAP, Inc., April 24 2004. http://www.opendap.org/api/wc-html/writing_client_6.html. OPeNDAP, Inc., 2008, December 4, 2010. http://opendap.org/faq/whatServers.html, OPeNDAP.

OPeNDAP, Inc., 2008, December 4, 2010. http://www.opendap.org/faq/whatClients.html. Yang, Kent and Lee, Joe. The HDF Group, HDF/HDF-EOS Workshop XIV Slide 9, 10, September 28, 2010. http://hdfeos.org/workshops/ws14/presentations/day1/OPeNDAP_tutorial_WS14.pptx. NASAs Science Mission Directorate (SMD), archived and distributed by Goddard Earth Sciences Data and Information and Services Center, National Aeronautics and Space Administration. July 19, 2010. http://disc.gsfc.nasa.gov/services/opendap/. 125 References (cont.)

NASAs Science Mission Directorate (SMD) , archived and distributed by Goddard Earth Sciences (GES) Data and Information and Services Center (DISC), National Aeronautics and Space Administration. July 19, 2010. http://disc.sci.gsfc.nasa.gov/gesNews/opendap_AIRS_data_access http://wiki.esipfed.org/index.php/Making_Science_Data_Easier_to_Use_with_OPeNDAP Making Science Data Easier to Use with OPeNDAP, ESIP Federation wiki, July 20, 2010. Tool developers at Goddard Institute for Space Studies (Panoply), Unidata (IDV), U. Wisconsin (McIDAS-V), Pacific Marine Environmental Laboratory (Ferret) and Institute of Global Environment and Society (GrADS), ESIP Federation wiki, July 20, 2010. http://wiki.esipfed.org/index.php/Making_Science_Data_Easier_to_Use_with_OPeNDAP#IDV_Examples

http://acdisc.gsfc.nasa.gov/opendap/HDF-EOS5/Aura_OMI_Level3/OMNO2e.003//2010/OMI-Aura_L3-OMNO2e_2010m0707_v003-2010m0 708t161138.he5 http://wiki.esipfed.org/index.php/Making_Science_Data_Easier_to_Use_with_OPeNDAP 126 References (cont.) Making Science Data Easier to Use with OPeNDAP, ESIP Federation wiki, July 20, 2010. A9.com, Inc. http://www.opensearch.org/Home.

A9.com, Inc. http://www.opensearch.org/Specifications/OpenSearch/1.1#OpenSearch_description_document. DeWitt, Clinton, Tesler, Joe, Fagan, Michael, Gregorio, Joe, Sauve, Aaron, Snell, James http://www.opensearch.org/Specifications/OpenSearch/1.1#OpenSearch_response_elements . A9.com, Inc. http://www.opensearch.org/Community/OpenSearch_software. Lynnes, Chris, NASA/GSFC Beaumont, Bruce,University of Alabama, Duerr, Ruth, National Snow and Ice Data Center, Hua, Hook, NASA/JPL, ESIP Federation. http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20100003371_2010003020.pdf

NASA GCMD DIF (Directory Interchange Format (DIF) Writers Guide, 2010. Global Change Master Directory, National Aeronautics and Space Administration. http://gcmd.nasa.gov/User/difguide/difman.html Federal Geographic Data Committee, September 2, 2010. http://www.fgdc.gov/metadata/geospatial-metadata-standards. 127 References (cont.) Federal Geographic Data Committee, September 2, 2010. http://www.fgdc.gov/metadata/csdgm/introduction.html. Unidata, University Corporation for Atmospheric Research, National Science Foundation. http://www.unidata.ucar.edu/software/netcdf/

http://cf-pcmdi.llnl.gov/ , UCRL-WEB-223427 International Standards for Business, Government and Society. http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?csnumber=26020 Open Geospatial Consortium, Inc. December 27, 1010. http://www.opengeospatial.org/projects/initiatives/gswie deLaBeaujardiere, Jeff, Building the IOOS Data Management Subsystem, Marine Technical Society Journal, IOOS Special Journal (Submitted 2010).

deLaBeaujardiere, Jeff, Building the IOOS Data Management Subsystem, Marine Technical Society Journal, IOOS Special Journal (Submitted 2010). 128 References (cont.) U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Weather Service, National Data Buoy Center, August 19, 2010. http://sdf.ndbc.noaa.gov/sos/server.php?request=GetObservation&service=SOS&offering=urn:ioos:station:wmo:46403&observedproperty=s ea_floor_depth_below_sea_surface&responseformat=text/csv&eventtime=2008-07-17T00:00Z

Center for Operational Oceanographic Products & Services (CO-OPS), National Oceanic and Atmospheric Administration. http://opendap.co-ops.nos.noaa.gov/ioos-dif-sos/get/currents/currentsprofile.jsp. http://www.ioos.gov/dif/ Bacharach, Sam, Open Geospatial Consortium, Inc. New Implementations of OGC Sensor Web Enablement Standards, Sensors Magazine, December 2007. National Aeronautics and Space Administration, Goddard Space Flight Center, EO-1 Mission Office. http://eo1.gsfc.nasa.gov/new/sensorWebExp/index.html.

National Aeronautics and Space Administration, Goddard Space Flight Center, EO-1 Mission Office. http://eo1.gsfc.nasa.gov/new/sensorWebExp/Components.html. Lynnes, Christopher; Keiser, Ken; Duerr, Ruth; Haran, Terry; Ballagh, Lisa; Raup, Bruce H.; Wilson, Bruce. Practical Data Interoperability for Earth Scientist. Version 1.0 Earth Science Data Systems Working Group, Tech Infusion. http://www.esdswg.org/techinfusion/downloads/pdies. 129 Thank you! 130

Recently Viewed Presentations

  • Physics 1251 The Science and Technology of Musical Sound

    Physics 1251 The Science and Technology of Musical Sound

    Physics 1251 The Science and Technology of Musical Sound Unit 2 Session 14 MWF Human Perception: Loudness Physics 1251 Unit 2 Session 14 Human Perception: Loudness Foolscap Quiz: A sound has an intensity of 0.005 W/m2.
  • SPACE EXPLORATION MERIT BADGE - troop158va.org

    SPACE EXPLORATION MERIT BADGE - troop158va.org

    SPACE EXPLORATION MERIT BADGE. Your Merit Badge Counselor (Me) : Gregg Dorn. ... The law of action-reaction. b. How rocket engines work. c. How satellites stay in orbit. d. How satellite pictures of Earth and pictures of other planets are...
  • Informative Speaking

    Informative Speaking

    INFORMATIVE SPEAKING Informative Speaking What is information? "That which reduces uncertainty." (DeVito, 2002) "Information speaking is discourse that imparts new information, secures understanding, or reinforces accumulated information." (Berko, et al.,2004) Informative Speaking What, then, is the theoretical basis for the...
  • Reemployment and Eligibility Assessments (REA) Innovations in ...

    Reemployment and Eligibility Assessments (REA) Innovations in ...

    The REA-U staff's flexibility and performance was phenomenal and the pilot model was well received and working for our customers. During Lab time, participants are accommodated in the workshop room engaged in job search activities, completion of paperwork and provided...
  • MATLAB Tutorial II

    MATLAB Tutorial II

    MATLAB Tutorial II. Power Electronics and Simulink. Simulink Basics. ... One can edit the model via operations like: copy, past, undo, align, distribute, resize. Please try to keep your models as organized and intuitive as possible. Others (and even you)...
  • Demonstration Scenario (Draft)

    Demonstration Scenario (Draft)

    ORBEET IML Demo Presentation. The role of this Demo session is to enable end users to get familiar with the expert mode of the tool towards a quick monitoring of the WSN status. The steps of the demo session: Connection...
  • Social stratification - slpbygeca.files.wordpress.com

    Social stratification - slpbygeca.files.wordpress.com

    Open system - also known as class system. - it has few impediments to social mobility. Close system-also known as caste system. - status is ascribed, and determined by . people at birth and people are locked into . their...
  • Approach to pneumonia By Clinical Syndrome

    Approach to pneumonia By Clinical Syndrome

    Cagliari 4 ,5 Ottobre 2008 Screenings sensoriali * Red Reflex 300 (sino al 6° m. a tutte le visite mensili) * Boel test 72 * Stereotest di Lang 125 * Lettura dell'ottotipo E 46 5° Congresso Nazionale C.I.PE - La...