Supporting HDF5 in GrADS

Jennifer M. Adams and Brian E. Doty IGES/COLA

What is GrADS?

GrADS is an interactive desktop tool used for easy access, analysis, and visualization of earth science data. Two data models: gridded and in situ
Handles many data formats: binary, GRIB, BUFR, netCDF, and HDF4 OPeNDAP-enabled client and server (GDS)

Outline
GrADS Development The Ensemble Dimension GrADS and HDF

GrADS Development
Fifth ensemble dimension New interfaces for GRIB2 and HDF5 Internal data handling in double precision Mask for missing data Graphics improvements New output options for GIS applications Third data model: Quasi-regular grids

Ensemble Handling

A true 5th dimension for ensemble members set X, Y, Z, T, or E A virtual dimension for forecast time offset display sst(ft=2) display sst(ftime=24hr)

GrADS Metadata Requirements for Ensemble Members

Unique name / number Initial time Length One time axis must span all members All members must share common grid

The GrADS-GDS Coupled System

The GDS serves any GrADS data set

GrADS is a client for all GDS data sets

The NetCDFification of the ensemble metadata must be meaningful to GrADS

Emerging metadata standard for ensembles

Ensembles: Same Length, Same Initial Time

Ensemble Member Time Axis ---->

Ensembles: Same Length, Different Initial Times

Ensemble Member Time Axis ---->

Ensembles: Same Length, Different Initial Times

Ensemble Member Time Axis ---->

Ensembles: Same Length, Different Initial Times

Ensemble Member Time Axis ---->

Ensembles: Same Length, Different Initial Times

Ensemble Member Time Axis ---->

NCEP CFS: Different Lengths, Different Initial Times

Ensemble Member
Time Axis ---->

 new = variable (ft=2)

Ensemble Member Time Axis ---->

 new = ftloop (variable, ftime=12hr, ftime=48hr)

Ensemble Member
Time Axis ---->

GrADS and HDF: A Brief History

GrADS reads HDF4 and NC files Metadata must be COARDS-compliant Uses NC v2 API GrADS writes HDF4 and NC files Uses LATS interface and NC v2 API GrADS becomes a DODS client NC v3 API is added

New interface for non-compliant HDF4 and NC files External metadata is user-provided Uses SD and NC APIs

GrADS 1.9 Executables

NetCDF HDF4
Read/Write -

DODS
Read

Libraries
Core libs
Core libs + libdf + libmfhdf Core libs + libnetcdf Core libs + libnc-dap

gradsc gradshdf gradsnc

Read Read/Write Read/Write

gradsdods

Core libs: X11, readline, zlib, png, gd Extra dependent libs: udunits, jpeg, szip, xml2, curl, dap, gadap

GrADS and HDF: Current Events

GrADS (now GPL) must drop (copyrighted) LATS

New interfaces to be written for NC and HDF output HDF4 handling isolated using SD API exclusively NC handling reworked using v3 API, ready for v4

GrADS 2.0 Executables

NetCDF HDF4
Read
(Write TBA)

OPeNDAP
Read

Libraries
Core libs + libdf + libmfhdf + libnetcdf Core libs + libdf + libmfhdf + libnc-dap

grads gradsdap

Read
(Write TBA)

Read
(Write TBA)

Read
(Write TBA)

Core libs: X11, readline, zlib, png, gd, jpeg, jasper, grib2c Extra dependent libs: udunits, szip, xml2, curl, dap, gadap

GrADS and HDF5: The Future

First Option: Link with NetCDF-4 library Very easy Only supports HDF5 created with NetCDF-4 Destined to repeat HDF4 history?

Second Option: Link with HDF5 library Requires a new HDF5 interface More general support of HDF5 Keeps NetCDF interface independent Necessary for quasi-regular swath data?