Towards Energy Efficient Numerical Weather

Prediction
Scalable Algorithms and Approaches

V. Venkatesh Shenoi
Center for Development of Advanced Computing (C-DAC), Pune

ACK:
S. Janakiraman (C-DAC)
Sandeep K. Joshi (C-DAC)
SCEC 2017, Jaipur – p.1/7
 Numerical Weather Prediction
A quantitative forecast of weather (or climate) based on model(temperature, wind, rain etc.
over a prescribed domain
• A system of coupled PDEs and other equations that describe the dynamic and
thermodyanamic processes in the Earth’s atmosphere
– Conservation laws (momentum, energy, mass)
– Equation of state (relating density, pressure, temperature)
– Fluid motion (relative to the Earth’s rotation)
• Approach: Discretize over the domain, converting the spatial and temporal derivatives
into a set of algebraic eqautions
– Suitable discretization, time integration schemes
– Constraint on the relative sizes of the space grid interval, time step and wind speed
(Courant-Fedrichs-Lewy stability criteria)
– Parameterization
– Boundary/Initial conditions
• Fluid velocity V ≡ (u, v, w), pressure, density, temperature, humidity
. Climate modeling Scientists and Engineers, J. B. Drake (2014)
SCEC 2017, Jaipur – p.2/7
 Shallow Water Equation (Model)
SWM: Two dimensional atmosphere-like fluid model, captures the essentials math. & comput.
properties of the meteorogical primitive equations (atmospheric flow). SWE on a sphere ⇔
conservation of momentum & mass.
– V = ui + vj, velocity on the sphere
dV
= −f k × V − ∇Φ – i, j, k: unit vectors on sphere
dt
– f : Coriolis force
dΦ
= −Φ∇ · V – Φ: geopotential
dt
– a: radius of sphere
d ∂
dt = ∂t + V · ∇ – λ: longitude
i ∂ j ∂
∇ = a(1−µ2 ) ∂λ + a ∂µ
– φ: latitude, µ = sin φ
• Spectral Transform method: equations are solved in terms of the vorticity η and the
horizontal divergence δ
• η = f + k · (∇ × V), δ = ∇ · V
• Avoid singularity in velocity at poles, transform (U, V ) = V cos φ
• Equations for scalar fields η, δ, Φ along with tranformation (U, V ) ⇒ (ψ, χ) relating
η ⇔ ψ & δ ⇔ χ (differential equations)
. NCAR technical note, J. J. Hack and R. Jacob (1992)
SCEC 2017, Jaipur – p.3/7
 Spectral Transform Method
The spectral representation of the scalar field variable ξ is represented by a truncated series
expansion in terms of spherical harmonic functions [Pnm (µ)eimλ ] as below,
M N (m) √
X X • i = −1
ξ(λ, µ) = ξnm Pnm (µ)eimλ
• λ: longitude
m=−M n=|m|
• φ: latitude
Z1 Z2π
 

ξnm =  1 ξ(λ, µ)e−imλ dλ Pnm (µ)dµ

• µ = sin φ
2π • m: wave number or Fourier
−1 0
mode
Z1
m • Pnm (µ): associate Legendre
≡ ξ (µ)Pnm (µ)dµ
function
−1
– M : the highest Fourier wavenumber included in the east-west representation
– N (m): the highest degree of the associated Legendre functions (north-south rep.)
– Triangular truncation: N (m) = M
– Since physical quantities are real, Pn−m = [Pnm ]∗ , this reduces computation complexity
and storage requirements (only coefficients of non-negative modes need to be stored)
. SIAM Rev. Sc. Comp. I. T. Foster and P. H. Worley (1997)
SCEC 2017, Jaipur – p.4/7
 Spectral Transform method (contd.)
– Spectral Transform = Fourier Transform followed by Legendre Transform
– A spherical harmonics transform is a Fourier transformation in longitude and a Legendre
transformation in latitude, thus keeping a latitude-longitude structure in gridpoint space.
– Computations are performed in both physical space and spectral space
– Physical quantities & Nonlinear terms computed in physical space
– Time stepping is performed in spectral space
– At each time step, the data is transformed between two spaces
– Fourier transform (FFT): complexity O[N 2 log N ]
– Conventional Legendre transform: complexity O[N 3 ]
– Global models using spectral transform are used by NCAR and ECMWF for forecasting
– For increasing horizontal resolution, the Legendre tranform becomes expensive part of
the computation
– With Fast Legendre transform (as well as due to truncation of series): complexity
O[N 2 log3 N ]
– ECMWF forecasting efforts have benefitted immensely form the FLT
. JMSJ, D. J. Williamson (2007)
. MWR, N. L. Wedi et.al (2013)
SCEC 2017, Jaipur – p.5/7
 Petascale/Exascale approaches to NWP

• 1.63 PFlops global shallow water model on Tianhe-2 based on stencil computation over
cubed sphere mesh using Xeon Phi accelerators along with Xeon within the power
envelope of 17.8 MW [W. Xue et.al, IPDPS(2014)]
• Shallow water equation solver on heterogeneous architecture (GPU, Xeon Phi, FPGA)
based on stencil over cubed sphere mesh [H. Fu et.al, PLOS One (2017)]
• ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at
Exascale http://www.hpc-escape.eu/home
- Focus on weather and climate dwarfs
- New algorithms and programming models
- Use of GPUs, Xeeon Phi and photonics technology for computation

SCEC 2017, Jaipur – p.6/7

.

Thank You
for your attention

SCEC 2017, Jaipur – p.7/7