Technologies for

Computational Science
Boyana Norris
Argonne National Laboratory
http://www.mcs.anl.gov/~norris
Outline
 Automatic differentiation
 Applications
in optimization
 How AD works

 Components for scientific computing

 Performance evaluation and modeling
 Bringing it all together

March 15, 2005 2

 What is automatic differentiation?

Automatic Differentiation (AD): a technology for

automatically augmenting computer programs,
including arbitrarily complex simulations, with
statements for the computation of derivatives,
also known as sensitivities.

The Computational Differentiation Project

at Argonne National Laboratory
March 15, 2005 3
What is it good for?
 The need to accurately and efficiently compute
derivatives of complicated simulation codes
arises regularly in
 Optimization (finding a minimum)
 Solving nonlinear differential equations
 Sensitivity and uncertainty analysis
 Inverse Problems, including:
 Data assimilation
 Parameter identification
 AD tools automate the generation of derivative
code without precluding the exploitation of high-
level knowledge.
March 15, 2005 4
 Sensitivity Analysis
MM5 (a mesoscale weather model, NCAR and Penn State)

Impact of perturbations of initial temperature on temperature in the system; low-

amplitude supersonic waves clearly visible with AD (left), but not visible with divided
difference approximations of derivatives (right).
March 15, 2005 5
 Parameter Tuning
Sea Ice Model (Todd Arbetter, University of Colorado)

Ice thickness for the standard (left) and tuned (right) parameter
values, with actual observations at two locations indicated.
March 15, 2005 6
 Optimization Problems
 Often we look for extreme, or optimum, values that a
function has on a given domain. More formally:

Given a function f :  n   and a set S   n ,

we are looking for x  S such that f attains a minimum
on S at x.
 Unconstrained minimization problems are ones in which
S  .
 Note: Since a maximum of f is a minimum of -f, we need
only to look for the minimum.

March 15, 2005 7

 Newton’s Method

• Method for finding x such that f(x) = 0

-1
xk +1 xk [  f ( x k )] f ( xk ), k  0,1,...
 -

• For optimization, we want f(x*) = 0, so iterate:

-1
xk +1 xk [ f ( x k )] f ( xk ), k  0,1,...
 -  2

March 15, 2005 8

 Example: Minimum Surface
Objective:
Find a surface

Solution
with the minimal
area that satisfies
Dirichlet boundary
conditions and is
constrained to lie
above a solid
plate.

Error
March 15, 2005 9
Example: Minimum Surface (Cont.)

Solution
Error
March 15, 2005 10
We can compute derivatives via:
 Analytic code
 By hand
 Automatic differentiation
 Numerical approximation: finite differencing (FD).
For finite differences, recall:
df f ( x + h) - f ( x )
 lim
dx h 0 h
d f f ( x + h) + 2 f ( x ) - f ( x - h)
2

 2 2
dx h
March 15, 2005 11
Why use AD?

 Compared with other methods (numerical

differentiation via finite differences, hand
coding, etc.), AD offers a number of
advantages:
 Accuracy
 Performance
 Reduced effort
 Algorithm-awareness

March 15, 2005 12

More accurate derivatives = faster convergence

Application: modeling transonic flow over an ONERA M6 airplane wing.

March 15, 2005 13
Who uses it?
AD has been successfully employed in applications in:
 Atmospheric chemistry
 Breast cancer modeling
 Computational fluid dynamics
 Mesoscale climate modeling
 Network Enabled Optimization System
 Semiconductor device modeling
 And also: groundwater remediation, multidisciplinary
design optimization, reactor engineering, super-
conductor simulation, multibody simulations, molecular
dynamics simulations, power system analysis, water
reservoir simulation, and storm modeling.
March 15, 2005 14
 How AD Works
 Every programming language provides a limited number of
elementary mathematical functions, e.g., +, -, *, /, sin,
cos,…
 Thus, every function computed by a program may be
viewed as the composition of these so-called intrinsic
functions
 Derivatives for the intrinsic functions are known and can
be combined using the chain rule of differential calculus

f ( g ( x))  g ( x)  f ( g ( x))
( f1 ( f 2 (( f N ( x)))))  f N  f N -1 ( f N ( x))  f N - 2 ( f N -1 ( f N ( x)))
  f1( f 2 (( f N ( x))))
March 15, 2005 15
A Simple Example (Fortran)
Original program Differentiated program
x = 3.14159265/4.0
x = 3.14159265/4.0
dxdx = 1.0 ! Initialize “seed matrix”
a = sin(x) a = sin(x)
dadx = cos(x)*dxdx ! TL/CR
b = cos(x)
b = cos(x)
t = a/b dbdx = -sin(x)*dxdx ! TL/CR
t = a/b
Key
dtda = 1.0/b ! TL
t
dtdx: dtdb = -a/(b*b) ! TL
x
CR: Chain rule dtdx = dtda*dadx + dtdb*dbdx ! CR
TL: Table lookup
March 15, 2005 16
Modes of AD
 Forward mode
 Mode used in simple example
 Propagates derivative vectors, often denoted u or g_u
 Derivative vector u contains derivatives of u with respect to
independent variables
 Time and storage proportional to vector length (# indeps)
 Reverse (or adjoint) mode
 Propagates adjoints, denoted ū or u_bar
 Adjoint ū contains derivatives of dependent variables with
respect to u
 Propagation starts with dependent variables—must reverse
flow of computation
 Time proportional to adjoint vector length (# dependents)
 Storage proportional to number of operations
 Because of this limitation, often applied to subprograms

March 15, 2005 17

 Another Simple Example (C code)
Original code: y = x1*x2*x3*x4;
typedef struct { DERIV_val(y): value of program
double value; variable y
double grad[ad_GRAD_MAX]; DERIV_grad(y):derivative object
} DERIV_TYPE; associated with y

ad_loc_0 = DERIV_val(x1) * DERIV_val(x2);

ad_loc_1 = ad_loc_0 * DERIV_val(x3); dy/dx4
ad_loc_2 = ad_loc_1 * DERIV_val(x4); y reverse
ad_adj_0 = ad_loc_0 * DERIV_val(x4); dy/dx3 (or adjoint)
ad_adj_1 = DERIV_val(x3) * DERIV_val(x4); mode of AD
ad_adj_2 = DERIV_val(x1) * ad_adj_1; dy/dx2
ad_adj_3 = DERIV_val(x2) * ad_adj_1; dy/dx1

ad_axpy_4(DERIV_grad(y), ad_adj_3, forward

DERIV_grad(x1), ad_adj_2, DERIV_grad(x2), mode of AD
ad_adj_0, DERIV_grad(x3), ad_loc_1, DERIV_grad(x4));

DERIV_val(y) = ad_loc_2; original value

March 15, 2005 18
 The AD Process
Application AD Code with
Code Tool Derivatives

Control
AD
Files Compile
Support
& Link
Libraries

User’s
Derivative Derivative
Driver Program

March 15, 2005 19

Ways of Implementing AD
 Operator Overloading
 Use language features to generate trace (“tape”) of
computation -> implicit computational graph
 Easy to implement; hard to optimize
 Examples: ADOL-C

 Source Transformation (ST)

 Relies on compiler technology
 Hard to implement; more powerful
 Examples: ADIFOR, ADIC, ODYSSEE, TAMC

March 15, 2005 20

Example AD Tool Architecture (ST)
 AD engine isolated front-
and backends via XAIF
(XML AD Interface
Format)
 XML representation of the
computational graph
 Unifies “relevant” Fortran
and C constructs
 Implements abstractions,
e.g. “derivative object”
 Shared “plug-in”
differentiation modules

March 15, 2005 21

 XAIF
Representation
Reverse Mode

March 15, 2005 22

 XAIF - Abstraction of the Program at
“AD-Level”: Expression Example
Only the core
structure of the
program is reflected
in XAIF:
=
•Control flow
•Variable information
var_1 + for active variables
•Basic blocks
– Expression
const * DAGs

var_2 var_3

March 15, 2005 23

Estimates of Incremental
Computational Costs
Flops Memory
F: Rn  Rm C W
J [FM] nC nW
J [RM] mC mW + C
g (m=1) [FM] nC nW
g [RM] C W+C
Jv [FM] C W
JTv [FM] nC nW
JTv [RM] C W+C
H (m=1) [FM] n2C n2W
Hv [FM] nC nW
Hv [FM/RM] C W+C
vTHv [FM] C W
March 15, 2005 24
Techniques for Improving Performance of
AD Code
 Exploit sparsity (SparsLinC and/or coloring)
 Exploit parallelism
 data: stripmine derivative computation
 task: multithread independent loops
 time: break computation into phases; pipeline derivative
computations
 Exploit interface contractions
 For computations of the form

f ( g ( x)) :    , g ( x) :    , n~  n
n~
n m ~
m

 Compute dg/dx, df/dg, multiply to form df/dx CD

 Exploit mathematics (e.g., differentiating through
linear/nonlinear equation solvers)
March 15, 2005 26
 ANL Tools for AD
 ADIFOR was developed in collaboration with Rice University
 full support for Fortran 77
 support for parallelism via MPI and PVM
 support for sparse Jacobians
 ADIC is the first & only compiler-based AD tool for ANSI C
 support for the complete ANSI standard
 will soon support a large subset of C++
 www.mcs.anl.gov/adic, www.mcs.anl.gov/adicserver
 XAIF specification and differentiation modules (OpenAD
project)
 http://www-unix.mcs.anl.gov/~utke/OpenAD

March 15, 2005 27

AD in Numerical Toolkits
 NEOS Network-Enabled Optimization Server
 http://neos.mcs.anl.gov
 Efficient
computation of gradients for large problems,
where the objective function has the form
f ( x)   f i ( x)
 PETSc (Portable Extensible Toolkit for Scientific
Computation) solvers (work in progress)
 User only needs to provide the sequential “subdomain
update” function in F77 or ANSI-C.
 Differentiated version of toolkit enables
optimization/sensitivity analysis of models based on
PETSc
 www.mcs.anl.gov/petsc 28
March 15, 2005
 Optimization Solution (PETSc & TAO)
Main Routine
Unconstrained Minimization
Newton-based Methods Limited Memory Conjugate Gradient Methods
Variable Metric Others
Fletcher- Polak- Polak-
Line Search Trust Region (LMVM) Method
Reeves Ribiére Ribiére-Plus
Nonlinear Solvers (SNES)
Bound Constrained Optimization
Newton Trust Region GPCG Interior Point LMVM KT Others
Solve
Linear Solvers (SLES) min F(u)
Nonlinear Least Squares
Levenberg Gauss- Levenberg Marquardt LMVM with
LMVM Others
Marquardt Newton with Bound Constraints Numerical
Bound Constraints
PC KSP Library
TAO interfaces to external libraries for parallel Complementarity
vectors, matrices, and linear solvers:
Semi-smooth Methods Others
• PETSc
Application (initial interface)
Minimum Function Gradient Hessian Post-
• Trilinos (SNL - capability via ESI – thanks to M.
Initialization Evaluation Evaluation Evaluation Processing
Heroux and A. Williams)
• Global Arrays (PNNL, J. Nieplocha et al.)
• Etc.User code PETSc code AD-generated code
March 15, 2005 29
 Using AD with the Toolkit for Advanced
Optimization (TAO)
Global-to-local
scatter of ghost values
Local Function Local Min.Function
computation computation
Parallel function
assembly Script file

Global-to-local
scatter of ghost values ADIFOR or ADIC
Coded manually;
Seed matrix can be automated
initialization

Local Hessian Local Hessian

computation computation

Parallel Hessian
assembly User code PETSc code AD-generated code
March 15, 2005 30
Outline
 Automatic differentiation
 Components for scientific computing
 Introduction
 Example applications
 Performance evaluation and modeling
 Summary

CCA
Common Component Architecture

March 15, 2005 31

Software development approaches

Architectures

Components

Object-oriented libraries: collections of classes

Libraries: collections of subroutines

Unstructured code (everything in main)

March 15, 2005 32

Components
 Working definition: a component is a piece of software that
can be composed with other components within a
framework; composition can be either static (at link time)
or dynamic (at run time)
 “plug-and-play” model for building applications
 For more info: C. Szyperski, Component Software: Beyond Object-
Oriented Programming, ACM Press, New York, 1998
 Components enable
 Software and tool interoperability
 Automation of performance instrumentation/monitoring
 Application adaptivity (automated or user-guided)
 Pictorial intro

March 15, 2005 33

Object-oriented vs
component-oriented development
 Component-oriented development can be viewed as augmenting OOD with
certain policies, e.g., require that certain abstract interfaces be implemented
 Components, once compiled, require a special execution environment
 OO techniques are useful for building individual components by relatively
small teams; component technologies facilitate sharing of code developed
by different groups by addressing issues in
 Language interoperability
 Via interface definition language (IDL)
 Well-defined abstract interfaces
 Enable “plug-and-play”
 Dynamic composability
 Components can discover information about their environment (e.g., interface
discovery) from framework and connected components
 Can convert from an object orientation to a component orientation
 Automatic tools can help with conversion (ongoing work by C. Rasmussen and
M. Sottile, LANL)

March 15, 2005 34

 Motivating scientific applications
Physics

Optimization Adaptive Solution

Meshes
Diagnostics
Derivative Computation
Discretization Steering
Molecular
structures Visualization
Algebraic Solvers
Astrophysics

Data Redistribution

Parallel I/O
Aerodynamics
Fusion

March 15, 2005 35

Motivation: For Application
Developers and Users
 You have difficulty managing multiple third-party libraries
in your code
 You (want to) use more than two languages in your
application
 Your code is long-lived and different pieces evolve at
different rates
 You want to be able to swap competing implementations
of the same idea and test without modifying any of your
code
 You want to compose your application with some
other(s) that weren’t originally designed to be combined

March 15, 2005 36

The model for scientific component
programming

Science
Industry

?CA
C
March 15, 2005 37
 CCA Delivers
Performance
Local
 No CCA overhead within components
 Small overhead between components
 Small overhead for language interoperability Maximum 0.2% overhead for CCA vs
 Be aware of costs & design with them in mind native C++ code for parallel molecular
dynamics up to 170 CPUs
 Small costs, easily amortized

Parallel
 No CCA overhead on parallel computing
 Use your favorite parallel programming model
 Supports SPMD and MPMD approaches
Distributed (remote)
 No CCA overhead – performance depends
on networks, protocols
 CCA frameworks support OGSA/Grid
Services/Web Services and other Aggregate time for linear solver
component in unconstrained minimization
approaches problem w/ PETSc
March 15, 2005 38
Overhead from Component
Invocation
 Invoke a component with
different arguments Function arg
 Array type f77 Component
 Complex
 Double Complex
 Compare with f77 method
invocation Array 80 ns 224ns
 Environment
 500 MHz Pentium III
 Linux 2.4.18
 GCC 2.95.4-15
Complex 75ns 209ns
 Components took 3X longer
 Ensure granularity is
appropriate! Double
 Paper by Bernholdt, Elwasif, complex 86ns 241ns
Kohl and Epperly
March 15, 2005 39
Language interoperability: what is
so hard?
Native

f77 cfortran.h

SWIG
C f90
JNI

Siloon
C++ Python
Chasm

Java Platform
Dependent
March 15, 2005 40
SIDL/Babel makes all supported
languages peers
f77 This is not a
Lowest Common
Denominator
C f90 Solution!

C++ Python

Java
March 15, 2005 41
 CCA Concepts: Components and Ports
 Components provide or use OptimizerPort FunctionPort FunctionPort
one or more ports
GradientPort
 Components include some Objective Function
code which interacts with a HessianPort
CCA framework GradientPort
Optimization Algorithm
 Frameworks provide
services, such as Function Gradient
component instantiation and
port connection
HessianPort

Implementation details: Function Hessian

 CCA components…
• Inherit from gov.cca.Component
• Implement setServices method to register ports this component will provide and use
• Implement the ports they provide
• Use ports on other components
• Call getPort/releasePort methods of framework Services object
 Ports (interfaces) extend the gov.cca.Port interface

March 15, 2005 42

 Example:
Unconstrained Minimization Problem
 Given a rectangular 2-dimensional domain and
boundary values along the edges of the domain
 Find the surface with minimal area that satisfies the
boundary conditions, i.e., compute
min f(x), where f: R  R
 Solve using optimization
components based on
TAO (ANL)

March 15, 2005 43

Unconstrained Minimization Using a Structured
Mesh

Reused
TAO Solver
Driver/Physics

March 15, 2005 44

 Computational Chemistry:
Molecular Optimization
• Investigators: Yuri Alexeev (PNNL), Steve Benson (ANL),
Curtis Janssen (SNL), Joe Kenny (SNL), Manoj Krishnan
(PNNL), Lois McInnes (ANL), Jarek Nieplocha (PNNL),
Jason Sarich (ANL), Theresa Windus (PNNL)

• Goals: Demonstrate interoperability among software

packages, develop experience with large existing code
bases, seed interest in chemistry domain

 Problem Domain: Optimization of

molecular structures using quantum
chemical methods

March 15, 2005 45

 Molecular Optimization Overview
 Decouple geometry optimization from electronic structure
 Demonstrate interoperability of electronic structure components
 Build towards more challenging optimization problems, e.g.,
protein/ligand binding studies

Components in gray can be swapped in to create new applications

March 15, 2005 with different capabilities. 46
 Wiring Diagram for Molecular Optimization

 Electronic structures components: • Optimization components: TAO (ANL)

http://www.mcs.anl.gov/tao
• MPQC (SNL)
http://aros.ca.sandia.gov/~cljanss/mpqc • Linear algebra components:
• NWChem (PNNL) • Global Arrays (PNNL)
http://www.emsl.pnl.gov:2080/docs/global/ga.html
http://www.emsl.pnl.gov/pub/docs/nwchem
• PETSc (ANL)
http://www.mcs.anl.gov/petsc 47
March 15, 2005
Outline
 Automatic differentiation
 Components for scientific computing
 Performance evaluation and modeling
 Performance evaluation challenges
 Component-based approach
 Motivating example: adaptive linear system solution
 A component infrastructure for performance
monitoring and adaptation of applications
 Summary

March 15, 2005 48

Why Performance Model?
 Performance models enable understanding of
the factors that affect performance
 Inform the tuning process (of application and
machine)
 Identify bottlenecks
 Identify underperforming components
 Guide applications to the best machine
 Enable applications-driven architecture design
 Extrapolate the performance of future systems

March 15, 2005 49

Challenges in performance evaluation
+ Many tools for performance data gathering and analysis
 PAPI, TAU, SvPablo, Kojak, …
 Various interfaces, levels of automation, and approaches to
information presentation
 User’s point of view
- What do the different tools do? Which is most appropriate for a
given application?
- (How) can multiple tools be used in concert?
- I have tons of performance data, now what?
- What automatic tuning tools are available, what exactly do they
do?
- How hard is it to install/learn/use tool X?
- Is instrumented code portable? What’s the overhead of
instrumentation? How does code evolution affect the
performance analysis process?

March 15, 2005 50

Incomplete list of tools
 Source instrumentation: TAU/PDT, KOJAK (MPI/OpenMP),
SvPablo, Performance Assertions, …
 Binary instrumentation: HPCToolkit, Paradyn, DyninstAPI,
…
 Performance monitoring: MetaSim Tracer (memory), PAPI,
HPCToolkit, Sigma++ (memory), DPOMP (OpenMP),
mpiP, gprof, psrun, …
 Modeling/analysis/prediction: MetaSim Convolver
(memory), DIMEMAS(network), SvPablo (scalability),
Paradyn, Sigma++, …
 Source/binary optimization: Automated Empirical
Optimization of Software (ATLAS), OSKI, ROSE
 Runtime adaptation: ActiveHarmony, SALSA

March 15, 2005 51

Incomplete list of tools
 Source instrumentation: TAU/PDT, KOJAK (MPI/OpenMP),
SvPablo, Performance Assertions, …
 Binary instrumentation: HPCToolkit, Paradyn, DyninstAPI,
…
 Performance monitoring: MetaSim Tracer (memory), PAPI,
HPCToolkit, Sigma++ (memory), DPOMP (OpenMP),
mpiP, gprof, psrun, …
 Modeling/analysis/prediction: MetaSim Convolver
(memory), DIMEMAS(network), SvPablo (scalability),
Paradyn, Sigma++, …
 Source/binary optimization: Automated Empirical
Optimization of Software (ATLAS), OSKI, ROSE
 Runtime adaptation: ActiveHarmony, SALSA

March 15, 2005 52

Incomplete list of tools
 Source instrumentation: TAU/PDT, KOJAK (MPI/OpenMP),
SvPablo, Performance Assertions, …
 Binary instrumentation: HPCToolkit, Paradyn, DyninstAPI,
…
 Performance monitoring: MetaSim Tracer (memory), PAPI,
HPCToolkit, Sigma++ (memory), DPOMP (OpenMP),
mpiP, gprof, psrun, …
 Modeling/analysis/prediction: MetaSim Convolver
(memory), DIMEMAS(network), SvPablo (scalability),
Paradyn, Sigma++, …
 Source/binary optimization: Automated Empirical
Optimization of Software (ATLAS), OSKI, ROSE
 Runtime adaptation: ActiveHarmony, SALSA

March 15, 2005 53

Challenges (where is the complexity?)
 More effective use  integration
 Tool developer’s perspective
 Overhead of initially implementing one-to-one interoperabilty
 Ongoing management of dependencies on other tools
 Individual Scientist Perspective
 Learning curve for performance tools  less time to focus on own
research (modeling, physics, mathematics, optimization)
 Potentially significant time investment needed to find out
whether/how using someone else’s tool would improve
performance  tend to do own hand-coded optimizations (time-
consuming, non-reusable)
 Lack of tools that automate (at least partially) algorithm discovery,
assembly, configuration, and enable runtime adaptivity

March 15, 2005 54

What can be done
 How to manage complexity? Provide
 Performance tools that are truly interoperable
 Uniform easy access to tools
 Component implementations of software, esp.
supporting numerical codes, such as linear algebra
algorithms
 New algorithms (e.g., interactive/dynamic techniques,
algorithm composition)
 Implementation approach: components,
both for tools and the application software
March 15, 2005 55
Performance Evaluation Research
Center (http://perc.nersc.gov)

March 15, 2005 56

What is being done
 No “integrated” environment for performance
monitoring, analysis, and optimization (yet)
 Most past efforts
 One-to-one tool interoperability
 More recently
 OSPAT (initial meeting at SC’04), focus on common
data representation and interfaces
 Tool-independent performance databases: PerfDMF
 Eclipse parallel tools project (LANL)
…

March 15, 2005 57

Example: component infrastructure for
multimethod linear solvers
 Goal: provide a framework for
 Performance monitoring of numerical components
 Dynamic adaptativity, based on:
 Off-line analyses of past performance information
 Online analysis of current execution performance information
 Motivating application examples:
 Driven cavity flow [Coffey et al, 2003], nonlinear PDE solution
 FUN3D – incompressible and compressible Euler equations
 Prior work in multimethod linear solvers
 McInnes et al, ’03, Bhowmick et al,’03 and ’05, Norris at al. ’05.

March 15, 2005 59

Adaptive Linear System Solution
 Motivation:
 Approximately 80% of total solution time devoted to linear
system solution
 Multi-phase nonlinear solution method, requiring the solution of
linear systems with varying levels of ill-conditioning [Kelley and
Keyes, 1998]
 New approach aiming to reduce overall time to solution
 Combine more robust (but more costly) methods when needed
in some phases with faster (but less powerful) methods in other
phases
 Dynamically select a new preconditioner in each phase based on
CFL number

March 15, 2005 60

 Example: driven cavity flow

 Linear solver: GMRES(30), vary only fill level of ILU preconditioner

 Adaptive heuristic based on:
 Previous linear solution convergence rate, nonlinear solution convergence rate, rate
of increase of linear solution iterations
 96x96 mesh, Grashof = 105, lid velocity = 100
 Intel P4 Xeon, dual 2.2 GHz, 4GB RAM
March 15, 2005 61
Bringing it all together
 Integration of ongoing efforts in
 Performance tools: common interfaces and data
represenation (leverage OSPAT, PerfDMF, TAU
performance interfaces, and similar efforts)
 Numerical components: emerging common interfaces
(e.g., TOPS solver interfaces) increase choice of
solution method  automated composition and
adaptation strategies
 Code generation, e.g., AD

 Long term
 Is a more organized (but not too restrictive)
environment for scientific software lifecycle
development possible/desirable?
March 15, 2005 62
 Multimethod linear solver components
Physics Nonlinear Linear
Solver Solver

Mesh

Physics Nonlinear
Solver

Mesh Adaptive Linear

Heuristic Solver A

Linear
Performance Solver B
Checkpointing
Monitor
Linear
Solver C
March 15, 2005 63
AD as Component Factory
 Both NEOS and
PETSc rely on a well-
Function defined function
interface in order to
provide derivatives
via AD
 Extend this idea to
components
AD Tool
Jacobian

March 15, 2005 64

Summary
 Automation at all levels of the application development
process can simplify and speed up application
development and result in better software quality and
performance
 AD addresses the wide-spread need for accurate and efficient
derivative computations
 CCA defines a high-performance component model, enabling
large-scale software development
 A growing array of performance tools and methodologies aid in
understanding and fine-tuning application performance
 Current and future work: bringing these technologies
together in a coherent way, making large-scale scientific
application development as easy as possible

March 15, 2005 65

Acknowledgments
 Paul Hovland, Jean Utke, Lois Curfman McInnes
(ANL)
 Sanjukta Bhowmick (ANL/Columbia)
 Ivana Veljkovic, Padma Raghavan (Penn State)
 Sameer Shende, Al Malony (U. Oregon)
 CCA and PERC members
 Funding: DOE and NSF

March 15, 2005 66

For More Information
 Automatic differentiation
 Andreas Griewank. Evaluating Derivatives: Principles and Techniques
of Alogrithmic Differentiation, SIAM, 2000.
 www.autodiff.org : publications, tools, etc.
 www.mcs.anl.gov/adicserver : ADIC server
 neos.mcs.anl.gov : NEOS server
 Common component architecture
 www.cca-forum.org
 Performance tools
 perc.nersc.gov
 Student opportunities at MCS/ANL
 www-fp.mcs.anl.gov/division/information/educational_programs/studentopps.html
 Boyana Norris
 Email: norris@mcs.anl.gov, Web: www.mcs.anl.gov/~norris
March 15, 2005 67