Numerca anayss s the study of agorthms that use numerca

approxmaton (as opposed to genera symboc manpuatons) for the

probems of mathematca anayss (as dstngushed from dscrete
mathematcs).
One of the earest mathematca wrtngs s a Babyonan tabet from the Yae
Babyonan Coecton (YBC 7289), whch gves a sexagesma numerca
approxmaton of \sqrt{2}, the ength of the dagona n a unt square. Beng
abe to compute the sdes of a trange (and hence, beng abe to compute
square roots) s extremey mportant, for nstance, n astronomy, carpentry
and constructon.|2|
Numerca anayss contnues ths ong tradton of practca mathematca
cacuatons. Much ke the Babyonan approxmaton of \sqrt{2}, modern
numerca anayss does not seek exact answers, because exact answers are
often mpossbe to obtan n practce. Instead, much of numerca anayss s
concerned wth obtanng approxmate soutons whe mantanng reasonabe
bounds on errors.
Numerca anayss naturay nds appcatons n a eds of engneerng and
the physca scences, but n the 21st century aso the fe scences and even
the arts have adopted eements of scentc computatons. Ordnary
dherenta equatons appear n ceesta mechancs (panets, stars and
gaaxes); numerca near agebra s mportant for data anayss; stochastc
dherenta equatons and Markov chans are essenta n smuatng vng
ces for medcne and boogy.
Before the advent of modern computers numerca methods often depended
on hand nterpoaton n arge prnted tabes. Snce the md 20th century,
computers cacuate the requred functons nstead. These same nterpoaton
formuas nevertheess contnue to be used as part of the software agorthms
for sovng dherenta equatons.The overa goa of the ed of numerca
anayss s the desgn and anayss of technques to gve approxmate but
accurate soutons to hard probems, the varety of whch s suggested by the
foowng:
Advanced numerca methods are essenta n makng numerca weather
predcton feasbe.
Computng the tra|ectory of a spacecraft requres the accurate numerca
souton of a system of ordnary dherenta equatons.
Car companes can mprove the crash safety of ther vehces by usng
computer smuatons of car crashes. Such smuatons essentay consst of
sovng parta dherenta equatons numercay.
Hedge funds (prvate nvestment funds) use toos from a eds of numerca
anayss to attempt to cacuate the vaue of stocks and dervatves more
precsey than other market partcpants.
Arnes use sophstcated optmzaton agorthms to decde tcket prces,
arpane and crew assgnments and fue needs. Hstorcay, such agorthms
were deveoped wthn the overappng ed of operatons research.
Insurance companes use numerca programs for actuara anayss.
The rest of ths secton outnes severa mportant themes of numerca
anayss.
Hstory|edt|
The ed of numerca anayss predates the nventon of modern computers
by many centures. Lnear nterpoaton was aready n use more than 2000
years ago. Many great mathematcans of the past were preoccuped by
numerca anayss, as s obvous from the names of mportant agorthms ke
Newton's method, Lagrange nterpoaton poynoma, Gaussan emnaton,
or Euer's method.
To factate computatons by hand, arge books were produced wth formuas
and tabes of data such as nterpoaton ponts and functon coemcents.
Usng these tabes, often cacuated out to 16 decma paces or more for
some functons, one coud ook up vaues to pug nto the formuas gven and
acheve very good numerca estmates of some functons. The canonca
work n the ed s the NIST pubcaton edted by Abramowtz and Stegun, a
1000-pus page book of a very arge number of commony used formuas and
functons and ther vaues at many ponts. The functon vaues are no onger
very usefu when a computer s avaabe, but the arge stng of formuas can
st be very handy.
The mechanca cacuator was aso deveoped as a too for hand
computaton. These cacuators evoved nto eectronc computers n the
1940s, and t was then found that these computers were aso usefu for
admnstratve purposes. But the nventon of the computer aso nuenced
the ed of numerca anayss, snce now onger and more compcated
cacuatons coud be done.
Drect and teratve methods|edt|
Drect vs teratve methods
Consder the probem of sovng
3x3 + 4 = 28
for the unknown quantty x.
Drect method
3x3 + 4 = 28.
Subtract 4 3x3 = 24.
Dvde by 3 x3 = 8.
Take cube roots x = 2.
For the teratve method, appy the bsecton method to f(x) = 3x3 - 24. The
nta vaues are a = 0, b = 3, f(a) = -24, f(b) = 57.
Iteratve method
a b md f(md)
0 3 1.5 -13.875
1.5 3 2.25 10.17...
1.5 2.25 1.875 -4.22...
1.875 2.25 2.0625 2.32...
We concude from ths tabe that the souton s between 1.875 and 2.0625.
The agorthm mght return any number n that range wth an error ess than
0.2.
Dscretzaton and numerca ntegraton|edt|
Schumacher (Ferrar) n practce at USGP 2005.|pg
In a two hour race, we have measured the speed of the car at three nstants
and recorded them n the foowng tabe.
Tme 0:20 1:00 1:40
km/h 140 150 180
A dscretzaton woud be to say that the speed of the car was constant from
0:00 to 0:40, then from 0:40 to 1:20 and nay from 1:20 to 2:00. For
nstance, the tota dstance traveed n the rst 40 mnutes s approxmatey
(2/3h 140 km/h) = 93.3 km. Ths woud aow us to estmate the tota
dstance traveed as 93.3 km + 100 km + 120 km = 313.3 km, whch s an
exampe of numerca ntegraton (see beow) usng a Remann sum, because
dspacement s the ntegra of veocty.
I-condtoned probem: Take the functon f(x) = 1/(x - 1). Note that f(1.1) =
10 and f(1.001) = 1000: a change n x of ess than 0.1 turns nto a change n
f(x) of neary 1000. Evauatng f(x) near x = 1 s an -condtoned probem.
We-condtoned probem: By contrast, evauatng the same functon f(x) = 1/
(x - 1) near x = 10 s a we-condtoned probem. For nstance, f(10) = 1/9 =
0.111 and f(11) = 0.1: a modest change n x eads to a modest change n f(x).
Drect methods compute the souton to a probem n a nte number of steps.
These methods woud gve the precse answer f they were performed n
nnte precson arthmetc. Exampes ncude Gaussan emnaton, the OR
factorzaton method for sovng systems of near equatons, and the smpex
method of near programmng. In practce, nte precson s used and the
resut s an approxmaton of the true souton (assumng stabty).
In contrast to drect methods, teratve methods are not expected to
termnate n a nte number of steps. Startng from an nta guess, teratve
methods form successve approxmatons that converge to the exact souton
ony n the mt. A convergence test, often nvovng the resdua, s speced
n order to decde when a sumcenty accurate souton has (hopefuy) been
found. Even usng nnte precson arthmetc these methods woud not reach
the souton wthn a nte number of steps (n genera). Exampes ncude
Newton's method, the bsecton method, and |acob teraton. In
computatona matrx agebra, teratve methods are generay needed for
arge probems.
Iteratve methods are more common than drect methods n numerca
anayss. Some methods are drect n prncpe but are usuay used as though
they were not, e.g. GMRES and the con|ugate gradent method. For these
methods the number of steps needed to obtan the exact souton s so arge
that an approxmaton s accepted n the same manner as for an teratve
method.
Dscretzaton|edt|
Furthermore, contnuous probems must sometmes be repaced by a dscrete
probem whose souton s known to approxmate that of the contnuous
probem; ths process s caed dscretzaton. For exampe, the souton of a
dherenta equaton s a functon. Ths functon must be represented by a
nte amount of data, for nstance by ts vaue at a nte number of ponts at
ts doman, even though ths doman s a contnuum.
Generaton and propagaton of errors|edt|
The study of errors forms an mportant part of numerca anayss. There are
severa ways n whch error can be ntroduced n the souton of the probem.
Round-oh|edt|
Round-oh errors arse because t s mpossbe to represent a rea numbers
exacty on a machne wth nte memory (whch s what a practca dgta
computers are).
Truncaton and dscretzaton error|edt|
Truncaton errors are commtted when an teratve method s termnated or a
mathematca procedure s approxmated, and the approxmate souton
dhers from the exact souton. Smary, dscretzaton nduces a
dscretzaton error because the souton of the dscrete probem does not
concde wth the souton of the contnuous probem. For nstance, n the
teraton n the sdebar to compute the souton of ''3x3+4=28'', after 10 or
so teratons, we concude that the root s roughy 1.99 (for exampe). We
therefore have a truncaton error of 0.01.
Once an error s generated, t w generay propagate through the
cacuaton. For nstance, we have aready noted that the operaton + on a
cacuator (or a computer) s nexact. It foows that a cacuaton of the type
a+b+c+d+e s even more nexact.
What does t mean when we say that the truncaton error s created when we
approxmate a mathematca procedure? We know that to ntegrate a functon
exacty requres one to nd the sum of nnte trapezods. But numercay
one can nd the sum of ony nte trapezods, and hence the approxmaton
of the mathematca procedure. Smary, to dherentate a functon, the
dherenta eement approaches to zero but numercay we can ony choose a
nte vaue of the dherenta eement.
Numerca stabty and we-posed probems|edt|
Numerca stabty s an mportant noton n numerca anayss. An agorthm
s caed numercay stabe f an error, whatever ts cause, does not grow to
be much arger durng the cacuaton. Ths happens f the probem s we-
condtoned, meanng that the souton changes by ony a sma amount f the
probem data are changed by a sma amount. To the contrary, f a probem s
-condtoned, then any sma error n the data w grow to be a arge error.
Both the orgna probem and the agorthm used to sove that probem can
be we-condtoned and/or -condtoned, and any combnaton s possbe.
So an agorthm that soves a we-condtoned probem may be ether
numercay stabe or numercay unstabe. An art of numerca anayss s to
nd a stabe agorthm for sovng a we-posed mathematca probem. For
nstance, computng the square root of 2 (whch s roughy 1.41421) s a we-
posed probem. Many agorthms sove ths probem by startng wth an nta
approxmaton x1 to \sqrt{2}, for nstance x1=1.4, and then computng
mproved guesses x2, x3, etc.. One such method s the famous Babyonan
method, whch s gven by xk+1 = xk/2 + 1/xk. Another teraton, whch we
w ca Method X, s gven by xk + 1 = (xk2-2)2 + xk.|3| We have cacuated
a few teratons of each scheme n tabe form beow, wth nta guesses x1 =
1.4 and x1 = 1.42.
Babyonan Babyonan Method X Method X
x1 = 1.4 x1 = 1.42 x1 = 1.4 x1 = 1.42
x2 = 1.4142857... x2 = 1.41422535... x2 = 1.4016 x2 = 1.42026896
x3 = 1.414213564... x3 = 1.41421356242... x3 = 1.4028614... x3 =
1.42056...
... ...
x1000000 = 1.41421... x28 = 7280.2284...
Observe that the Babyonan method converges fast regardess of the nta
guess, whereas Method X converges extremey sowy wth nta guess 1.4
and dverges for nta guess 1.42. Hence, the Babyonan method s
numercay stabe, whe Method X s numercay unstabe.
Numerca stabty s ahected by the number of the sgncant dgts the
machne keeps on, f we use a machne that keeps on the rst four oatng-
pont dgts, a good exampe on oss of sgncance s gven by these two
equvaent functons

f(x)=x\eft(\sqrt{x+1}-\sqrt{x}\rght)
\text{ and } g(x)=\frac{x}{\sqrt{x+1}+\sqrt{x}}.
If we compare the resuts of
f(500)=500 \eft(\sqrt{501}-\sqrt{500} \rght)=500 \eft(22.3830-22.3607
\rght)=500(0.0223)=11.1500
and
\begn{agnat}{3}g(500)&=\frac{500}{\sqrt{501}+\sqrt{500}}\\
&=\frac{500}{22.3830+22.3607}\\
&=\frac{500}{44.7437}=11.1748
\end{agnat}
by ookng to the two resuts above, we reaze that oss of sgncance whch
s aso caed Subtractve Canceaton has a huge ehect on the resuts, even
though both functons are equvaent; to show that they are equvaent smpy
we need to start by f(x) and end wth g(x), and so
\begn{agnat}{4}
f(x)&=x \eft(\sqrt{x+1}-\sqrt{x} \rght)\\
& =x \eft(\sqrt{x+1}-\sqrt{x} \rght)\frac{\sqrt{x+1}+\sqrt{x}}
{\sqrt{x+1}+\sqrt{x}}\\
&=x\frac{(\sqrt{x+1})2-(\sqrt{x})2}{\sqrt{x+1}+\sqrt{x}}\\
& =x\frac{x+1-x}{\sqrt{x+1}+\sqrt{x}} \\
& =x\frac{1}{\sqrt{x+1}+\sqrt{x}} \\
&=\frac {x}{\sqrt{x+1}+\sqrt{x}}
\end{agnat}
The true vaue for the resut s 11.174755..., whch s exacty g(500) =
11.1748 after roundng the resut to 4 decma dgts.
Now magne that ots of terms ke these functons are used n the program;
the error w ncrease as one proceeds n the program, uness one uses the
sutabe formua of the two functons each tme one evauates ether f(x), or
g(x); the choce s dependent on the party of x.
The exampe s taken from Mathew; Numerca methods usng matab, 3rd ed.
Areas of study|edt|
The ed of numerca anayss ncudes many sub-dscpnes. Some of the
ma|or ones are:
Computng vaues of functons|edt|
Interpoaton: We have observed the temperature to vary from 20 degrees
Cesus at 1:00 to 14 degrees at 3:00. A near nterpoaton of ths data woud
concude that t was 17 degrees at 2:00 and 18.5 degrees at 1:30pm.
Extrapoaton: If the gross domestc product of a country has been growng an
average of 5% per year and was 100 bon doars ast year, we mght
extrapoate that t w be 105 bon doars ths year.
A ne through 20 ponts
Regresson: In near regresson, gven n ponts, we compute a ne that
passes as cose as possbe to those n ponts.
How much for a gass of emonade?
Optmzaton: Say you se emonade at a emonade stand, and notce that at
$1, you can se 197 gasses of emonade per day, and that for each ncrease
of $0.01, you w se one gass of emonade ess per day. If you coud charge
$1.485, you woud maxmze your prot, but due to the constrant of havng
to charge a whoe cent amount, chargng $1.49 per gass w yed the
maxmum ncome of $220.52 per day.
Wnd drecton n bue, true tra|ectory n back, Euer method n red.
Dherenta equaton: If you set up 100 fans to bow ar from one end of the
room to the other and then you drop a feather nto the wnd, what happens?
The feather w foow the ar currents, whch may be very compex. One
approxmaton s to measure the speed at whch the ar s bowng near the
feather every second, and advance the smuated feather as f t were movng
n a straght ne at that same speed for one second, before measurng the
wnd speed agan. Ths s caed the Euer method for sovng an ordnary
dherenta equaton.
One of the smpest probems s the evauaton of a functon at a gven pont.
The most straghtforward approach, of |ust puggng n the number n the
formua s sometmes not very emcent. For poynomas, a better approach s
usng the Horner scheme, snce t reduces the necessary number of
mutpcatons and addtons. Generay, t s mportant to estmate and
contro round-oh errors arsng from the use of oatng pont arthmetc.
Interpoaton, extrapoaton, and regresson|edt|
Interpoaton soves the foowng probem: gven the vaue of some unknown
functon at a number of ponts, what vaue does that functon have at some
other pont between the gven ponts?
Extrapoaton s very smar to nterpoaton, except that now we want to nd
the vaue of the unknown functon at a pont whch s outsde the gven
ponts.
Regresson s aso smar, but t takes nto account that the data s mprecse.
Gven some ponts, and a measurement of the vaue of some functon at
these ponts (wth an error), we want to determne the unknown functon. The
east squares-method s one popuar way to acheve ths.
Sovng equatons and systems of equatons|edt|
Another fundamenta probem s computng the souton of some gven
equaton. Two cases are commony dstngushed, dependng on whether the
equaton s near or not. For nstance, the equaton 2x+5=3 s near whe
2x2+5=3 s not.
Much ehort has been put n the deveopment of methods for sovng systems
of near equatons. Standard drect methods, .e., methods that use some
matrx decomposton are Gaussan emnaton, LU decomposton, Choesky
decomposton for symmetrc (or hermtan) and postve-dente matrx, and
OR decomposton for non-square matrces. Iteratve methods such as the
|acob method, Gauss-Sede method, successve over-reaxaton and
con|ugate gradent method are usuay preferred for arge systems. Genera
teratve methods can be deveoped usng a matrx spttng.
Root-ndng agorthms are used to sove nonnear equatons (they are so
named snce a root of a functon s an argument for whch the functon yeds
zero). If the functon s dherentabe and the dervatve s known, then
Newton's method s a popuar choce. Lnearzaton s another technque for
sovng nonnear equatons.
Sovng egenvaue or snguar vaue probems|edt|
Severa mportant probems can be phrased n terms of egenvaue
decompostons or snguar vaue decompostons. For nstance, the spectra
mage compresson agorthm|4| s based on the snguar vaue
decomposton. The correspondng too n statstcs s caed prncpa
component anayss.
Optmzaton|edt|
Man artce: Mathematca optmzaton
Optmzaton probems ask for the pont at whch a gven functon s
maxmzed (or mnmzed). Often, the pont aso has to satsfy some
constrants.
The ed of optmzaton s further spt n severa subeds, dependng on the
form of the ob|ectve functon and the constrant. For nstance, near
programmng deas wth the case that both the ob|ectve functon and the
constrants are near. A famous method n near programmng s the smpex
method.
The method of Lagrange mutpers can be used to reduce optmzaton
probems wth constrants to unconstraned optmzaton probems.
Evauatng ntegras|edt|
Man artce: Numerca ntegraton
Numerca ntegraton, n some nstances aso known as numerca
quadrature, asks for the vaue of a dente ntegra. Popuar methods use one
of the Newton-Cotes formuas (ke the mdpont rue or Smpson's rue) or
Gaussan quadrature. These methods rey on a "dvde and conquer" strategy,
whereby an ntegra on a reatvey arge set s broken down nto ntegras on
smaer sets. In hgher dmensons, where these methods become
prohbtvey expensve n terms of computatona ehort, one may use Monte
Caro or quas-Monte Caro methods (see Monte Caro ntegraton), or, n
modesty arge dmensons, the method of sparse grds.
Dherenta equatons|edt|
Man artces: Numerca ordnary dherenta equatons and Numerca parta
dherenta equatons
Numerca anayss s aso concerned wth computng (n an approxmate way)
the souton of dherenta equatons, both ordnary dherenta equatons and
parta dherenta equatons.
Parta dherenta equatons are soved by rst dscretzng the equaton,
brngng t nto a nte-dmensona subspace. Ths can be done by a nte
eement method, a nte dherence method, or (partcuary n engneerng) a
nte voume method. The theoretca |ustcaton of these methods often
nvoves theorems from functona anayss. Ths reduces the probem to the
souton of an agebrac equaton.
Software|edt|
Man artces: Lst of numerca anayss software and Comparson of
numerca anayss software
Snce the ate twenteth century, most agorthms are mpemented n a
varety of programmng anguages. The Netb repostory contans varous
coectons of software routnes for numerca probems, mosty n Fortran and
C. Commerca products mpementng many dherent numerca agorthms
ncude the IMSL and NAG brares; a free aternatve s the GNU Scentc
Lbrary.
There are severa popuar numerca computng appcatons such as MATLAB,
TK Sover, S-PLUS, LabVIEW, and IDL as we as free and open source
aternatves such as FreeMat, Scab, GNU Octave (smar to Matab), IT++ (a
C++ brary), R (smar to S-PLUS) and certan varants of Python.
Performance vares wdey: whe vector and matrx operatons are usuay
fast, scaar oops may vary n speed by more than an order of magntude.|5|
|6|
Many computer agebra systems such as Mathematca aso benet from the
avaabty of arbtrary precson arthmetc whch can provde more accurate
resuts.
Aso, any spreadsheet software can be used to sove smpe probems reatng
to numerca anayss.