MovingAverageProcessTechnicalNote 1 SpiderFinancialCorp,2014

Technical Note: Moving Average Model

Occasionally,wereceiverequestsforatechnicalissueaboutARMAmodelingbeyondourregularNumXL
support,whichdelvesmoreintothemathematicalformulationofARMA.Wearealwayshappytohelp
ouruserswithanyquestiontheymayhave,sowedecidedtoshareourinternaltechnicalnoteswith
you.
Thesenoteswereoriginallycomposedwhenwesatinonatimeseriesanalysisclass.Overtheyears,
wevemaintainedthesenoteswithnewinsights,empiricalobservationsandintuitionsacquired.We
oftengobacktothesenotesforresolvingdevelopmentissuesand/ortoproperlyaddressaproduct
supportmatter.
Inthispaper,wellgooverasimple,yetfundamental,econometricmodel:movingaverage.Thismodel
servesasacornerstoneforallseriousdiscussiononARMA/ARIMAmodels.
Background
Amovingaveragemodeloforder q (i.e. MA(q) )isdefinedasfollows:

1 1 2 2
...
~i.i.d~ (0,1)
t t t q t q t
t t
t
x a a a a
a
N
u u u
o c
c

= + + + + +
=
Where
-
t
a istheinnovationsorshocksforourprocess
- o istheconditionalstandarddeviation(akavolatility)
Theoutputvalue(
t
x )issolelydeterminedbyalongrunaverage( )andaweightedsumofpastshocks
orinnovations({ }
t
a ).
Stability
Bydefinition,theMAprocessisstableandhasafinitelongrunmean( )andvariance:
(1) Theunconditional(i.e.longrun)meanissimply

1 1 2 2
[x ] [ ... ]
t t t q t q t
E E a a a a u u u

= + + + + + =
(2) Theunconditional(i.e.longrun)varianceisdefinedasfollows:

MovingAverageProcessTechnicalNote 2 SpiderFinancialCorp,2014

2
1 1 2 2
2 2 2 2
1 2
Var[ ]= [(x )(x )] [( ... ) ]
Var[ ] (1 ... )
t t t t t q t q t
t q
x E E a a a a
x
u u u
u u u o

= + + + +
= + + + +

a. Forafiniteorder q ,theprocessisguaranteedtobestable(i.e.doesnotconvergeto
infinity).
b. Foraninfiniteorder q (i.e. ( ) MA ),theprocessisstableonlyifthelongrunvarianceis
finite:

2 2 2 2 2
1 2
1
[ ] (1 ...) (1 )
t i
i
Var x u u o u o

=
= + + + = +

Inotherwords,thesumofthesquaredvaluesoftheMAcoefficientsisfinite.

2
1
i
i
u

=
<

Forecast
Givenaninputsampledata
1 2
{ , ,..., }
T
x x x ,wecancalculatevaluesofthemovingaverageprocessfor
future(i.e.outofsample)valuesasfollows:

1 1 2 2
1 1 1 2 1 1
2 1 2 3 1 2
1
1
0
...
[ | , ,..., ] ...
[ | , ,..., ] ...
...
[ | , ,..., ]
[ | , ,..., ]
T T T T q T q
T T T T q T T q T q
T T T T q T T q T q
T q T T T q q T
T q k T T T q
k
x a a a a
E x a a a a a a
E x a a a a a a
E x a a a a
E x a a a
u u u
u u u
u u u
u

+ +
+ +
+
+ +
>
= + + + + +
= + + + +
= + + + +
= +
=

Thevariance(squaredstandarderror)oftheoutofsamplevaluesisexpressedasfollows:

2
1 1 1 1 2 1 1
2 2
2 1 2 1 1 2 2 1
3 1 3 1 2 2 1 3
Var[ | , ,..., ] Var[ ... ]
Var[ | , ,..., ] Var[ ... ] (1 )
Var[ | , ,..., ] Var[ ... ] (
T T T T q T T T q T q
T T T T q T T T q T q
T T T T q T T T q T q
x a a a a a a a
x a a a a a a a
x a a a a a a a
u u u o
u u u u o
u u u
+ + +
+ + + +
+ + + + +
= + + + + =
= + + + + = +
= + + + + =
2 2 2
1 2
2 2 2 2
1 1 2 1
2 2 2 2
1 1 2
0 0
1 )
...
Var[ | , ,..., ] (1 ... )
Var[ | , ,..., ] Var[ ] (1 ... )
T q T T T q q
T q k T T T q T q k q
k k
x a a a
x a a a x
u u o
u u u o
u u u o
+
+ + + +
> >
+ +
= + + + +
= = + + + +

MovingAverageProcessTechnicalNote 3 SpiderFinancialCorp,2014

Note:Theconditionalvariancegrowscumulativelyoverqstepstoreachitslongrun(unconditional)
variance.
Stationarity
Bydefinition,aweaksensestationary(weakstationary)processrequiresthefirstmoment(i.e.mean)
andcovariancedonotvarywithrespecttotime.
Forthefirstmoment,theconditionalmeanisobviouslytimeinvariant:

0 0
[x ] [x ] [x ]
t t k t k
k k
E E E
+
> >
= = =
Forthesecondmoment(i.e.varianceandautocovariance),letsexaminethisassumption.Bydefinition,
theautocovarianceforlagorderjisexpressedasfollows:
[( )( )]
j t t j
E x x

=
Theautocovarianceoflagorderzero(
o
)isthesameastheunconditionalvariance:

2 2 2 2
1 2
(1 ... )
o q
u u u o = + + + +

2
1 1 1 2 2 3 3 4 1
2
2 2 1 3 2 4 3 5 2
2
3 3 1 4 2 5 3 6 3
2
( ... )
( ... )
( ... )
...
0
q q
q q
q q
q q
k q
u u u u u u u u u o
u u u u u u u u u o
u u u u u u u u u o
u o

>
= + + + + +
= + + + + +
= + + + + +
=
=

Toexamineforvariabilitywithrespecttotime,itissufficienttoexaminethefollowing:

[( )( )] [( )( )]
( )
j t t j j t t j
j j
E x x E x x
F j


+

= = =
= =

Usingeitherdefinition,wecaneasilyshowthattheautocovariancefunctiondoesnotvarywithrespect
totime(t),butisratherdeterminedsolelybythelagorderj.Insum,themovingaverageprocessisa
weaksensestationaryprocess.

MovingAverageProcessTechnicalNote 4 SpiderFinancialCorp,2014

Correlogram
Whatdothemovingaveragecorrelogramplotslooklike?Canweidentifyamovingaverageprocess
(anditsorder)fromcorrelogramplots(i.e.ACFandPACF)?
Theautocorrelationfunction(ACF)isdefinedastheratioofautocovarianceandunconditional
variance:
ACF =
j
j j
o

=
Bydefinition; 1
o
=

andthenextqACF:

1 1 2 2 3 3 4 1
1 2 2 2 2
1 2 3
2 1 3 2 4 3 5 2
2 2 2 2 2
1 2 3
3 1 4 2 5 3 6 3
3 2 2 2 2
1 2 3
2 2 2 2
1 2 3
...
1 ...
...
1 ...
...
1 ...
...
1 ...
0
q q
q
q q
q
q q
q
q
q
q
k q
u u u u u u u u u

u u u u
u u u u u u u u u

u u u u
u u u u u u u u u

u u u u
u

u u u u

>
+ + + + +
=
+ + + +
+ + + + +
=
+ + + +
+ + + + +
=
+ + + +
=
+ + + +
=

Whatdoesthismean?IftheACFplotexhibitssignificantvaluesforthefirstjlagsandthendropsto
zero,wecanprobablymodeltheprocesswithamovingaveragemodeloforderj.
UsingonlytheACFplot,IshouldbeabletoconstructanMAmodelforanyprocess,right?Yes.
CantwoormoredifferentMAprocesseshavethesameACFfunctionvalues?Yes.
Isthereaspecialcase(restricted)ofMAprocessesthathaveauniqueACFfunction?Yes,theyarecalled
invertibleMAprocesses.
Intheory,notwoinvertibleMAprocesseshavethesameACFfunction.

MA Invertability
AninvertibleMAmodel MA(q) isonethatcanberepresentedasaconverginginfiniteorderauto
regressive AR( ) model.Byconverging,wemeanthattheARcoefficientdecreasestozeroaswego
backintime.

MovingAverageProcessTechnicalNote 5 SpiderFinancialCorp,2014

Usingthelagoperator( L )notation,themovingaverageprocesscanberepresentedasfollows:

2
1 2
2
1 2
(1 ... )
(1 ... )
q
t t q t
t
t q
q
x y L L L a
y
a
L L L
u u u
u u u
= = + + + +
=
+ + + +

Usingpartialfractiondecomposition:

1 2
1 2
1 2
(1 )(1 )...(1 )
...
1 1 1
t
t
q
q
t t
q
y
a
L L L
c
c c
y a
L L L


=
+ + +
(
+ + + =
(
+ + +
(

Where{ }
i
isthesetofcharacteristicrootsfortheMAprocess.
i
cantakerealorcomplexvalues.
Now,assumingthat
i
fallsinsidetheunitcircle(i.e. 1
i
< )for i q s ,eachfractioncanbe
expressedasaconverginggeometricseries:

2 2 3 3 4 4
1
1 ...
1
i i i i
i
L L L L
L

= + +
+

Applyingthistransformationtoallpartialfractions,youcanexpresstheMAprocessasaninfiniteorder
ARprocessfollows:

2 2 2 2
1 2 1 1 2 2 1 1 2 2
2 3
1 2 3
( .. ) ( .. )L ( .. )L ...
[ ... ( 1) L ...)
q q q q q t t
k k
o k t t
c c c c c c c c c y a
L L L y a

| | | | |
( + + + + + + + + + + =

+ + + + =

Where
-
1 1 2 2
..
k k k
k q q
c c c | = + + +
- lim 0
k
k
|

=
Tocheckforinvertibility,itissufficienttofindthecharacteristicsroots
i
andverifythattheirvaluesfall
insidetheunitcircle( 1
i
< ).Goingforward,wewillonlyconsidertheinvertibleMAprocess.
Example:MA(1)
(1 )
t t
x L a u = +

MovingAverageProcessTechnicalNote 6 SpiderFinancialCorp,2014

ThecharacteristicrootfortheMA(1)processisu .Assuming 1 u < ,thealgebraicARrepresentationof

theMA(1)isexpressedasfollows:

2 2 3 3
(1 ...)( )
1
t
t t
x
L L L x a
L

u u u
u

= + + =
+

TheARrepresentationisconvergingas lim 0
k
k
u

= for 1 u <
Wearenotquitedoneyet.Thereareacoupleofpivotalmathematicaltricksthatwestillneedtocover:
(1)Impulseresponsefunction(IRF)and(2)Integratedmovingaverageprocess.
1. Impulse Response Function (IRF)
TheimpulseresponsefunctiondescribesthemodeloutputtriggeredbyasingleshockattimeT.

0 1
1 1
t
t
a
t
=
=

=

Applyingittoamovingaveragemodeloforderq,theprocessvalueisasfollows:

1
2 1
3 2
1 x
x
x
u
u
=
=
=

1
1
...
q q
q q
x
x
u
u

+
=
=

Note:theIRFforanMAprocessisfinite( 1 q + )anditsvaluesareequaltothecoefficient.
IfwehaveanIRFofsomeunknownprocess,canwemodelitasanMAprocess?Youbet!
FindingthevaluesoftheMAcoefficientsalgebraicallycanbetediousandcomplex,butusingtheIRF
approachcanvastlysimplifythetask.
Example:ARMA(p,q)
ConsiderthegeneralARMA(p,q)process:

1 1 2 2 1 1 2 2
2 2
1 2 1 2
x x x ... x ...
OR
(1 ... )x (1 ... )
t t t p t p t t t q t q
p q
p t q t
a a a a
L L L L L L a
| | | u u u
| | | u u u

= + + + + + + + +
= + + + +

MovingAverageProcessTechnicalNote 7 SpiderFinancialCorp,2014

WewishtoderivetheMArepresentationoftheprocess.Wecandividethetwocomponents(i.e.AR
andMA)intopolynomials:

2
1 2
2
1 2
(1 ... )
x
(1 ... )
q
q
t t p
p
L L L
a
L L L
u u u
| | |
+ + + +
=


OrwecanusetheIRFoftheARMAprocesstoderivethevaluesoftheMAcoefficients:

1 1 1 1 1
2 1 1 2 2 1 1 1 2 2
3 1 2 2 1 3 3 1 2 2 1 3 3
1 1 2 1 1
x 1
x x
x x x ( )
x x x x x x
...
x x x ... x
t t
t t t
t t t t
t t t t t t t
t q t q t q q t
p q
a
a
a
a
| u | u
| | u | | u | u
| | | u | | | u
| | |
+
+ +
+ + + + +
+ + + + +
>
= =
= + = +
= + + = + + +
= + + + = + + +
= + + +

DerivingtheMAcoefficientvaluesisaniterativeandstraightforwardprocessthatwillsaveusfrom
carryingoutcomplexpolynomialdivision.
Bynow,youmaybewonderingwhywewouldwishtoconvertafiniteorderARMAprocesstoan
infiniteorderMArepresentation.Forstarters,forecasting(meananderror)usinganMArepresentation
ismucheasierthanusingtheoriginalhigherorderARMArepresentation.
2. Integration
Integration(i.e.unitroot)oftenarisesintimeseries(e.g.randomwalk,ARIMA,etc.).Inthesesituations,
wemodelthedifferencedtimeserieswithanARMAclassmodel:

2 2
1 2 1 2
(1 L) (1 )
(1 ... ) (1 ... )
d s D
t t
p q
p t q t
L x y
L L L y L L L a | | | u u u
=
= + + + +

ButhowdowetaketheARMAoutputsbacktotheundifferencedscale?
Example1:ConsiderafirstorderintegrationoftheMA(q)process:

2
1 2
(1 L) (1 ... )
q
t q t
x L L L a u u u = + + + +
Tocalculatetheoutofsample(i.e.forecast)values:

MovingAverageProcessTechnicalNote 8 SpiderFinancialCorp,2014

2
1 1 2 1
1 1 1 2 1 1
2 1 1 1 2 3 1 2
2 2 1 1 1 2 2 3 1 1 2 1
3 3
(1 ... )
...
...
(1 ) ( ) ( ) ... ( )
(
q
T T q T
T T T T T q T q
T T T T T q T q
T T T T T T q q T q q T q
T T T
x x L L L a
x x a a a a
x x a a a a
x x a a a a a a
x x a
u u u
u u u
u u u u
u u u u u u u u
+ +
+ + +
+ + + +
+ + + + +
+ +
= + + + +
= + + + + +
= + + + + + =
= + + + + + + + + + + +
= + +
1 2 1 2 1
1 2 3 2 1 3 1 2 1
1 1 1 2 2 1 2
1 2 2
1 ) (1 )
+( ) ... ( ) ( )
(1 ) (1 ) ... (1 ... )
+( .. ) ( ..
T T
T q q q T q q q T q q T q
T k T T k T k T k q T k q
k q
q T
a a
a a a a
x x a a a a
a
u u u
u u u u u u u u u
u u u u u u
u u u u
+ +
+ + +
+ + + + +
>
+ + + +
+ + + + + + + + +
= + + + + + + + + + + + +
+ + + + + +
1 1
) ...

q T q T q
a a u u
+
+ +

Alternatively,

2 3
1 1 2 1 2 3 1 2
1 2 2 1 3 1 1
(1 (1 ) (1 )L (1 )L ... (1 .. )L )
( .. ) ( .. ) ( .. ) ...
q
T k q T k
k q
T q T q T q T q T q
x L a M
M x a a a a
u u u u u u u u u
u u u u u u u u
+ +
>
+
= + + + + + + + + + + + + + + + +
= + + + + + + + + + + + +

Furthermore,thevarianceoftheforecastvalueisconstantafterqsteps:

2 2 2 2 2
1 1 2 1 2 3 1 2
Var[ ] (1 (1 ) (1 ) (1 ) ... (1 .. ) )
T k q
k q
x u u u u u u u u u o
+
>
= + + + + + + + + + + + + + + +
Insum,theintegratedzeromeanMAprocessyieldedanotherMAprocess,butwithamean.
Whatifthedifferencedtimeserieshasanonzeromean?

2
1 2
(1 L) (1 ... )
q
t q t
x L L L a u u u = + + + + +
Inthiscase,wehaveatimetrendofrateequalto

2 3
1 1 2 1 2 3 1 2
1 2 2 1 3 1 1
(1 (1 ) (1 )L (1 )L ... (1 .. )L )
( .. ) ( .. ) ( .. ) ...
q
T k q T k
k q
T q T q T q T q T q
x k L a M
M x a a a a
u u u u u u u u u
u u u u u u u u
+ +
>
+
= + + + + + + + + + + + + + + + + +
= + + + + + + + + + + + +

Forhigherorderintegrationd( 1 d > ),werepeattheprocedureabovedtimes,butwewouldneed

morerealizedobservationsthanthelastone.Forexample:

2 2
1 2
(1 L) (1 L) (1 ... )
(1 L)
q
t t q t
t t
x y L L L a
x y
u u u = = + + + + +
=

MovingAverageProcessTechnicalNote 9 SpiderFinancialCorp,2014

Tosolvefor
t
y ,wedneed
T
y
,
whichiscalculatedby
1 T T
x x

.Next,tosolvefor
t
x usingthe
t
y process,
wedneed
T
x .Insum,
1
( , )
T T
x x

arerequiredasinitialconditionsfortheintegration.
Conclusion
Themovingaverageprocess,despiteitssimplicity,isaratherusefulmodeltoworkwith,especially
whenitcomestoforecasting.
Armedwithacoupleofmathematicaltricks(IRFandIntegration),wecantacklemanymorecomplex
processesbyrepresentingthemfirstbyanMA.
Infuturetechnicalnotes,wewilldiscussadvancedmodels,butoftenrefertotheMAprocessand
materialpresentedhere.





References
- Hamilton, J .D.; Time Series Analysis , Princeton University Press (1994), ISBN 0-691-04289-6
- D. S.G. Pollock,; Handbook of Time Series Analysis, Signal Processing, and Dynamics , Academic Press (1999),
ISBN: 0125609906
- Box, J enkins and Reisel; Time Series Analysis: Forecasting and Control , J ohn Wiley & SONS. (2008) 4th
edition, ISBN:0470272848