28/09/2015

ML:LinearAlgebraReviewCoursera

CourseraBeta
ANOOP

ContactUs
MyContributions
LogOut

ML:LinearAlgebraReview
FromCoursera
KhanAcademyhasexcellentLinearAlgebraTutorials(http://www.khanacademy.org/#linearalgebra).
ThisonlineLinearAlgebratext
(http://en.wikipedia.org/wiki/Linear_least_squares_%28mathematics%29#Derivation_of_the_normal_equations)
isalsoanexcellentresource,particularlyforaproofofthenormalequation.

Contents
1MatricesandVectors
2AdditionandScalarMultiplication
3MatrixVectorMultiplication
4MatrixMatrixMultiplication
5MatrixMultiplicationProperties
6InverseandTranspose
7Footnotes

MatricesandVectors
Matricesare2dimensionalarrays:
a

Theabovematrixhasfourrowsandthreecolumns,soitisa4x3matrix.
Avectorisamatrixwithonecolumnandmanyrows:

w
x
y
z

https://share.coursera.org/wiki/index.php/ML:Linear_Algebra_Review

1/4

28/09/2015

ML:LinearAlgebraReviewCoursera

Sovectorsareasubsetofmatrices.Theabovevectorisa4x1matrix.
Notationandterms:
referstotheelementintheithrowandjthcolumnofmatrixA.
Avectorwith'n'rowsisreferredtoasan'n'dimensionalvector
v i referstotheelementintheithrowofthevector.
Ingeneral,allourvectorsandmatriceswillbe1indexed.
Matricesareusuallydenotedbyuppercasenameswhilevectorsarelowercase.
"Scalar"meansthatanobjectisasinglevalue,notavectorormatrix.
R referstothesetofscalarrealnumbers
n
R referstothesetofndimensionalvectorsofrealnumbers
A ij

AdditionandScalarMultiplication
Additionandsubtractionareelementwise,soyousimplyaddorsubtracteachcorrespondingelement:
[

]+ [

] = [

a+ w

b+ x

c+ y

d + z

Toaddorsubtracttwomatrices,theirdimensionsmustbethesame.
Inscalarmultiplication,wesimplymultiplyeveryelementbythescalarvalue:
[

] x = [

a x

b x

c x

d x

MatrixVectorMultiplication
Wemapthecolumnofthevectorontoeachrowofthematrix,multiplyingeachelementandsummingtheresult.

x
[

a x + b y

] = c x + d y

e x + f y

Theresultisavector.Thevectormustbethesecondtermofthemultiplication.Thenumberofrowsofthe
vectormustequalthenumberofcolumnsofthematrix.
Annxmmatrixmultipliedbyanmx1vectorresultsinannx1vector.

MatrixMatrixMultiplication
Wemultiplytwomatricesbybreakingitintoseveralvectormultiplicationsandconcatenatingtheresult

a w + b y

] = c w + d y

e w + f y

https://share.coursera.org/wiki/index.php/ML:Linear_Algebra_Review

a x + b z

c x + d z
e x + f z

2/4

28/09/2015

ML:LinearAlgebraReviewCoursera

Anmxnmatrixmultipliedbyannxomatrixresultsinanmxomatrix.Intheaboveexample,a3x2matrix
timesa2x2matrixresultedina3x2matrix.
Tomultiplytwomatrices,thenumberofcolumnsofthefirstmatrixmustequalthenumberofrowsofthe
secondmatrix.

MatrixMultiplicationProperties
Notcommutative.A B B A
Associative.(A B) C = A (B C)
The"identitymatrix",whenmultipliedbyanymatrixofthesamedimensions,resultsintheoriginalmatrix.It'sjust
likemultiplyingnumbersby1.Theidentitymatrixsimplyhas1'sonthediagonaland0'selsewhere.
Whenmultiplyingtheidentitymatrixaftersomematrix,thesquareidentitymatrixshouldmatchtheothermatrix's
columns.Whenmultiplyingtheidentitymatrixbeforesomeothermatrix,thesquareidentitymatrixshouldmatch
theothermatrix'srows.

InverseandTranspose
TheinverseofamatrixAisdenotedA

.Multiplyingbytheinverseresultsintheidentitymatrix.

Anonsquarematrixdoesnothaveaninversematrix.Wecancomputeinversesofmatricesinoctavewiththe
[1]
pinv(A)function .
Thetranspositionofamatrixislikerotatingthematrixonceclockwiseandthenreversingit:

A =

= [

Inotherwords:
T

A ij = A
ji

Footnotes
[1]:Asdescribedinthecoursevideo,thisoctavefunctioncomputesthepseudoinverse
(http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose%20pseudoinverse)forsingularmatriceswhichdo
nothaveinverses.
Next:LinearRegressionwithMultipleVariablesBacktoIndex:Main
Retrievedfrom"https://share.coursera.org/wiki/index.php?title=ML:Linear_Algebra_Review&oldid=18780"
https://share.coursera.org/wiki/index.php/ML:Linear_Algebra_Review

3/4

28/09/2015

Category:

ML:LinearAlgebraReviewCoursera

ML:LectureNotes
Thispagewaslastmodifiedon17November2013,at16:52.
Thispagehasbeenaccessed27,421times.
Privacypolicy
AboutCoursera
Disclaimers

https://share.coursera.org/wiki/index.php/ML:Linear_Algebra_Review

4/4