Math 2940: Linear Algebra for Engineers, Lecture 1:

Introduction to linear systems

Professor Katya Scheinberg

Spring 2020

Example

3x1 + 2x2 = 9 x =1
⇒ 1
x1 + x2 = 4 x2 = 3
Linear equations and their matrices dominate science, engineering, economics and management.
Matrices are a key concept in Data Science. Why? There are two main uses - 1. to describe
relationship via linear equations, e.g. linear regression and 2. to encode data, e.g. images, user
preferences, social network connections.
     
3 2 x1 9
=
1 1 x2 |{z} 4
| {z } | {z } | {z }
matrix A x vectors b
Solve in MATLAB (or Octave Online) 2 × 2 and n × n random examples.
Rank and Data Compression
Consider a large matrix: m rows, n columns, m × n entries with row rank k (we will understand
rank later, but for now it means that only k rows of the matrix are ”important”.
Each row is a sum of multiples of the ”important” k rows.

• compression factor
k(m + n)
mn
Image Compressions
Replace image by close low-rank image
Example 4000 × 2500 = 107

• Rank 50 means it will have a compression of 3%

• produces a gray-scale matrix of pixels with the condition 0 ≤ weight ≤ 1

Show example in Matlab, using images with 2448 × 4352 = 10653696 pixels. In one case (fish)
k = 217 (13.8% compression) in the other (view) k = 100 (6.3% compression). Then show
enhanced fish image with k = 500 (around 30% compression).
Goals Course stresses concepts:

• What to calculate?

1
 • What do answers mean?

• Hand calculations help us understand.

Example Cancel matrix rows using steps of the form:

• Multiply a row by a nonzero constant?

• Add a row to another row?

Aim Cancel as many rows as possible

 
1 −1 0
 2 0 2 
 
 −1 1 0 
1 1 2

Definition 1. Number of remaining rows is rank.

•
• Add 1 to 3 •• rank = 2

• Add 1 to 4  
1 −1 0
 −2 0 −2 
• Multiply 2 by -1  
 0 0 0 
• Add 2 to 4 0 0 0

Same idea for columns

•
• Add 2 to 1 •• ”column” rank = 2
 
1 −1 0
• Multiply 1 by -1  −2
 0 −2  
 0 0 0 
• Add 1 to 3 0 0 0

Theorem 1. For any matrix, column rank = row rank

Linear Systems
Made up of equations (real of complex)

• a1 x1 + a2 x1 + · · · + an xn = b

• an are coefficient; xn are variables

• A solution is a choice of values for the variables satisfying the system

• Next time we’ll see

Theorem 2. A linear system has either

• no solution, we call it inconsistent

• one solution, or

2
 • a set of infinitely many solution

Example: 2 equations, 2 variables.

Systems are equivalent when they have the same solution set, as happens under elementary operations

• swap two equations

• multiply an equation by a nonzero constant

• add a multiple of one equation to another

Reason: These operations are reversible. Hence there is a way to answer our key questions
systematically.
Example:

1~x1 − x2 − x3 = 0
   
1 −1 −1 1 −1 −1 0

  −1 4 7   −1 4 7 12 
2~−x + 4x + 7x = 12 2 1 6 2 1 6 14
 1 2 3

3~2x1 + x2 + 6x3 = 14
 coefficient matrix ( 3 × 3 ) augmented matrix ( 3 × 4 )

Solving the problem:

 
Steps: 1. x1 − x2 − x3 = 0 1 −1 −1 0
1. Add
1 to row
2 2. 3x2 + 6x3 = 12  0 3 6 12 
3. 3x2 + 8x3 = 14 0 3 8 14
2. Add (
1 ×−2 ) to row
3 | {z }
After corresponding elementary row operations

The matrix we derived from the corresponding elementary row operations is row equivalent to
the example. We can continue to solve for the most basic matrix aka set of equations:

Multiply 1
3 ×
2 Add (
2 × −3) to
3 Multiply (
2 × 1
2 )
     
1 −1 −1 0 1 −1 −1 0 1 −1 −1 0
 0 1 2 4   0 1 2 4   0 1 2 4 
0 3 8 14 0 0 2 2 0 0 1 1

This final matrix is in triangular form, which shows the unique solution by back substitution

3
 
1~x1 − x2 − x3 = 0
 other posiblities:

 
1 −1 −1 0
2~x2 + 2x3 = 4
  0 1 2 4  • last row was (0000)
 0 0 1 1 meaning infinite solu-
3~x3 = 1
 tions (x3 is arbitrary)

• last row was (0002)

meaning it is inconsis-
tent

Finish solving:

1. Add
3 to row
 
1 1 −1 0 1
 0 1 0 2 
2. Add (
1 × −2 ) to row
3
0 0 1 1

Add
2 to row
1 Solution: •
•• x1 = 1, x2 = 2, x3 = 3
 
1 0 0 3
 0 1 0 2 
0 0 1 1