Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

MH2100
Chapter IV Continuity and differentiability

Mehdi Ghasemi
mghasemi@ntu.edu.sg

September, 2013

1 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

O UTLINE

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

2 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

R EVIEW OF LIMITS AND CONTINUITY

Denition
Let f : M Rm Rn be a map, a Rm and Rn . We say f has limit at a and write lim f (x) = if for any > 0, there
xa

exists > 0 such that x M with x a < implies f (x) < . is continuous at a if a M and lim f (x) = f (a).
xa

Here for x = (x1 , . . . , xk ) Rk , the norm of x is dened by: x :=

2 x2 1 + + xk .

3 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

S PECIAL CASE : f : Rm R1
Theorem
Let f , g : Rm R, a Rm , lim f (x) =
xa 2. 2. 1

and lim g(x) =

xa

2.

Then 1. lim(f (x) + g(x)) =

xa 1 1
1 2

+ .

2. lim(f (x) g(x)) =

xa

3. If

= 0, lim g(x) =
xa

f (x)

Therefore, if f and g are continuous at a, then so are f + g, f g f and g when g(a) = 0.

4 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

S PECIAL CASE : f : Rm R1
Proof of 1.
Since lim f (x) =
xa 1

and lim g(x) =

xa

for any > 0 there exist

and 1 , 2 > 0 such that x a < 1 implies |f (x) 1 | < 2 x a < 2 implies |g(x) 2 | < 2 . For min{1 , 2 }, and any x with x a < , both and |g(x) 2 | < 2 holds. Applying inequalities |f (x) 1 | < 2 triangle inequality to |f (x) + g(x) ( 1 + 2 )| we get

|f (x) + g(x) (

2 )|

= |(f (x) 1 ) + (g(x) 2 )| |f (x) 1 | + |g(x) 2 | < 2 +2 = ,

which completes the proof of part 1.

5 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

S PECIAL CASE : f : Rm R1
Proof of 2.
xa Fix > 0 and let 1 = 2 k2 and 2 = 2k1 where | 1 | < k1 and | 2 | < k2 . Since lim f (x) = 1 and lim g(x) = 2 , corresponding to 1 , 2 > 0, x a

there exist 1 , 2 > 0 such that x a < 1 x a < 2 |f (x) 1 | < 1 |g(x) 2 | < 2 .

Now for x a < min(1 , 2 ) we have |f (x)g(x)

1 2|

= < = =

|f (x)g(x)f (x) 2 + f (x) 2 1 2 | | 2 | |f (x) 1 | + |f (x)| |g(x) 2 | k2 1 + k1 2 k2 2 k2 + k1 2k1 2 + 2 = .

3 follows similarly.
6 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

S PECIAL CASE : f : Rm R1
Recall that any projection p : Rm Rn (n m) dened by p(x1 , . . . , xm ) = (x1 , . . . , xn ) is continuous. In particular for 1 i m, pi (x1 , . . . , xm ) = xi is continuous. Combining this with previous theorem, we see that every polynomial function in x1 , . . . , xn is continuous. l Example: xk i xj = pi (x) pi (x) pj (x) pj (x).
k times l times

Also any rational function is continuous on its domain (a p(x1 ,...,xn ) rational function is a function of the form q(x1 ,...,xn ) where p and q are polynomial functions). All elementary functions, i.e., trigonometric, exponential, logarithmic and their composition are continuous on their domain.
7 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

G ENERAL CASE
Theorem
Let a Rm and f : M Rm Rn be a map with component functions f1 (x1 , . . . , xm ), . . . , fn (x1 , . . . , xm ), i.e., f (x1 , . . . , xm ) = (f1 (x1 , . . . , xm ), . . . , fn (x1 , . . . , xm )) . Then
xa

lim f (x) = = ( 1 , . . . ,

n)

if and only if
xa

lim fi (x) = i ,

i = 1, . . . , n.

8 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

G ENERAL CASE
Proof.
() Suppose that lim f (x) = = ( 1 , . . . ,
xa n ).

Then for every

> 0 there exists a > 0 such that x M, x a < f (x) Note that for each j = 1, . . . , n, |fj (x) j | = (fj (x) j )2 < .

n 2 i=1 (fi (x) i ) = f (x) <

which implies lim fj (x) = j .

xa
9 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

G ENERAL CASE
() Conversely, assume that for each j = 1, . . . , n, lim fj (x) = j .
xa

Therefore, for every > 0 and each j = 1, . . . , n there exists j > 0 such that x M, x a < j |fj (x) j | < . n If we take min(1 , . . . , n ), then x a < implies |fj (x) j | < , for each j = 1, . . . , n. Hence n f (x) = < = = , which completes the proof.
10 / 37

n 2 i=1 (fi (x) i ) n 2 i=1 ( n ) n 2 i=1 n

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

G ENERAL CASE
Corollary
Let f : Rm Rn be a map with component functions f1 , . . . , fn all of which are elementary functions. Then f is continuous at any point in its domain. Moreover domf = domf1 domfn .

Example
x sin x 2 Let f (x, y) = ( xsin 2 +y2 , ln y). Then dom x2 +y2 = R \ {(0, 0)} and dom ln y = {(x, y) : y > 0}. Therefore

domf = (R2 \ {(0, 0)}) the upper half of xy-plane.

(R R+ ) = R R+ ,

11 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

G ENERAL CASE
Example
t1 ), nd lim f (t). Let f (t) = (tt , cost2

Solution. According to the theorem, it sufces to nd lim tt and lim ln y

cos t1 . For the rst limit, let y = tt t2 t0+ t = t ln t = ln 1/t , and by Hopitals rule: t0+

t0+

and hence

t0+

lim

ln t 1/t = lim = 0, = 1/t t0 1/t2

lim
ln t 1/t

and hence
t0+

lim = et0+

= e0 = 1.

12 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

G ENERAL CASE

...continue
The second limit is
t0+

lim

cos t1 t2

1 = 2 .

t0+

lim

sin t 2t

Therefore

1 lim f (t) = (1, ). 2 t0+

13 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

H OW TO GO FROM SINGLE VARIABLE TO MULTI ?

Recall that for a function f : R R, the derivative of f at the point x = a is dened as A = f (a) = lim f (a + h) f (a) . h h0

In other words, the derivative of f at x = a is A if 0 = lim = or

f (a+h)f (a) A h f (a+h)f (a)Ah lim , h h0 h0

|f (a + h) f (a) A h| = 0. |h| h0 lim

14 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

H OW TO GO FROM SINGLE VARIABLE TO MULTI ?

Now suppose f : Rm R1 and lets develop the same idea in this case. Since we should deal with m variables, suppose h = (h1 , . . . , hm ). Following the previous idea, we may assume that A = f (a) exists, but instead of a number, it is a vector and replace | | with as we usually do in multi-variable case. Then A should satisfy the following: |f (a + h) f (a) A h| = 0. h h0 lim

15 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

H OW TO GO FROM SINGLE VARIABLE TO MULTI ?

Since the existence of limit does not depend on how we approach 0, lets assume that except for a xed 1 i m, hj = 0 when i = j. Then 0 = lim = = =
i

h0

|f (a+h)f (a)(A1 ,...,Am )(h1 ,...,hm )| h |f (a1 ,...,ai +hi ,ai+1 ,...,am )f (a)(A1 ,...,Am )(0, ,hi ,0, ,0)| lim |h i | h 0

hi 0

lim

|f (a1 ,...,ai +hi ,ai+1 ,...,an )f (a)Ai hi | |hi |

f (a1 ,...,ai +hi ,ai+1 ,...,am )f (a) hi hi 0

lim |

Ai |

= |fxi (a) Ai |. Thus Ai = fxi (a) and A = f (a) = (fx1 (a), . . . , fxm (a)). (1)
16 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

H OW TO GO FROM SINGLE VARIABLE TO MULTI ?

Now if f : Rm Rn , clearly, f1 (a + h) f1 (a) . . f (a + h) f (a) = . fn (a + h) fn (a) h1 . is an n-vector and hence in the following, A . . should hm be an n-vector as well which satises: lim f (a + h) f (a) A hT h

h0

Since hT is m 1, we conclude that A must be an n m matrix.

17 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

H OW TO GO FROM SINGLE VARIABLE TO MULTI ?

Denition
Let f : M Rm Rn be a map and a M. We say f is differentiable at x = a, if there exists an n m matrix A such that lim f (a + h) f (a) A hT = 0. h (2)

h0

The matrix A is denoted by f (a).

18 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

H OW TO GO FROM SINGLE VARIABLE TO MULTI ?

For any i = 1, . . . , n, the ith row of numerator in (2) must be fi (a + h) fi (a) Ai h. According to (1), Ai = ((fi )x1 (a), . . . , (fi )xm (a)), and eventually . .. . A= . . x1 fn (a)
x1 f1 (a)

. . . . xm fn (a)

xm f1 (a)

The matrix Anm is called the Jacobian of f at x = a and is denoted by f (a).

19 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

J ACOBIAN MATRIX
Example
Let f (x, y) = x + y, then the Jacobian matrix of f is the 1 2 matrix: f = fx fy = 1 1 . Let g(x, y) = xy, then the Jacobian matrix of g is a 1 2 matrix: g = gx gy = y x . Let h(x) = (sin x, ex ), then the Jacobian matrix of h is a 2 1 matrix: cos x x h1 h = = . ex x h2

20 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

J ACOBIAN MATRIX
Example
For f (x, y) = (x sin(xy), arctan(x + y)), then the Jacobian matrix of f is a 2 2 matrix: f =
x f1 x f2 y f1 y f2

sin(xy) + xy cos(xy) x2 cos(xy)

1 1+(x+y)2 1 1+(x+y)2

Consider the map g(x, y) = (xy2 , sin(x2 y), ex2y ), then the Jacobian matrix is 3 2: y2 2xy x g1 y g1 g 2x cos(x2 y) cos(x2 y) . g = x 2 y g2 = ex2y 2ex2y x g3 y g3

21 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

J ACOBIAN MATRIX
Theorem
If f : Rm Rn is differentiable at x = a, then f is continuous at a.

Proof
Since f is differentiable lim f (a + h) f (a) A hT = 0. h

h0

Multiplying by h = 0, we get to
h0

lim f (a + h) f (a) A hT = 0

or equivalently lim f (a + h) f (a) A hT = 0.

h0
22 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

J ACOBIAN MATRIX
But

a11 h1 + + a1m hm . . A hT = , . an1 h1 + + anm hm 0 . T and hence lim A h = . . . 0 Therefore lim f (a + h) f (a) = 0 or lim f (a + h) = f (a) which
h0 h0 h0

implies the continuity of f at x = a.

23 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

P ROPERTIES OF J ACOBIAN
Proposition
Let f , g : Rm Rn be two differentiable functions at x = a and , R. Then 1. Differentiation is linear, i.e., (f + g) (a) = f (a) + g (a). 2. Product rule: if n = 1, then (f g) (a) = f (a) g(a) + f (a) g (a).

Proof of 1
We need to show that
h 0

lim

f (a + h) + g(a + h) f (a) g(a) f (a) h g (a) h = 0. h

But this is clear:

24 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

J ACOBIAN MATRIX
...continue
by triangle inequality
f (a+h)+ g(a+h)f (a) g(a)f (a)h g (a)h h (f (a+h)f (a)f (a)h)+ (g(a+h)g(a)g (a)h) h g(a+h)g(a)g (a)h f (a+h)f (a)f (a)h + | | || h h

= 0.

Proof of 2
(fg) (a) = = = = = ((fg)x1 (a), . . . , (fg)xm (a)) (fx1 (a)g(a) + f (a)gx1 (a), . . . , fxm (a)g(a) + f (a)gxm (a)) (fx1 (a)g(a), . . . , fxm (a)g(a)) + (f (a)gx1 (a), . . . , f (a)gxm (a)) (fx1 (a), . . . , fxm (a)) g(a) + f (a) (gx1 (a), . . . , gxm (a)) f (a)g(a) + f (a)g (a).
25 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

T HE CASE f : R R:
The tangent space at the graph of a single variable function f (x) is a one dimensional line:
y = f(x)

Tangent line

15

10

-1

-0.5 -5

0.5

1.5

-10

Its equation is y = f (a) + f (a)(x a), or (f (a), 1) (x a, y f (a)) = 0.

26 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

T HE CASE f : R2 R:
The tangent space at the graph of a function in two variables f (x, y) is a two dimensional plane:

It consists of all linear combinations of the vectors (1, 0, fx (a1 , a2 )) and (0, 1, fy (a1 , a2 )). If v = (v1 , v2 , v3 ) is a non-zero vector orthogonal to both of these vectors, at point (a1 , a2 , f (a1 , a2 )), then, the tangent plane is all points (x, y, z) satisfying: (x a1 , y a2 , z f (a1 , a2 )) (v1 , v2 , v3 ) = 0.
27 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

T HE CASE f : R2 R:
The vector v should be orthogonal to (1, 0, fx ) and (0, 1, fy ), so v (1, 0, fx ) = 0, v (0, 1, fy ) = 0.

Without loss of generality, we assume that v3 = 1 and the above equations read as v1 + fx = 0 v2 + fy = 0, hence v = (fx , fy , 1), therefore the tangent plane is the set of all points satisfying: (x a1 , y a2 , z f (a)) (fx , fy , 1) = 0, in other words: z = f (a) + fx (a)(x a1 ) + fy (a)(y a2 ) = f (a) + f (a) (x a)
28 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

T HE GENERAL CASE f : Rm Rn

Denition
Given a map f : Rm Rn differentiable at a Rm , the set of all point (x, y) satisfying y = f (a) + f (a) (x a) is called the tangent space to the graph of f at point x = a.

29 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES
Example
Let f (x, y) = x2 ln(1 + xy). Find the equation of the tangent plane to the graph of f at point (2, 0). Solution. The partial derivatives of f are fx (x, y) = 2x ln(1 + xy) + fy (x, y) =
x3 1+xy . x2 y 1+xy

The point (2, 0) is in the domain of f , fx and fy . Therefore the tangent plane to Gf exists and its equation is z = f (2, 0) + fx (2, 0)(x 2) + fy (2, 0)(y 0) = 0 + 0(x 2) + 8y = 8y.
30 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES

31 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES
Example
Let g(x, y) = (x2 + y2 ). Find the equation of the tangent plane to Gg at point (0, 0). Solution. The partial derivatives of g are gx (x, y) = 2x gy (x, y) = 2y. The equation of the tangent plane at (0, 0) is z = g(0, 0) + gx (0, 0)(x 0) + gy (0, 0)(y 0) = 0 + 0x + 0y = 0, which is the equation of the xy-plane.
32 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES

33 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES
Example
Let h(x, y) = (x2 + y2 + 1) cos(x2 + y2 ). Find the equation of the tangent plane to Gh at point (0, 0). Solution. The partial derivatives of g are hx (x, y) = 2x cos(x2 + y2 ) 2x(x2 + y2 ) sin(x2 + y2 ) hy (x, y) = 2y cos(x2 + y2 ) 2y(x2 + y2 ) sin(x2 + y2 ). The equation of the tangent plane at (0, 0) is z = h(0, 0) + hx (0, 0)(x 0) + hy (0, 0)(y 0) = 1 + 0x + 0y = 1, which is parallel to the xy-plane.
34 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES

35 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES
Example
Let f (x, y) = sin(xy) 1. Find the equation of the tangent plane to Gf at point (0, 0). Solution. The partial derivatives of g are fx (x, y) = y cos(xy) fy (x, y) = x cos(xy). The equation of the tangent plane at (0, 0) is z = f (0, 0) + fx (0, 0)(x 0) + fy (0, 0)(y 0) = 1 + 0x + 0y = 1, which is parallel to the xy-plane.
36 / 37

Limits and Continuity

Derivatives and Jacobian

Tangent Spaces

TANGENT SPACES

37 / 37