Welcome to CS 136 (Winter 2011)

Instructors: Terry Anderson, Byron Weber Becker, John Whissell

Tutors: Zachary Frenette, Alex Lake, Andrew Naismith, Sunny Wu,

Huma Zafar

Instructional Assistants (IAs): John Doucette, David Loker,

Caelyn McAulay, Alejandro Salinger

Web page (main information source):

http://www.student.cs.uwaterloo.ca/˜cs136/
Lectures: 5 sections, each Tuesday and Thursday

Tutorials: 9 sections on Mondays

CS 136 Winter 2011 01: Introduction to CS 136 1

Textbooks: “How to Design Programs” (HtDP) by Felleisen, Flatt,
Findler, Krishnamurthi (http://www.htdp.org) and “C
Programming: A Modern Approach” (CP:AMA) by K. N. King.

Presentation handouts: available on Web page

Marking Scheme: 25% assignments (weekly), 30% midterm, 45%

final exam + bonus

⇒ Your assignment average and weighted exam average must

both be greater than 50%

Software: RunC will be used for Racket/Scheme and C

CS 136 Winter 2011 01: Introduction to CS 136 2

Where to begin?

Specification Abstract Data

An Idea
Type (ADT)

Implementation

Time and Analysis Code

Space
(Racket, C, ...)
Complexity

CS 136 Winter 2011 01: Introduction to CS 136 3

An Idea
A collection of items such that:
add remove
• We can add an item to the collection
• We keep
track of the order in which items were added 4

• The item that was added most recently 22

is the only one that can be seen or removed

18

• We can see
if there is at least one item in the collection

CS 136 Winter 2011 01: Introduction to CS 136 4

Abstract Data Types (ADTs)
This idea can be precisely described as an Abstract Data Type
(ADT)

An ADT specifies two things:

• The type of data that is stored

• A set of operations that can be performed on the data
This specification does not depend on any specific programming
language

CS 136 Winter 2011 01: Introduction to CS 136 5

ADT Stack
Our idea is called a Stack

The type of data is described as a sequence of items

[a1 , a2 , . . . , an ] such that n ≥ 0

CS 136 Winter 2011 01: Introduction to CS 136 6

We will allow the following five operations:

• Make an empty stack (make-stack)

• Add an item to the top of a stack (push)
• Remove an item from the top of a stack (pop)
• Look at the item at the top of a stack (peek)
• Check if the stack is empty (stack-empty?)
However, these descriptions are not precise

CS 136 Winter 2011 01: Introduction to CS 136 7

Specification of Operations
For each ADT operation, we must specify:

• The parameters (if any)

• Preconditions: What must be true before performing this
operation?

• Postconditions: Assuming the preconditions have been satisfied,

what will be true after the operation has been performed?

What will happen if the preconditions of an operation are not

satisfied?

CS 136 Winter 2011 01: Introduction to CS 136 8

Specification of Stack Operations
make-stack

• Parameters: None
• Pre: True (i.e. there are no preconditions)
• Post: The empty stack S = [] is produced
push

• Parameters: An item x and a stack S = [a1 , . . . , an ], n ≥ 0

• Pre: True
• Post: The stack S 0 = [x, a1 , . . . , an ] is produced

CS 136 Winter 2011 01: Introduction to CS 136 9

Implementation of an ADT
Implementation: Code to make each operation work as intended

This is dependent on:

• The choice of programming language

• How data is organized and stored within the language
Additional software engineering considerations:

• Grouping ADT implementation details together

• Controlling access to those details

CS 136 Winter 2011 01: Introduction to CS 136 10

Implementation of ADT Stack
;; stack1.ss
#lang racket

;; specify the operations that are ’visible’ to the user

(provide make-stack push pop peek stack-empty?)

(define make-stack (lambda () empty))

(define (push item S) (cons item S))
(define pop rest)
(define peek first)
(define stack-empty? empty?)

CS 136 Winter 2011 01: Introduction to CS 136 11

Using this Implementation
;; stack-test.ss
#lang racket

;; specify the implementation we are using

(require "stack1.ss")

(stack-empty? (pop (push 1 (make-stack))))

CS 136 Winter 2011 01: Introduction to CS 136 12

What can go wrong here?
The implementation in stack1.ss works

However, it is possible to violate the requirements that items can

only be added, removed, or accessed at the top of the stack:

(append (push 7 (push 1 (make-stack))) (list 4 5 6))

Solution: “Hide” the data inside of a structure

CS 136 Winter 2011 01: Introduction to CS 136 13

A Better Implementation
;; stack2.ss
#lang racket
(provide make-stack push pop peek stack-empty?)
;; ’hide’ the data using a structure
(define-struct stk (lst))
(define (make-stack) (make-stk empty))
(define (push v s) (make-stk (cons v (stk-lst s))))
(define (pop s) (make-stk (rest (stk-lst s))))
(define (peek s) (first (stk-lst s)))
(define (stack-empty? s) (empty? (stk-lst s)))

CS 136 Winter 2011 01: Introduction to CS 136 14

Introduction to Algorithm Analysis
(define (len lst)
(cond
[(empty? lst) 0]
[else (+1 (len (rest lst)))]))

Let T (n) be the running time of len when applied to a list of length
n (measured as the number of substitution steps required)

CS 136 Winter 2011 01: Introduction to CS 136 15

A condensed trace
Consider the case where n =4
(len ’(p q r s))
⇒ . . . ;; simplification of cond
⇒ (+ 1 (len ’(q r s)))
⇒ . . . ;; T(3) steps
⇒ (+ 1 3)
⇒4

In general, the recursive application (len (rest lst)) takes T (n − 1)

steps

CS 136 Winter 2011 01: Introduction to CS 136 16

We can give the value of T (n) as a recurrence relation

 a n=0
T (n) =
 b + T (n − 1) n > 0

Here, a and b are constants

Solution: T (n) = a + bn

CS 136 Winter 2011 01: Introduction to CS 136 17

Another Example
(define (sum-of-divisors n)
(foldr + 0
(filter (lambda (y) (zero? (remainder n y)))
(build-list n (lambda (x) (add1 x))))))

A quicker analysis:

• Number of steps for build-list, filter, and foldr?

CS 136 Winter 2011 01: Introduction to CS 136 18

Big-oh Notation
A convenient way to classify the “size” of a function. Common
classifications include:

• O(1) (constant)
• O(log n) (logarithmic)
• O(n) (linear)
• O(n log n) (loglinear)
• O(n2 ) (quadratic)
• O(2n ) (exponential)

CS 136 Winter 2011 01: Introduction to CS 136 19

Informally, use the leading term and ignore coefficients

len has running time T (n) = a + bn ∈ O(n)

sum-of-divisors has running time
T (n) = an + bn + cn = (a + b + c)n ∈ O(n)
O(n) is called linear running time

CS 136 Winter 2011 01: Introduction to CS 136 20

Formal Definition of Big-oh
f (n) is O(g(n)) if there exist positive constants c and
Definition:
n0 such that f (n) ≤ c · g(n) for all n ≥ n0
“c · g(n) eventually dominates f (n)”

CS 136 Winter 2011 01: Introduction to CS 136 21

Example: f (n) = 3n2 + 2 and g(n) = n2

2
c g(n) = 5n

f(n) = 3n2 + 2

n0 = 1 n

Selecting c = 5 and n0 = 1 shows that f (n) ∈ O(g(n))

CS 136 Winter 2011 01: Introduction to CS 136 22

Big-oh Proofs
• 3n3 + 4n2 − 3n + 10 is O(n3 )
• 3n3 + 4n2 − 3n + 10 is O(n9 )
• 2n2 + 1 is not O(n)
• If f (n) ∈ O(g(n)) and g(n) ∈ O(h(n)) then
f (n) ∈ O(h(n))

CS 136 Winter 2011 01: Introduction to CS 136 23

Back to Algorithm Analysis
From now on, we will use big-oh notation to describe the running
times of functions (don’t worry about ”exact” running times)

Running times of some common built-in functions:

O(1) (Constant running time): cons, first, rest, empty?, and, or, not,
numeric compuations on bounded numbers, operations on
structures, operations on symbols

O(n) (Linear running time): length, reverse, append, string

comparisons

O(n2 ) (Quadratic running time): Quicksort (worst case)

CS 136 Winter 2011 01: Introduction to CS 136 24

Addition Rule: If function A takes O(f (n)) time and function B
takes O(g(n)) time, then executing A followed by B takes
O(f (n) + g(n)) time.
For example, build-list, filter, and foldr each take O(n) time

Therefore sum-of-divisors takes

O(n + n + n) = O(3n) = O(n) time

CS 136 Winter 2011 01: Introduction to CS 136 25

Multiplication Rule: If function A takes O(f (n)) time, then
executing A O(g(n)) times takes O(f (n) · g(n)) time in total.

(define (reverse-search lst target)

(cond
[(empty? lst) false]
[(equal? (first lst) target) true]
[else (reverse-search (reverse (rest lst)) target)]))

What is the running time of reverse-search?

We can also determine the running time by using a summation

CS 136 Winter 2011 01: Introduction to CS 136 26

Best Case vs. Worst Case
But what if target is the first item in lst?

Best Case: For an input of size n, what contents of the input will
give the function its quickest running time?

• The fewest number of steps

• reverse-search has a best case running time of O(1) when (first
lst) equals target

It is incorrect to say that “the best case running time for a function is
O(1) when n=0” (running times must hold for all n)

CS 136 Winter 2011 01: Introduction to CS 136 27

Worst Case: For an input of size n, what contents of the input will
give the function its slowest running time?

• The largest number of steps

• reverse-search has a worst case running time of O(n2 ) when
target is not in lst

Some functions have the same best and worst case running times;
here the contents of the input do not matter

CS 136 Winter 2011 01: Introduction to CS 136 28

The Tabular Method of Analysis
;; The Insertion Sort algorithm
;; isort: (listof num) → (listof num)
(define (isort alon)
(cond
[(empty? alon) empty]
[else (insert (first alon)
(isort (rest alon)))]))

CS 136 Winter 2011 01: Introduction to CS 136 29

;; insert: num (listof num) → (listof num)
(define (insert n alon)
(cond
[(empty? alon) (cons n empty)]
[(< n (first alon)) (cons n alon)]
[else (cons (first alon)
(insert n (rest alon)))])))

CS 136 Winter 2011 01: Introduction to CS 136 30

Example with Local
;; Finding the maximum of a list
;; list-max: (listof num) → num
(define (list-max alon)
(cond
[(empty? (rest alon)) (first alon)]
[else
(local [(define max-rest (list-max (rest alon)))]
(cond
[(> (first alon) max-rest) (first alon)]
[else max-rest]))]))

CS 136 Winter 2011 01: Introduction to CS 136 31

Example using Recursion Depth
;; Compute the binary representation of a natural number
;; binary-digit-list: nat → (listof nat[≥1])
(define (binary-digit-list n)
(cond
[(zero? n) empty]
[(even? n) (cons 0 (binary-digit-list (quotient n 2)))]
[else (cons 1 (binary-digit-list (quotient n 2))]))

CS 136 Winter 2011 01: Introduction to CS 136 32

True or False?
• Analysis of an algorithm’s running time depends on the
computer used to run the algorithm

• If f (n) is O(n2 ), then it is also O(n3 ) and O(2n )

• If Algorithm A has O(n) running time and Algorithm B has
O(n2 ) running time, then A will always perform faster than B
• f (n) = loga n is O(logb n) where a and b are two different
positive constants

• The best case running time of any function is O(1) which

occurs when n = 0

CS 136 Winter 2011 01: Introduction to CS 136 33

Order Notation
An alternative definition of Big-O (from a document on the Handouts
web page) is:

Definition 2: O(g(n)) is the set of functions f (n) such that

there exist constants c and n0 such that for all n ≥ n0 ,
f (n) ≤ c · g(n).
The growth rates of sample members of these sets is shown on the
next slide.

CS 136 Winter 2011 01: Introduction to CS 136 34

18000

16000 O(log2 n)
O(n)
O(n log2 n)
14000
O(n2)
O(n3)
12000

10000

8000

6000

4000

2000

0
50
1

0
10

15

20

25

30

35

40

45

50

55

60

65
CS 136 Winter 2011 01: Introduction to CS 136 35
Does it really work?
Jon Bentley describes an experiment in Programming Pearls, p. 75.
The problem is to take a list of N real numbers and return the
maximum sum found in any contiguous sublist. For example:

31 -41 59 26 -53 58 97 -93 -23 84

He describes four algorithms to solve the problem. They are O(n3 ),

O(n2 ), O(n lg n), and O(n). To prove that constant factors don’t
matter much, he deliberately tried to make the constant factors of
the O(n3 ) and O(n) algorithms differ by as much as possible.

CS 136 Winter 2011 01: Introduction to CS 136 36

 O(n) on a TRS-80 hobbyist O(n3 ) on a Cray supercom-
computer puter

CS 136 Winter 2011 01: Introduction to CS 136 37

O(n3 ) algorithm: Cray-1, finely-tuned Fortran, 3.0n3 nanoseconds
O(n) algorithm: TRS-80, interpreted Basic, 19.5n milliseconds =
19, 500, 000n nanoseconds
n O(n) O(n3 )
10 .2 secs .000003 secs

100

1,000

10,000

100,000

1,000,000

CS 136 Winter 2011 01: Introduction to CS 136 38

O(n3 ) algorithm: Cray-1, finely-tuned Fortran, 3.0n3 nanoseconds
O(n) algorithm: TRS-80, interpreted Basic, 19.5n milliseconds =
19, 500, 000n nanoseconds
n O(n) O(n3 )
10 .2 secs .000003 secs

100 2.0 secs .003 secs

1,000 20 secs

10,000

100,000

1,000,000

CS 136 Winter 2011 01: Introduction to CS 136 39

Goals of this module
You should understand the idea of an ADT, the examples given, and
how we can provide several implementations of an ADT. We will be
using ADTs in lecture and on assignments.

You should understand how to hide implementation details within a

Scheme module.

You should strengthen your intuition regarding the use of O -notation

in describing the running time of functions.

We will be asking you to aim for certain running times on

assignments and to provide analyses of your own code.

CS 136 Winter 2011 01: Introduction to CS 136 40