Chapter 12:

Memory Hierarchy
Main Memory
RAM
Volatile, unless backed up with battery
Stores active programs and data

Communicates directly with the CPU

Auxiliary Memory
Magnetic (disks & tapes)
Non volatile
Saves files (programs & data)

Communicates with the CPU through a controller

1 / 19

Memory Hierarchy

CPU

Main Memory

I/O Processor

Cache

Magnetic
Disks

Magnetic Tapes
2 / 19

Cache Memory
High speed (towards CPU speed)

Small size (power & cost)

Miss

Main
Memory

CPU

Hit

Cache
(Fast)
Cache

(Slow)

Mem

95% hit ratio

Access = 0.95 Cache + 0.05 Mem

3 / 19

Cache Memory

CPU

30-bit Address
Main
Memory
Cache
1 Mword

1 Gword

Only 20 bits !!!

4 / 19

Cache Memory
00000000

00000001
00000

Cache

Main
Memory

00001

FFFFF

Address Mapping !!!

3FFFFFFF
5 / 19

Associative Memory
Cache Location

00000000

00000001
00000

Cache

00001

00012000

15000000

00012000

Main
Memory

08000000

08000000

FFFFF

Address (Key)

Data

15000000

3FFFFFFF
6 / 19

Associative Mapping
Address

00012000

Can have
any number
of locations

Cache
00012000 0 1 A 6
Data
15000000 0 0 0 5

01A6

08000000 4 7 C C

How many
comparators?

30 Bits
(Key)

16 Bits
(Data)
7 / 19

Direct Mapping
Address

What happens
when Address
= 100 00500

000 00500

00000

Cache

00500 000 0 1 A 6
Tag
00900 080 4 7 C C

Data

000 0 1 A 6

01400 150 0 0 0 5
FFFFF
Compare

20
10
16
Bits Bits Bits
(Addr) (Tag) (Data)

Match
No match

8 / 19

Direct Mapping with Blocks

Address
000 0050 0

00000

Block Size = 16

Cache

00500
01A6
000
00501
0254

00900
47CC
080
00901
A0B4

01400
0005
150
01401
5C04

FFFFF

Tag

Data

000 0 1 A 6

Compare

20
10
16
Bits Bits Bits
(Addr) (Tag) (Data)

Match
No match

9 / 19

Set-Associative Mapping
Address
000 00500
2-Way Set Associative
00000

Cache

00500 000 0 1 A 6

010 0 7 2 1
Tag1 Data1 Tag2 Data2

00900 080 4 7 C C

000 0 8 2 2

01400 150 0 0 0 5

000 0 9 0 9

000 0 1 A 6 010 0 7 2 1

FFFFF
Compare

20
10
16
10
16
Bits Bits Bits
Bits Bits
(Addr) (Tag) (Data) (Tag) (Data)

Match

Compare

No match
10 / 19

Replacement Algorithms
For Associative & Set-Associative Cache
Which location should be emptied when the cache is full
and a miss occurs?
First In First Out (FIFO)

Least Recently Used (LRU)

Distinguish an Empty location from a Full one

Valid Bit

11 / 19

Replacement Algorithms
CPU
Reference

Cache
FIFO

Miss

Miss

Miss

Hit

Miss

Miss

Miss

Hit

Hit

Miss

A
B

A
B
C

A
B
C

A
B
C
D

E
B
C
D

E
A
C
D

E
A
C
D

E
A
C
D

E
A
F
D

Hit Ratio = 3 / 10 = 0.3

12 / 19

Replacement Algorithms
CPU
Reference

Cache
LRU

Miss

Miss

Miss

Hit

Miss

Miss

Hit

Hit

Hit

Miss

B
A

C
B
A

A
C
B

D
A
C
B

E
D
A
C

A
E
D
C

D
A
E
C

C
D
A
E

F
C
D
A

Hit Ratio = 4 / 10 = 0.4

13 / 19

Writing Policy
Read Operations
Main memory and cache have duplicate data values

Write Operations
If the CPU writes to the cache only (fast), then the main
memory will have a different copy.
At some time (before emptying the cache) Write Back
the new value to the main memory.
Mark the altered data location in cache with Dirty Bit
Write Through to both the cache and the main
memory (slow) then data is always consistent.

14 / 19

Homework
Chapter 12

12-15
12-16
12-17
12-18

15 / 19

Homework
Mano
12-15 A two-way set associative cache memory uses blocks of
four words. The cache can accommodate a total of 2048
words from main memory. The main memory size is
128K x 32.
a. Formulate all pertinent information required to
construct the cache memory?
b. What is the size of the cache memory?

16 / 19

Homework
12-16 The access time of a cache memory is 100 ns and that of
main memory 1000 ns. It is estimated that 80 percent of
the memory requests are for read and the remaining 20
percent for write. The hit ratio for read accesses only is
0.9. A write-through procedure is used.
a. What is the average access time for the system
considering only memory read cycles?
b. What is the average access time of the system for both
read and write requests?
c. What is the hit ratio taking into consideration the
write cycles?

17 / 19

Homework
12-17 A four-way set-associative cache memory has four words in
each set. A replacement procedure based on the least recently
used (LRU) algorithm is implemented by means of 2-bit
counters associated with each word in the set. A value in the
range 0 to 3 is thus recorded for each word. When a hit
occurs, the counter associated with the referenced word is set
to 0, those counters with values originally lower than the
referenced one are incremented by 1, and all others remain
unchanged. If a miss occurs, the word with counter value 3 is
removed, the new word is put in its place, and its counter is
set to 0. The other three counters are incremented by 1. Show
that this procedure works for the following sequence of word
reference: A, B, C, D, B, E, D, A, C, E, C, E. (Start with A, B,
C, D as the initial four words with word A being the least
recently used.)
18 / 19

Homework
12-18 A digital computer has a memory unit of 64K x 16 and a
cache memory of 1K words. The cache uses direct
mapping with a block size of four words.

a. How many bits are there in the tag, index, block, and
word fields of the address format?
b. How many bits are there in each word of cache, and
how are they divided into functions? Include a valid
bit.
c. How many blocks can the cache accommodate?

19 / 19