Cache Replacement Policy

Based on
Expected Hit Count

A. Vakil-Ghahani S. Mahdizadeh M. Lotfi-Namin

M. Bakhshalipour P. Lotfi-Kamran H. Sarbazi-Azad

CRC-2: The 2nd Cache Replacement Championship

ISCA 2017
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 2

Intro

Problem
• Off-chip accesses stall the processor for hundreds of
cycles
• Limited cache size
o Latency
o Limited silicon area

One Solution
• Improving replacement policy
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 3

Replacement Policy

• Determine a victim in the case of a conflict

• Most locality is captured by first level caches
◦ Simple approaches like LRU
◦ Inefficient for LLC

• Need a more accurate replacement policy

◦ Better approximation of Belady's MIN
◦ Last level cache
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 4

Observation

•Blocks with more remaining hit counts will re-reference

earlier in future
lbm libquantum mcf xalanc
100
Average Reuse Distance

80

60

40

20
1 2 3 4 5 6 7+
Number of Remaining Hits
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 5

The Proposal: Expected Hit Count (EHC)

Evict the block with the minimum expected remaining hit

count
Resident Blocks Incoming
Block
Tag A B C D E

Predicted Remaining 0 5 2 1 1
Hit Count

Comparator

Block with Minimum

Predicted Hit Count A
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 6

EHC

• Hit-count predictor
• Hit-counter per block
• number of hits of the block since the entrance to the cache
• Store the number of recently hit counts in a table (HHT)
o Set-associative structure
o LRU replacement policy
o Indexed by block’s tag
o To save area

• Use information from HHT for selecting a victim

• Need a baseline policy
o DRRIP
o good performance
o low area overhead
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 7

HHT Structure

• Set-associative, 16-way • LRU Replacement Policy

• Hit Count Array: A FIFO queue that stores the two
last experienced hit counts Hit Count Array
16 WAY

LRU HIT HIT

VALID RECENCY COUNTER COUNTER
(4- bit) (3- bit) (3- bit)
128 SETS

TAG (20- bit)

25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 8

Updating Metadata

• Eviction of a block • Saturation of hit count of a block

Hit
Data Tag RRPV
Cnt
. . . .
. . . .
. . . .
Hit
Data Tag RRPV
Cnt

7- LSB bits of tag

20- bit after index

HHT Index
HHT Tag

HHT

Hit Count of Evicted/

Saturated Block
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 9

EHC: Victim Selection

Calculate value for each block

Resident Blocks Incoming Block

A B C D E

Default HitCnt
RRPV HitCnt RRPV HitCnt RRPV HitCnt RRPV HitCnt
RRPV (0)

Tag EHC(E)
Predicted Remaining HC = |E – HitCnt|
A 4 Value= Predicted Remaining HC – RRPV
HHT
... ...

D 2
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 10

Example
RRPV

A B C D E

5 2 5 2 0 1 0 2 6 0

4  2  5  3 2  2  5  5 0  1  0  1 2  2  0  0 7  0  6  1

Tag A C D E
Minimum HHT
Value EHC 4 0 2 7
Evict B
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 11

Why baseline?

• HHT cannot predict:

• New incoming blocks (compulsory misses)

• Old blocks without any entry in HHT

• LRU, SRRIP or DRRIP as baseline

• DRRIP has low area and good performance

 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 12

EHC: Victim Selection (cntd.)

Evict/Bypass (Exclusive cache) block with the lowest

value

Replace the Block

Block with Minimum
Value
Bypass the Block
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 13

Methodology

● CRC framework based on CMPsim

● Single-core and four-core
● Core parameters
○ 6-stage pipeline, 256-entry ROB
○ L1 (I&D): 32 KB, 8-way
○ Private L2: 256 KB, 8-way
○ Shared LLC: 2MB per core, 16-way

● Benchmarks
o SPEC CPU2006
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 14

Prior Replacement Policies

• DRRIP
• Assigns re-reference interval prediction value (RRPV) for
each block
• Evicts block with maximum RRPV
• SHiP
• Classifies blocks into two categories
• Good Blocks and Bad Blocks
• Enhances DRRIP by predicting dead-on-arrival blocks
• EVA
• Reconciles hit probability and expected lifetime by measuring
time in cache as forgone hits
• Evicts candidate with lowest EVA
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 15

Trace Results

DRRIP SHiP EVA EHC

30%
up to on average
20% 24% 11%
MPKI reduction over LRU

10%

0%
bzip cactusADM mcf xalanc Average
-10%

-20%

-30%
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 16

Cycle-Accurate Simulation
DRRIP SHiP EVA EHC
Performance normailzed to LRU

1.2
No clue to
determine the Binary
1.15 dead blocks classification

1.1

1.05

0.95
cactusADM mcf xalanc Average Mix1 MIX2 Average
Single-Core Multi-Core
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 17

Hardware overhead

• 3-bit RRPV per block (baseline overhead)

• 3-bit hit count per block
• 2K entries in HHT
• Each HHT entry:
• 20-bit tag,
• 2×3 bit hit-count,
• 4-bit LRU recency,
• A valid bit
 25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 18

Hardware optimization

HHT with 1K entries performs well too

1K-entry 2K-entry
1.15
IPC normalized to LRU

1.1
1.05
1
0.95
0.9
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 19

Conclusions

EHC:
• Low-cost-yet-effective replacement policy
• Evicts block that predicted to have the
minimum expected remaining hit count
• Cache-like structure for HHT
• 31.75KB area overhead for a 2MB cache
• 19.75KB area overhead over baseline (DRRIP)
• 3.4% Performance improvement over baseline
25 June 2017 Cache Replacement Policy Based on Expected Hit Count CRC-2 20

Thank you
for
your attention!