COURSE NOTES: ECE8405 COMPUTER ARCHITECTURE

Spring 2004

Ed Char

1. INTRODUCTION Summary: Hierarchy as a way of controlling complexity, overview. Architecture is at the heart of computer engineering Computers are driving technology, the economy, entertainment and society. Advances are coming at a breakneck pace: Moores Law: processors and memory are doubling performance every 1.5 years! (Due to technology AND organization). Most nontrivial digital systems are based on computers. The computer has become the most complicated (centralized) humanmade object, with hundreds of millions of identifiable parts. How can such a complicated system work reliably? By Hierarchical organization (object-oriented design) makes it possible Many different systems have to come together to allow not only functionality, but high PERFORMANCE metrics. Central is the instruction set- the key component of the hardwaresoftware interface. This is the set of instructions implemented by the processor, and the bit patterns that encode them. Lets consider the major component categories of this hierarchy: Networking Operating Systems High-level languages and compilers Assembly-language and assemblers, linkers, loaders | INSTRUCTION SET | Architecture (registers, datapath and ALU) Systems design (CPU, memory, I/O, exceptions) Gate-level design, simulation VLSI design and validation not to mention Performance evaluation Chip and system packaging Clock speeds, CPU size, etc
2

SOFTWARE

HARDWARE

ECONOMICS

Standards define the interface between layers: Like in OOP, a public definition hides the complexity at a given layer e.g. Network protocols, HLL language syntax, O.S. calls, the instruction set of a microprocessor, inputs/outputs of a VLSI circuit module. A simple interface specification hides the underlying complexity. Allows designers at any level to work at a reasonable complexity. Permits design changes and migration to different platforms. Think of computer engineering from an object-oriented perspective To conclude this train of thought, lets consider the design of a new uP: The design of a new high-performance processor requires: Fast time to market to leapfrog competition Innovate using newest and fastest VLSI fabrication line New instruction set design, BUT Instruction-set choices influences performance at all other levels! (e.g. OS, HLL compilers may be inefficient if certain instructions not included, but hardware implementation of certain instructions may require large VLSI real-estate). SO ITS AN OPTIMIZATION PROBLEM AT ALL LEVELS! What actually happens is that the WHOLE PROJECT (instruction set, compilers, O.S., hardware) is SIMULATED over and over, until the best set of compromises are reached that give best performance at reasonable cost. Only then is the processor committed to silicon. Some processors have worked the first time they were integrated, others have been sold to the public with serious bugs. Major processors today: 80x86/Pentium/K6, PowerPC, DEC Alpha, MIPS, SPARC, HP

The five basic components of a computer Datapath (Chs. 4-6) Control (Chs. 5-6) Memory (Ch. 7) Input and Output (Ch 8) In more detail, what will we cover in this course? Performance metrics, because these allow us to determine whether one machine is better than another, or whether a design modification is worthwhile. Instruction set design principles and examples (MIPS RISC mainly) MIPS assembly language and mapping C onto MIPS ALU implementation concepts (integer and floating point) Block-diagram implementation of a basic MIPS processor Pipelining the implementation for better performance: Approaching one instruction per clock Superscalar implementation for better performance Issuing MORE than one instruction per clock! Memory hierarchy, including cache and virtual memory Multiprocessor systems (time permitting) Thus, we will look mainly at the hardware aspects of microprocessors, at a block diagram level. Why is this stuff important? You will learn how todays computers work. Performance analysis will allow you to make educated choices in purchasing systems Software design is helped by (and sometimes requires) an understanding of the underlying hardware concepts

2. Apples and Oranges: The importance of PERFORMANCE

Summary, performance is key. However, it is quite difficult to evaluate. We will present and critique various measures that are in use, both within and between architectures. Performance measures are key They influence numerous purchasing decisions every day! Can give a measure of usefulness to purchaserHow long will you have to wait for results? (finite element analysis) Within an architecture, can evaluate the effectiveness of design changes Textbooks would have you believe that people carefully compare the price/performance of different systems. Unlikely. Most users know what architecture they want (wintel, mac) Many then go for the cheapest systems they can buy, leading to problems with reliability, compatibility, bad monitor, inadequate cache memory, software incompatibilities, etc. Some go to a reputable dealer and pay more to get some assurances Very few actually shop around much! Why? .. Performance is difficult to characterize between different architectures Every user takes advantage of different capabilities (e.g. integer vs. floating point) Instructions different Superscalarity further muddies picture (how often are 2/4 instructions actually issued?)

Consider a first-order model of a processor:

CONTROL

DATAPATH I/O

MEMORY SYSTEM

CLOCK (Clock in cycles per second = Hertz, or cycle interval time=1/Hz in secs) Key Features: Control determines path of data through system Instructions decoded by control CLOCK sets rate at which instructions are: 1. Fetched 2. Decoded 3. Executed How about clock rate as a performance measure? One instruction could take several clock cycles Different machines have different instructions- one machine may require half as many instructions to execute a program than another DRAM reads may take many clocks, so cache memory performance is very important Newer CPUs can issue multiple instructions each clock cycle! WITHIN an architecture, OK to use clock rate as performance measure: Performance(80MHz)/Performance(60Mhz) = 80 / 60 = 1.21 So we say that the 80 Mhz system is 21% (or 1.21 times) faster than the 60 MHz system. We cant say that one system is XXX% slower than another.
6

Another measure is CPI: average clocks per instruction. Can be determined by simulation or timing program execution OK within family and using same program: CPI = ExecutionTimeOfProgram / NumberInstructionsInProgram Perf(A)/Perf(B)=ExecutionTime(B)/ExecutionTime(A) = CPI(B)/CPI(A) Which program should one execute? Somewhat poor comparison between families, although most instruction sets are becoming more similar (RISC) How about MIPS: millions of instructions per second? Combines clock rate and CPI: PEAK MIPS is the fastest rate obtainable, its not measured MIPS Relative MIPS = ExecuteTimeref x MIPSref / ExecuteTimetest Works only for reference program and input So again, depends on program (or theory, for peak MIPS) You, as user, want the machine to respond as fast as possible, so lets define performance in terms of execution time of some program on machine X: Performance(X) = 1/ExecutionTime(X) Then we can compare two different machines X and Y using that program: Performance(X)/Performance(Y)=ExecTime(Y)/ExecTime(X)=n So X is n times faster than Y (if Y is faster switch X and Y). Which program do we use? You can measure performance using the program (or algorithm type) which YOU will run, or use an industrystandard program MIX that exercises different machine functions (integer, float, memory access, I/O, etc). Well consider benchmarks shortly. How can we measure execution time? On UNIX machines, its possible to obtain an estimate using the time command: Time someCommand args Returns: 90.7u 12,9s 2:39 65% First val is secs executing user (application) code Second val is secs executing (operating) system code Third value is total elapsed time as seen by user Fourth value is % of elapsed time that program was executed
7

Lets consider CPU execution time only (user + sys). Then for a program: CPUexecTime = CPUclockCycles x clockCycleTime CPUclockCycles = InstructsInProgram x AvgClockCyclesPerInstruct Can improve performance by decreasing clock cycle time (faster clock rate), or reducing the number of clock cycles required to execute program. The latter can be addressed by: More powerful instructions (maybe) gives fewer instructions Fewer clocks per instruction more efficient instruction execution More instructions running in parallel Faster memory access (many cycles wasted waiting for memory) NOTE: This is often ignored! Better compiler issues fewer instructions More efficient O.S. executes fewer instructions per system call .. and so on! This gives a hint of where we are going in this course. We will study the techniques that are being used in todays computers to improve performance. EXAMPLE: We are designing an architecture, and find that in order to reduce the average clocks per instruction from 3.5 to 2.7 (due to an improvement in the ALU), we have to decrease the clock speed by 25%. Is this worth it? CPUtime = InstructionCount x CPI x ClockCycleTime So, divide original by modified: CPUtime(o)/CPUtime(m) = 3.5 x 1.0 / (2.7 x 1.25) = 1.037 So the modified version is almost 4% faster than the original. BENCHMARK SUITES a group of programs used to gauge performance To use a BENCHMARK suite of programs to more fairly allow comparison between different architectures, we need to consider: 1. Which programs should we use, and 2. How can we combine them into one performance measure? Unfortunately, neither of these questions is trivial!
8

Which programs should we use? Every user uses a different WORKLOAD (i.e. set of programs run) Different programs may have very different performance measures Want to exercise different capabilities of computer in one (or two) benchmark(s) BEST: you evaluate systems using the exact mix of programs you will use on your computer! How should they be combined? Arithmentic mean- average all execution times on a machine AM = 1/n TimeI Problem: longer-running programs wash out more fleeting progs. Geometric mean- independent of running time, each program has same weight GM = nth root of ( Execution Time Ratioi ) Where the Execution time ration is ExecTimetest / ExecTimeref The ExecTime ratio of the computer under test to a reference machine Neither is intrinsically better: depends what you want from your benchmark! Example: Test: Ref: App1 = 20 secs App1 = 100 secs App2 = 240 secs App2 = 120 secs ref=110sec -> ref machine is 18% faster

Arithmetic mean: Test=130 sec

Geometric mean: Test = sq. root(0.2 x 2.0)= 0.632 Ref = sq. root(5.0 x 0.5)= 1.58 -> test machine is 2.5 times faster!! BOTTOM LINE: Be careful how you slice your performance evaluation data.

Amdahls Law
Expecting the improvement of one aspect of a machine to increase performance by an amount proportional to the size of the improvement is wrong. For example: Suppose a program runs in 100 seconds on a machine,
9

with multiply instructions responsible for 80 seconds of this time. How much faster do I have to improve the speed of multiplication if I want my program to run 5 times faster? So the execution time is equal to the speed of the multiplications plus the other parts: Exec time after improvement = 80/n + (100 80 seconds) n is what we need to divide by to get our performance: Since we want the execution to be 5 times faster: 20 = 80/n +20 0 = 80/n which doesnt work. You need to work on others sections as well to get a five-fold increase in performance. Example Problems: 1) (2.3) If the clock rates of machines M1 and M2 are 200 MHz and 300 MHz, find the clock cycles per instruction for program 1 given the following: Program Time on M1 Time on M2 1 10 sec 5 sec 2 3 sec 4 sec Program 1 Instructions on M1 200x106 Instructions on M2 160x106

Solution: CPI = Cycles per second / Instructions per second. So M1 = 200x106 / (200x106 / 10 sec) = 10 cycles per instruction M2 = 160x106 / (160x106 / 5 sec) = 9.4 cycles per instruction 2) (2.10) Consider 2 different implementations, M1 and M2 of the same instruction set. There are four classes of instructions (A, B, C and D) in the instruction set. M1 has a clock rate of 500MHz. The average number of cycles for each instruction class on M1 is: Class CPI for this class A 1 B 2
10

C D

3 4

M2 has a clock rate of 750 MHz. The average number of cycles for each instruction class on M2 is: Class CPI for this class A 2 B 2 C 4 D 4 Assume that peak performance is defined as the fastest rate that a machine can execute an instruction sequence chosen to maximize that rate. What are the peak performances of M1 and M2 expressed as instructions per second? Solution: For M1 the peak performance will be achieved with a sequence on instructions of class A, which have a CPI of 1. The peak performance is thus 500 MIPS. For M2 , a mixture of A and B instructions both of which have a CPI of 2, will achieve peak performance which is 375 MIPS. 3) (2.13) Consider 2 different implementations, M1 and M2, of the same instruction set. There are three classes of instructions (A, B and C) in the instruction set. M1 has a clock rate of 400 MHz and M2 has a clock rate of 200 MHz. The average number of cycles for each instruction set is given in the following: Class A B C CPI M1 4 6 8 CPI M2 2 4 3 C1 Usage 30% 50% 20% C2 Usage 30% 20% 50% 3rd party 50% 30% 20%

The table also contains information of how the three different compilers use the instruction set. C1 is a compiler produced by the makers of M1, C2 is produced by the makers of M2 and the other is a third party compiler. Assume that each compiler uses the same number of instructions for a given program but that the instruction mix is as described in the table.
11

Using C1 on both M1 and M2 how much faster can the makers of M1 claim that M1 is over M2? Using C2 how much faster is M2 over M1? If you purchase M1 which compiler should you use? For M2? Solution: Using C1, the CPI on M1 = 4 * .3 + 6 * .5 + 8 * .2 = 5.8 M2 = 2 * .3 + 4 * .5 + 3* .2 = 3.2 So assuming that M1 is faster we have: (3.2/200E6) / (5.8/400E6) = 1.10 or M1 is 10% faster than M2 using C1 Using C2, the CPI on M1 = 4*.3 + 6*.2 + 8*.5 = 6.4 M2 = 2 * .3 + 4 * .2 + 3* .5 = 2.9 Assuming that M2 is faster we have (6.4/400E6) / (2.9/200E6) = 1.10 or now M2 is 10% faster than M1 using C2. For the 3rd party compiler: M1 = 4*.5 + 6*.3 + 8*.2 = 5.4 M2 = 2 * .5 + 4 * .3 + 3* .2 = 2.8 rd This tells us that the 3 party compiler is better for both machines since the CPI is lower. If we try M2 as the faster machine we get: (5.4/400E6) / (2.8/200E6) = .964 So M1 is the faster machine. How much faster? Reverse the equation: (2.8/200E6) / (5.4/400E6) = 1.037 or M1 is 3.7% faster than M2. 4) (2.15) Consider a program P, with the following mix of operations: floating-point multiply 10% floating-point add 15% floating-point divide 5% integer instructions 70% Machine MFP has floating point hardware and can implement the floating point operations directly. It requires the following number of clock cycles for each instruction class: floating-point multiply 6 floating-point add 4 floating-point divide 20 integer instructions 2 Machine MNFP has no floating point hardware and so must emulate the floating-point operations using integer instructions. The integer instructions

12

take 2 clock cycles. The number of integer instructions needed to implement each of the floating point operations is: floating-point multiply 30 floating-point add 20 floating-point divide 50 Both machines have a clock rate of 1000 MHz. Find the native MIPS rating for both machines. Solution: MIPS = Clock Rate / CPI x106 CPI for MFP = .1*6 + .15*4 + 0.05*20 + .7*2 = 3.6 CPI of MNFP = 2. MIPS for MFP = 1000/CPI = 278 MIPS for MNFP = 1000/CPI = 500 5) (2.16) If the machine in MFP in the last exercise needs 300 million instructions for a program, how many integer instructions does the machine MNFP require for the same program? Solution: This is really just a ratio type problem: INSTRUCTION Count on MFP in 106 floating-point multiply 30 floating-point add 45 floating-point divide 15 integer instructions 210 TOTALS 300

Count on MNFP in 106 900 900 750 210 2760

6) (2.26) The table below shows the number of floating point operations executed in two different programs and the runtime for those programs on three different machines: (times given in seconds) Program FP ops 1 10,000,000 2 100,000,000 Computer A 1 1000 Computer B 10 100 Computer C 20 20

Which machine is faster given total execution time? How much faster is it than the other two?

13

Solution: Total execution time for Computer A is 1001 seconds, Computer B is 110 seconds and C 40 seconds. So C is the fastest. It is 1001/40 = 25 times faster than A and 110/40 = 2.75 times faster than B.

Homework problems: 1) Using the information from Example 2 above on M1 and M2, If the number of instructions executed in a certain program is divided equally among the classes of instructions, how much faster is M2 over M1? (Hint: find CPI of each machine first) 2) (2.18) You are the lead designer of a new processor. The processor design and compiler are complete, and now you must decide whether to produce the current design as it stands or spend additional time to improve it. Your options are: a) Leave the design as it stands. Call this base machine Mbase. It has a clock rate of 500 MHz and the following measurements have been made using a simulator: Instruction Class A B C D CPI 2 3 3 5 Frequency 40% 25% 25% 10%

b) Optimize the hardware. The hardware team claims that it can improve the processor design to give it a clock rate of 600 MHz. Call this machine MOPT. The following measurements were made for MOPT: Instruction Class A B C D What is the CPI for each machine?
14

CPI 2 2 3 4

Frequency 40% 25% 25% 10%

3) (2.21) Using the material for Mbase and MOPT above, a compiler team proposes to improve the compiler for the machine to further enhance performance. Call this combination of the improved compiler and the base machine Mcomp. The instructions improvements from this enhanced compiler have been estimated as follows: Instruction Class % of instructions executed vs. base machine A 90% B 90% C 85% D 95% For example, if Mbase executed 500 Class A instructions, Mcomp would execute .9*500 = 450 instructions. What is the CPI for Mcomp? 4) (2.27) Given the material from Exercise 6 (problem 2.26) Which machine is fastest using Geometric mean? 5) (2.44) You are going to enhance a machine, and there are two possible improvements: either make multiply instructions run four times faster than before, or make memory access instructions run two times faster than before. You repeatedly run a program that takes 100 seconds to execute. Of this time, 20% is used for multiplications, 50% for memory instructions and 30% for other tasks. What will the speedup be if you improve only multiplication? What will the speedup be if you improve only memory access? What will the speedup be if both improvements are made?

15