PIPELINE CONTROL

Because of the nonlinear usage of the stages of an arithmetic pipeline, it becomes more difficult to determine when to initiate each successive task in order to obtain maximum performance without conflict. We wish to initiate new tasks at as high a rate as possible; but we must also ensure that a single stage will never be needed by two tasks at the same time. In some pipelines, more than one type of task may be scheduled at the same time. In this discussion we will assume that all tasks are identical. The problem of pipeline scheduling can be addressed by means of a scheduling diagram known as a reservation table. This technique will be briefly described in the following sections. Reservation Tables A reservation table is a two-dimensional table in which each row represents a pipeline stage (processing element) and each column represents a cycle in the execution of a task. Within each column, an X is placed in the entries corresponding to the stages required by that task during that cycle. Figure 1 shows a typical reservation table for a three-stage pipeline and a six-cycle task. This table shows that the task uses stage A for two cycles. During the second cycle, Stage B is also used; this stage is still used in cycle three. Cycle four uses stage C. During the fifth cycle, the task returns again to Stage A. Finally, Stage B is used in the last cycle.

Figure 1 -- A Reservation Table Forbidden Latencies Our goal is to determine how many cycles we must wait, after one task is initiated, until another task can be initiated without conflict. The number of cycles between initiations is called the latency. We want to design a schedule which has

the minimum average latency (MAL) for our pipeline. However, such a schedule may not have the same latency between each successive task. The first step is to identify the forbidden latencies revealed by the diagram. A latency is forbidden if it will lead to a collision, that is, a simultaneous need for the same stage by two different tasks. The table shows that stage A is required during both the first and second cycles. We cannot initiate a new task after only one cycle, or there would be an immediate collision. Suppose instead that we try to begin the second task three cycles after the first. There will be no collision during its initial cycle. During the second cycle for the new task, however, the original task will have reached cycle five, and both will require stage A. Fortunately, there is a systematic way to identify all forbidden latencies from a given reservation table. For each row containing more than one X, we write down the distance between every pair of X's. Each such distance represents a forbidden latency. In our example, row A contains three X's which can form three distinct pairs. The latencies blocked by these pairs are 1, 3, and 4. Row B yields an identical set of forbidden latencies. There are no forbidden latencies for row C, since it contains only one X. The set of forbidden latencies is then summarized in a bit string called a collision vector. The collision vector contains a bit for each possible latency from one up to the total length of the task. The bits are numbered from left to right (but some authors use the opposite order). The collision vector for our example table is 101100. State Transitions A collision vector may be thought of as a representation of the current state of the pipeline. Our example pipeline will be in state 101100 at the moment that the first task is initiated. This pattern tells us that we may not initiate another task after 1, 3, or 4 cycles, but we may initiate a new task after 2, 5, or 6 cycles. Of course, we may also initiate after any number of cycles greater than 6, since the original task will then be completed.

Suppose that we begin a second task after two cycles. Can we then wait another two cycles and begin a third task? Unfortunately, the answer is no. To see this, consider the time at which the second task has just been initiated. The first task still has four cycles to complete. We can represent this situation by a new reservation table, in

which the stages reserved by the first task are marked by 1's, and those reserved by the second task are marked by 2's.

Figure 2 -- Reservation Table for Two Tasks

The conflicts in this table are represented by the union of two collision vectors. The collision vector for the first task is the original vector with the first two positions shifted out to the left. Its form is 110000. The vector for the second task, just begun, has the original form. The collision vector formed by the union of these two is 111100. This is the state represented by the table in Figure 2. State Diagrams Using a similar analysis, the set of possible states for a given pipeline may be identified. The original collision vector shows allowable transitions for latencies of 2, 5, or 6. In general each of these will lead to a new state. However, in this example, the resulting states for latencies of 5 or 6 are identical to the original state. Latency 2, as we have seen, leads to a distinct state. Further transitions from this state can occur only at latencies of 5 or 6 (or greater). Each of these transitions produce additional states; in this case these are again identical to the original state.

By a systematic analysis of each possible transition, all possible states can be identified together with the full set of allowable latencies. Fortunately, the set of distinct states is usually small. A state diagram can be used to represent these states; the state diagram corresponding to our example is shown in Figure 3.

Figure 3 -- A State Diagram

Finding the MAL A sequence of allowable task initiations corresponds to a sequence of allowable transitions in the state diagram. By inspection we can find all cycles in the diagram; each cycle identifies a repeatable sequence of initiations. The cycle with the lowest value for all of the transitions it contains represents the initiation sequence with the minimum average latency. In our example, The desired cycle contains alternating latencies of 2 and 5, giving an average latency of 3.5. We will obtain maximum throughput for this task in this pipeline by waiting alternately two cycles followed by five cycles for successive initiations. In many cases, the cycle having the minimum average latency is a greedy cycle, that is, one that makes the earliest possible transition to each new state. However, there are cases in which a non-greedy strategy must be used to obtain the MAL. Implementation In the design of a pipeline which is dedicated to a single type of task with a fixed reservation pattern, the above analysis may be performed during the design stage. Simple sequencing logic is then developed to ensure that the correct schedule is followed.

Alternately, logic can be included in some pipelines to keep track of the current collision vector for the system and make scheduling decisions dynamically. This is most appropriate for pipelines that may accept more than one kind of task.