CS 152 Computer Architecture and Engineering

Transcription

1 CS 152 Computer rchitecture and Engineering Lecture 8 Pipelining II John Lazzaro ( Ts: Udam Saini and Jue Sun www-inst.eecs.berkeley.edu/~cs152/

2 + Last Time: Introduction to Pipelining 1 2 IF Stage Instr Fetch ID/RF Stage Decode & Reg Fetch 3 EX Stage Execution 4 E Stage emory 5 WB Write Back, emtoreg ux,logic op D PC Q 0x4 ddr Instr em Data RegFile rs1 rs2 rd1 ws rd2 wd L U Y Data emory ddr Dout Din emtoreg R Ext B Welcome to Lab 3! CS 152 L7: Pipelining I

3 From First Class: The rchitect s Contract To the program, it appears that instructions execute in the correct order defined by the IS. s each instruction completes, the machine state (regs, mem) appears to the program to obey the IS. What the machine actually does is up to the hardware designers, as long as the contract is kept. The primary challenge of 152 CPU projects!

4 Last Time: Performance and Hazards + Seconds Program Instructions Program Cycles Instruction Seconds Cycle D PC Instr Fetch Decode & Reg Fetch Stage #3 Q 0x4 ddr Instr em Data Some ways to cope with hazards makes CPI > 1 stalling pipeline rs1 rs2 ws wd RegFile rd1 rd2 Ext B dded logic to detect and resolve hazards increases clock period Software slows the machine down Seymour Cray CS 152 L7: Pipelining I

5 Today: Hazards Visualizing pipelines to evaluate hazard detection and resolution. taxonomy of pipeline hazards. tool kit for hazard resolution. Tuesday: We apply this knowledge to design a pipelined IPS CPU that obeys the contract with the programmer.

6 Reminder: Do the Reading! The book presentation of pipelined processors is sufficient to do Lab 3. These lectures are not. The lectures are a gentle introduction, to prepare you to read the book...

7 Visualizing Pipelines

8 Pipeline Representation #1: Timeline IF (Fetch) ID (Decode) EX (LU) E WB 0x4 + PC Instr em Good for visualizing pipeline fills. D Q ddr Data Sample Program I1: I2: I3: I4: I5: DD R4,R3,R2 ND R6,R5,R4 SUB R1,R9,R8 XOR R3,R2,R1 OR R7,R6,R5 Time: Inst I1: I2: I3: I4: I5: I6: t1 t2 t3 t4 t5 t6 t7 t8 IF ID IF EX ID IF Pipeline is full E EX ID IF WB E EX ID IF WB E EX ID IF WB E EX ID WB E EX

9 Representation #2: Resource Usage + IF (Fetch) ID (Decode) EX (LU) E WB 0x4 PC Instr em Good for visualizing pipeline stalls. D Q ddr Data Sample Program I1: I2: I3: I4: I5: DD R4,R3,R2 ND R6,R5,R4 SUB R1,R9,R8 XOR R3,R2,R1 OR R7,R6,R5 Time: Stage IF: ID: EX: E: WB: t1 t2 t3 t4 t5 t6 t7 t8 I1 I2 I1 I3 I2 I1 Pipeline is full I4 I3 I2 I1 I5 I4 I3 I2 I1 I6 I5 I4 I3 I2 I7 I6 I5 I4 I3 I8 I7 I6 I5 I4

10 Hazard Taxonomy

11 Structural Hazards Several pipeline stages need to use the same hardware resource at the same time. Solution #1: dd extra copies of the resource (only works sometime). Solution #2: Change resource so that it can handle concurrent use. Solution #3: Stages take turns by stalling parts of the pipeline.

12 HW 1: Structural Hazard (Single emory) IF Stage ID/RF Stage EX Stage E Stage WB HW uses Solution 3 (stalling pipeline) ux,logic, emtoreg op PC Data emory ddr Dout Din RegFile rs1 rs2 rd1 ws rd2 wd L U To branch logic Y emtoreg R Ext B

13 + Lab 2/3 solution: Extra copies of memory 1 2 IF Stage Instr Fetch ID/RF Stage Decode & Reg Fetch 3 EX Stage Execution 4 E Stage emory 5 WB Write Back, emtoreg ux,logic op D PC Q 0x4 ddr Instr em Data RegFile rs1 rs2 rd1 ws rd2 wd L U Y Data emory ddr Dout Din emtoreg R Ext B I and D caches (Final Project) are a hybrid solution

14 + Solution #2 : Concurrent use IF Stage Instr Fetch ID/RF Stage Decode & Reg Fetch 3 EX Stage Execution 4 E Stage emory 5 WB Write Back, emtoreg ux,logic op D PC Q 0x4 ddr Instr em Data RegFile rs1 rs2 rd1 ws rd2 wd L U Y Data emory ddr Dout Din emtoreg R Ext B ID and WB stages use register file in same clock cycle

15 Data Hazards: 3 Types (RW, WR, WW) Several pipeline stages read or write the same data location in an incompatible way. Read fter Write (RW) hazards. Instruction I2 expects to read a data value written by an earlier instruction, but I2 executes too early and reads the wrong copy of the data. Note data value, not register. Data hazards are possible for any architected state (such as main memory). In practice, main memory hazard avoidance is the job of the memory system.

16 Recall from last lecture: RW example Stage #1 Stage #2 Stage #3 Instr Fetch Decode & Reg Fetch Sample program DD R4,R3,R2 OR R5,R4,R2 + D PC Q 0x4 ddr Instr em Data OR R5,R4,R2... wrong value of R4 fetched from RegFile, contract with programmer broken! Oops! rs1 rs2 ws wd RegFile rd1 rd2 Ext B DD R4,R3,R2 R4 not written yet... This is what we mean when we say Read fter Write (RW) Hazard

17 Data Hazards: 3 Types (RW, WR, WW) Write fter Read (WR) hazards. Instruction I2 expects to write over a data value after an earlier instruction I1 reads it. But instead, I2 writes too early, and I1 sees the new value. Write fter Write (WW) hazards. Instruction I2 writes over data an earlier instruction I1 also writes. But instead, I1 writes after I2, and the final data value is incorrect. WR and WW not possible in our 5-stage pipeline. But are possible in other pipeline designs.

18 Control Hazards: taken branch/jump + IF (Fetch) ID (Decode) EX (LU) E WB 0x4 D PC Q ddr Instr em Data Note: with branch delay slot, I2 UST complete, I3 UST NOT complete. Sample Program Time: t1 t2 t3 t4 t5 t6 t7 t8 (IS w/o branch Inst EX stage delay slot) I1: IF ID EX E WB computes I2: IF ID if branch I1: BEQ R4,R3,25 I3: IF is taken I2: ND R6,R5,R4 I4: I3: SUB R1,R9,R8 If branch is taken, I5: these instructions I6: UST NOT complete!

19 Hazards Recap Structural Hazards Data Hazards (RW, WR, WW) Control Hazards (taken branches and jumps) On each clock cycle, we must detect the presence of all of these hazards, and resolve them before they break the contract with the programmer.

20 dministrivia: Upcoming deadlines... Friday 9/22: Xilinx Checkoff, in section. onday 9/25: Lab 2 final report due via the submit program, 11:59 P. Lab 3 now available on the web site Thursday 9/28: t 11:59 P via Lab 2 peer evaluations, and Lab 3 preliminary design document due.

21 Updated Office Hours Udam: W 2-3 P, 125 Cory Jue: T4-5 P, Th 3-4 P, 125 Cory John: TTh 10-11, 315 Soda

22 Crunch Week: Homework, idterm, Lab Graded on effort Thursday review session. Will cover format, material, and ground rules for test. Preliminary design document idterm two weeks from today, in evening, no class that day. CS 152 L7: Pipelining I Lab 3 final design doc, checkoffs, later in week...

23 Hazard Resolution Tools

24 The Hazard Resolution Toolkit Stall earlier instructions in pipeline. Forward results computed in later pipeline stages to earlier stages. dd new hardware or rearrange hardware design to eliminate hazard. Change IS to eliminate hazard. Kill earlier instructions in pipeline. ake hardware handle concurrent requests to eliminate hazard.

25 Resolving a RW hazard by stalling Stage #1 Stage #2 Stage #3 Instr Fetch Decode & Reg Fetch Sample program DD R4,R3,R2 OR R5,R4,R2 + D PC Q 0x4 ddr Instr em Data OR R5,R4,R2 Keep executing OR instruction until R4 is ready. Until then, send NOPS to 2/3. rs1 rs2 ws wd RegFile rd1 rd2 DD R4,R3,R2 Let DD proceed to WB stage, so that R4 is written to regfile. New datapath hardware (1) ux into 2/3 to feed in NOP. Freeze PC and until stall is over. Ext B (2) Write enable on PC and 1/2

26 The Hazard Resolution Toolkit Stall earlier instructions in pipeline. Forward results computed in later pipeline stages to earlier stages. dd new hardware or rearrange hardware design to eliminate hazard. Change IS to eliminate hazard. Kill earlier instructions in pipeline. ake hardware handle concurrent requests to eliminate hazard.

27 Resolving a RW hazard by forwarding + IF Stage Instr Fetch Sample program DD R4,R3,R2 OR R5,R4,R2 0x4 1 2 ID/RF Stage Decode & Reg Fetch OR R5,R4,R2 Just forward it back! EX Stage Execution op L U 3 DD R4,R3,R2 LU computes R4 in the EX stage, so... Y RegFile D PC Q ddr Instr em Data rs1 rs2 ws wd rd1 rd2 Ext B Unlike stalling, does not change CPI. ay hurt cycle time.

28 The Hazard Resolution Toolkit Stall earlier instructions in pipeline. Forward results computed in later pipeline stages to earlier stages. dd new hardware or rearrange hardware design to eliminate hazard. Change IS to eliminate hazard. Kill earlier instructions in pipeline. ake hardware handle concurrent requests to eliminate hazard.

29 Control Hazards: Fix with more hardware + IF (Fetch) ID (Decode) EX (LU) E WB 0x4 D PC Q ddr Instr em Data If we add hardware, can we move it here? Sample Program Time: t1 t2 t3 t4 t5 t6 t7 t8 (IS w/o branch Inst EX stage delay slot) I1: IF ID EX E WB computes I2: IF ID if branch I1: BEQ R4,R3,25 I3: IF is taken I2: ND R6,R5,R4 I4: I3: SUB R1,R9,R8 If branch is taken, I5: these instructions I6: UST NOT complete!

30 + Resolving control hazard with hardware Stage #1 Stage #2 Stage #3 Instr Fetch Decode & Reg Fetch To branch control logic == 0x4 RegFile D PC Q ddr Instr em Data rs1 rs2 ws wd rd1 rd2 Ext B

31 Control Hazards: fter more hardware + IF (Fetch) ID (Decode) EX (LU) E WB 0x4 D PC Q ddr Instr em Data If we change IS, can we always let I2 complete ( branch delay slot ) and eliminate the control hazard. Sample Program Time: t1 t2 t3 t4 t5 t6 t7 t8 (IS w/o branch Inst ID stage delay slot) I1: IF ID EX E WB computes I2: IF if branch I1: BEQ R4,R3,25 I3: is taken I2: ND R6,R5,R4 I4: I3: SUB R1,R9,R8 If branch is taken, this I5: instruction UST NOT I6: complete!

32 From Lecture 1: BEQ $1,$2,25 Instruction Fetch Instruction Decode Operand Fetch Execute Fetch branch inst from memory opcode rs rt offset I-Format Decode fields to get: BEQ $1, $2, 25 Retrieve register values: $1, $2 Compute if we take branch: $1 == $2? Result Store Next Instruction CS 152 L1: The IPS IS LWYS prepare to fetch instr that follows the BEQ in the program ( delayed branch ). IF we take branch, the instr we fetch FTER that instruction is PC PC == Program Counter

33 The Hazard Resolution Toolkit Stall earlier instructions in pipeline. Forward results computed in later pipeline stages to earlier stages. dd new hardware or rearrange hardware design to eliminate hazard. Change IS to eliminate hazard. Kill earlier instructions in pipeline. ake hardware handle concurrent requests to eliminate hazard.

34 Resolve control hazard by killing instr Stage #1 Stage #2 Stage #3 Instr Fetch Decode & Reg Fetch Sample program (no delay slot) J 200 OR R5,R4,R2 + D PC Q 0x4 ddr Instr em Data J 200 Detect J instruction, mux a NOP into 1/2 rs1 rs2 ws wd RegFile rd1 rd2 This hurts CPI. Can we do better? Compute new Ext PC using hardware not shown... B

35 The Hazard Resolution Toolkit Stall earlier instructions in pipeline. Forward results computed in later pipeline stages to earlier stages. dd new hardware or rearrange hardware design to eliminate hazard. Change IS to eliminate hazard. Kill earlier instructions in pipeline. ake hardware handle concurrent requests to eliminate hazard.

36 Structural hazard solution: concurrent use + D PC IF Stage Instr Fetch Does not come for free... Q 0x4 1 2 ddr Instr em Data ID/RF Stage Decode & Reg Fetch ux,logic rs1 rs2 ws wd RegFile rd1 rd2 EX Stage Execution 3 op L U Y, emtoreg Data emory ddr Din 4 E Stage emory Dout emtoreg R 5 WB Write Back Ext B ID and WB stages use register file in same clock cycle

37 Summary: Hazards Visualizing pipelines to evaluate hazard detection and resolution. taxonomy of pipeline hazards. tool kit for hazard resolution. Interesting question raised last term...

38 Write contract to match the hardware? + PC IF Stage Instr Fetch What if we left hazards to the compiler? 0x4 1 2 Instr em ID/RF Stage Decode & Reg Fetch ux,logic rs1 rs2 ws wd RegFile rd1 rd2 EX Stage Execution 3 op L U Y, emtoreg Data emory ddr Din 4 E Stage emory Dout emtoreg R 5 WB Write Back D Q ddr Data Ext B

39 Register RW hazards: No stalls or fwds. 1 2 IF Stage Instr Fetch ID/RF Stage Decode & Reg Fetch 3 EX Stage Execution 4 E Stage emory 5 WB Write Back How would the contract read? + PC 0x4 Instr em ux,logic RegFile rs1 rs2 rd1 ws rd2 wd op L U Y, emtoreg Data emory ddr R Dout Din emtoreg D Q ddr Data Ext B

40 Can you do forwarding in software? + IF Stage Instr Fetch Sample program DD R4,R3,R2 OR R5,R4,R2 0x4 1 2 ID/RF Stage Decode & Reg Fetch OR R5,R4,R2 Just forward it back! EX Stage Execution op L U 3 DD R4,R3,R2 LU computes R4 in the EX stage, so... Y RegFile D PC Q ddr Instr em Data rs1 rs2 ws wd rd1 rd2 Ext B Hint: Expose forwarding to the IS...

41 Yes! By exposing forwarding registers IF Stage ID/RF Stage Instr Fetch Decode & Reg Fetch 3 EX Stage Execution 4 E emory + Sample Program DD F,R3,R2 NOP OR R5,F,R2 NOP F DD F,R3,R2 op L U Clocked into forwarding register at end of cycle. Y RegFile 0x4 rs1 rs2 ws wd rd1 rd2 PC Instr em D Q ddr Data Ext B One cycle later...

42 One clock cycle later IF Stage ID/RF Stage Instr Fetch Decode & Reg Fetch 3 EX Stage Execution 4 E emory + Sample Program DD F,R3,R2 NOP OR R5,F,R2 OR R5,F,R2 F value left by DD... F op L U NOP Y RegFile 0x4 rs1 rs2 ws wd rd1 rd2 PC Instr em D Q ddr Data Ext B Trick used by RW project at IT...

43 Coming up next week... Tuesday: pplying hazard tools to a pipelined CPU design. Thursday: id-term review, HW 1 due in class.