reveal.js

# Pipeline Hazards

---

CS 130 // 2021-12-06

## Administrivia
- Quiz today (online, untimed, on Gradescope)

# Questions
## ...about anything?

# Pipelining

## Pipelined Datapath

## Pipelined Datapath
- Five stages:
    1. **IF**: Instruction Fetch
    2. **ID**: Instruction Decode
    3. **EX**: Execute
    4. **MEM**: Memory access
    5. **WB**: Write back

# Hazards

## Hazards
- Situations that prevent immediately executing the next instruction in the pipeline are called **hazards**
- Three types of hazards:
    1. Structure hazards
    2. Data hazards
    3. Control hazards

## Structure Hazards
- A **structure hazard** occurs when a required resource is busy performing another task
- 
 Our MIPS datapath is designed such that no structure hazards are a problem
    + Read/write to register file can happen in same cycle

## Data Hazards
- A **data hazard** occurs when one instruction depends on the result of a previous instruction
- For example:
    ```mips
    add $s0, $t0, $t1
    sub $t2, $s0, $t3
    ```

## Data Hazards
- Would need to stall for two clock cycles in order to wait for the value $s0 to be available for reading

![Data Hazard Stall](/teaching/2021f/cs130/assets/images/COD/data_hazard_stall.png)

## Forwarding (aka Bypassing)
- One way to avoid some data hazards without stalling is to **forward** the result to the next instruction immediately when it is available

![Data Hazard Forwarding](/teaching/2021f/cs130/assets/images/COD/data_hazard_forwarding.png)

## Data Hazard Exercise
- Consider the following MIPS code:
    ```mips
    lw $t0, 40($a3)
    add $t6, $t0, $t2
    sw $t6, 40($a3)
    ```
- 
 Assuming there is no forwarding implemented, are any stalls necessary?
- 
 How many clock cycles are required to execute these three lines of code without forwarding?

## Data Hazard Exercise
- Consider the following MIPS code:
    ```mips
    lw $t0, 40($a3)
    add $t6, $t0, $t2
    sw $t6, 40($a3)
    ```
- Assuming there IS forwarding implemented, are any stalls necessary?
- 
 How many clock cycles are required to execute these three lines of code with forwarding?

## Rearranging Instructions
- Another way to avoid data hazards is by **rearranging instructions**
- Consider the following MIPS code:
    ```mips
    lw  $t1, 0($t0)
    lw  $t2, 4($t0)
    add $t3, $t1, $t2
    sw  $t3, 12($t0)
    lw  $t4, 8($t0)
    add $t5, $t1, $t4
    sw  $t5, 16($t0)
    ```
- 
 Identify any stalls that are necessary even with forwarding and hazard detection active

## Rearranging Instructions
- These two stalls could be avoided by rearranging the code in the following way:
    ```mips
    lw  $t1, 0($t0)
    lw  $t2, 4($t0)
    lw  $t4, 8($t0)
    add $t3, $t1, $t2
    sw  $t3, 12($t0)
    add $t5, $t1, $t4
    sw  $t5, 16($t0)
    ```

# Control Hazards

## Control Hazards
- A **control hazard** (aka branching hazard) is when the next instruction to be executed is not yet known
- 
 Caused by **branching** instructions such as `beq`
- 
 During a `beq` instruction, at what pipeline stage do we know which branch will be taken?
    + 
 After Stage 3: EX

## Control Hazards
- One way to avoid control hazards is by stalling
    ![Stall on Branch](/teaching/2021f/cs130/assets/images/COD/stall_on_branch.png)
- 
 After every branch statement, we stall for one cycle

## Control Hazards
- The pros of the "always stall" approach are:
    1. 
 Simple and easy to understand
    2. 
 Will always work
- 
 The con of "always stall" is:
    + It is slow

## Control Hazards
- An alternative to the always stall approach is **branch prediction**
    + Make an educated guess on what the next instruction will be and execute that
    + 
 If incorrectly guessed, "undo" the steps and go to the correct branch

## Control Hazards
- **Static branch prediction** will always predict a certain branch depending on the branching behavior
    + 
 Predict forward branches not taken (conditionals)
    + 
 Predict backward branches taken (loops)
- 
 **Dynamic branch prediction** keeps track of how many times a branch is taken and updates its predictions based on history

# Pipelined Datapath Design

## Pipelined Datapath Design
<img src='/teaching/2021f/cs130/assets/images/COD/piplined_datapath.png' height='550'>

## Using Registers (Has Bug)
<img src='/teaching/2021f/cs130/assets/images/COD/pipelined_datapath_with_registers.png' height='500'>

## Using Registers (Bug Fixed)

# Pipelined Control

## Pipelined Control Simplified

![Pipeline Control Simplified](/teaching/2021f/cs130/assets/images/COD/pipelined_control_simplified.png)

## Pipelined Control Registers
- Control signals derived from instruction and passed through the relevant registers

![Pipeline Control Registers](/teaching/2021f/cs130/assets/images/COD/pipelined_control_registers.png)

## Pipelined Control Complete

# Data Hazards
## Forwarding VS Stalling

## Detecting Data Hazards
- Consider the following MIPS instructions:
    ```mips
    sub $2,  $1, $3
    and $12, $2, $5
    or  $13, $6, $2
    add $14, $2, $2
    sw  $15, 100($2)
    ```
- 
 How many data hazards are present?
- 
 All these hazards here can be resolved by forwarding
    + But how do we detect when we need to forward?

## Detecting Data Hazards
<img src='/teaching/2021f/cs130/assets/images/COD/data_hazard_deps.png' height='500'>

## Detecting Data Hazards
- Recall that each register has a number from 0 to 31
- 
 We refer to the two source registers as `Rs` and `Rt` and the destination register as `Rd`
- 
 Need to pass register numbers through pipeline
    + `ID/EX.RegRs`: Register number for `Rs` that is in the `ID/EX` pipeline register
- 
 ALU operand register numbers in `EX` stage are:
    + `ID/EX.RegRs` and `ID/EX.RegRt`

## Detecting Data Hazards
- Data hazards happen when:
    + 
 `EX/MEM.RegRd = ID/EX.RegRs`
    + 
  `EX/MEM.RegRd = ID/EX.RegRt`
    + 
  `MEM/WB.RegRd = ID/EX.RegRs`
    + 
  `MEM/WB.RegRd = ID/EX.RegRt`
- 
 But only when the forwarding instruction is writing
    + `EX/MEM.RegWrite` / `MEM/WB.RegWrite` must be asserted
- 
 And only if `Rd` for is not `$zero`
    + `EX/MEM.RegRd` / `MEM/WB.RegRd` is not 0

## Forwarding Paths

## Forwarding Conditions
- Whenever forwarding is needed, we must forward the result either to the `Rs` register or the `Rt` register
- 
 The EX phase must forward to `Rs` if:
    + `EX/MEM.RegWrite` is asserted
    + `EX/MEM.RegRd` is not `$zero`
    + `EX/MEM.RegRd = ID/EX.RegRs`}
- 
 Similarly, it must forward to `Rt` if:
    + `EX/MEM.RegWrite` is asserted
    + `EX/MEM.RegRd` is not `$zero`
    + `EX/MEM.RegRd = ID/EX.RegRt`

## Double Data Hazard
- Consider the following MIPS code:
    ```mips
    add $1,$1,$2
    add $1,$1,$3
    add $1,$1,$4
    ```
- 
 Both `EX` and `MEM` hazards occur
    + 
 Want to use the most recent `$1` value
- 
 Need to add MEM hazard condition:
    + Only forward if `EX` hazard condition isn't true

## Datapath with Forwarding

# Load-Use Data Hazard

## Load-Use Data Hazard

## Detecting Load-Use Hazards
- Check when using instruction is decoded in ID stage
- 
 ALU operand register numbers in ID stage are:
    + `IF/ID.RegRs`
    + `IF/ID.RegRt`
- 
 Load-use hazard occurs when
    + `ID/EX.MemRead` is asserted
    + One of the following is true:
        * `ID/EX.RegRt = IF/ID.RegRs`
        * `ID/EX.RegRt = IF/ID.RegRt`
- 
 If detected, we insert a stall

## How to Stall the Pipeline
- Have EX, MEM, and WB do a `nop` (no operation)
    + Can make control values zero to accomplish this
- 
 Prevent updating PC and IF/ID register
    + Causes the same instruction to be fetched and decoded a second time

## How to Stall the Pipeline
<img src='/teaching/2021f/cs130/assets/images/COD/pipeline_stall_diagram.png' height='500'>

## Datapath Stall Detection

## Performance
- Stalls reduce performance but are necessary to avoid errors
- 
 In practice, the compiler rearranges instructions to avoid as many stalls as possible
    + Requires detailed knowledge of pipeline implementation
- 
 When trying to optimize your high-level code, it is better to trust the compiler to optimize these tasks rather than trying to be clever

# Control Hazards

## Control Hazards
- A **control hazard** (aka branching hazard) is when the next instruction to be executed is not yet known
- Caused by **branching** instructions such as `beq`

## Control Hazards
- **Static branch prediction** will always predict a certain branch depending on its type
    + 
 Predict forward branches not taken (conditionals)
    + 
 Predict backward branches taken (loops)
- 
 **Dynamic branch prediction** keeps track of how many times a branch is taken and updates its predictions based on history
    + 
 Modern dynamic branch prediction guesses correctly over 90% of the time