Home / Predictive Modeling & Machine Learning / 203.4.6 model-Bias Variance Tradeoff

### Model Bias and Variance

• Over fitting
• Low Bias with High Variance
• Low training error – ‘Low Bias’
• High testing error
• Unstable model – ‘High Variance’
• The coefficients of the model change with small changes in the data
• Under fitting
• High Bias with low Variance
• High training error – ‘high Bias’
• testing error almost equal to training error
• Stable model – ‘Low Variance’
• The coefficients of the model doesn’t change with small changes in the data

### The Bias-Variance Decomposition

$Y = f(X)+\epsilon$ $Var(\epsilon) = \sigma^2$ $Squared Error = E[(Y -\hat{f}(x_0))^2 | X = x_0 ]$ $= \sigma^2 + [E\hat{f}(x_0)-f(x_0)]^2 + E[\hat{f}(x_0)-E\hat{f}(x_0)]^2$ $= \sigma^2 + (Bias)^2(\hat{f}(x_0))+Var(\hat{f}(x_0 ))$

Overall Model Squared Error = Irreducible Error + $$Bias^2$$ + Variance

### Bias-Variance Decomposition

• Overall Model Squared Error = Irreducible Error + $$Bias^2$$ + Variance
• Overall error is made by bias and variance together
• High bias low variance, Low bias and high variance, both are bad for the overall accuracy of the model
• A good model need to have low bias and low variance or at least an optimal where both of them are jointly low
• How to choose such optimal model. How to choose that optimal model complexity