Bias and Variance in Machine Learning

Bias and Variance in Machine Learning

The key to success is finding the balance between bias and variance.

In predictive analytics, we build machine learning models to make predictions on new, previously unseen samples. The whole purpose is to be able to predict the unknown. But the models cannot just make predictions out of the blue. We show some samples to the model and train it. Then we expect the model to make predictions on samples from the same distribution.

There is no such thing as a perfect model so the model we build and train will have errors. There will be differences between the predictions and the actual values. The performance of a model is inversely proportional to the difference between the actual values and the predictions. The smaller the difference, the better the model. Our goal is to try to minimize the error. We cannot eliminate the error but we can reduce it. The part of the error that can be reduced has two components: Bias and Variance .

The performance of a model depends on the balance between bias and variance. The optimum model lays somewhere in between bias and variance. Please note that there is always a trade-off between bias and variance. The challenge is to find the right balance.