Home / Predictive Modeling & Machine Learning / 203.7.3 Types of Ensemble Models

203.7.3 Types of Ensemble Models

In this short post we will just see the types of Ensemble models.

Types of Ensemble Models

  • The above example is a very primitive type of ensemble model. There are better and statistically stronger ensemble methods that will yield better results
  • Two most popular ensemble methodologies are
    • Bagging
    • Boosting


  • Take multiple boot strap samples from the population and build classifiers on each of the samples. For prediction take mean or mode of all the individual model predictions.
  • Bagging has two major parts 1) Boot strap sampling 2) Aggregation of learners
  • Bagging = Bootstrap Aggregating
  • In Bagging we combine many unstable models to produce a stable model. Hence the predictors will be very reliable(less variance in the final model).

Boot strapping

  • We have a training data is of size N
  • Draw random sample with replacement of size N – This gives a new dataset, it might have repeated observations, some observations might not have even appeared once.
  • We are selecting records one-at-a-time, returning each selected record back in the population, giving it a chance to be selected again
  • Create B such new datasets. These are called boot strap datasets

About admin

Check Also

204.7.9 Boosting Conclusion

When Ensemble doesn’t work? The models have to be independent, we can’t build the same …

Leave a Reply

Your email address will not be published. Required fields are marked *