Home / Python / Predictive Modeling & Machine Learning / 204.7.3 Types of Ensemble Models

204.7.3 Types of Ensemble Models

In this short post we will just see the types of Ensemble models.

Types of Ensemble Models

  • The above example is a very primitive type of ensemble model. There are better and statistically stronger ensemble methods that will yield better results
  • Two most popular ensemble methodologies are
    • Bagging
    • Boosting

Bagging

  • Take multiple boot strap samples from the population and build classifiers on each of the samples. For prediction take mean or mode of all the individual model predictions.
  • Bagging has two major parts 1) Boot strap sampling 2) Aggregation of learners
  • Bagging = Bootstrap Aggregating
  • In Bagging we combine many unstable models to produce a stable model. Hence the predictors will be very reliable(less variance in the final model).

Boot strapping

  • We have a training data is of size N
  • Draw random sample with replacement of size N – This gives a new dataset, it might have repeated observations, some observations might not have even appeared once.
  • We are selecting records one-at-a-time, returning each selected record back in the population, giving it a chance to be selected again
  • Create B such new datasets. These are called boot strap datasets

About admin

Check Also

204.7.6 Practice : Random Forest

Let’s implement the concept of Random Forest into practice using Python. Practice : Random Forest …

Leave a Reply

Your email address will not be published. Required fields are marked *