Home / Predictive Modeling & Machine Learning / 203.4.1 Model Section and Cross Validation

203.4.1 Model Section and Cross Validation

Model Validation Metrics

Model Validation

  • Checking how good is our model
  • It is very important to report the accuracy of the model along with the final model
  • The model validation in regression is done through R square and Adj R-Square
  • Logistic Regression, Decision tree and other classification techniques have very similar validation measures.
  • Till now we have seen confusion matrix and accuracy. There are many more validation and model accuracy metrics for classification models

Classification-Validation measures

  • Confusion matrix, Specificity, Sensitivity
  • ROC, AUC
  • KS, Gini
  • Concordance and discordance
  • Chi-Square, Hosmer and Lemeshow Goodness-of-Fit Test
  • Lift curve

All of them are measuring the model accuracy only. Some metrics work really well for certain class of problems. Confusion matrix, ROC and AUC will be sufficient for most of the business problems

Sensitivity and Specificity

Sensitivity and Specificity are derived from confusion matrix

  • Accuracy=(TP+TN)/(TP+FP+FN+TN)
  • Misclassification Rate=(FP+FN)/(TP+FP+FN+TN)
  • Sensitivity : Percentage of positives that are successfully classified as positive
  • Specificity : Percentage of negatives that are successfully classified as negatives

About admin

Check Also

204.4.12 Bootstrap Cross Validation

This will be our last post of our Model Selection and Cross Validation Series. Bootstrap …

Leave a Reply

Your email address will not be published. Required fields are marked *