Home / Predictive Modeling & Machine Learning / 203.6.1 Introduction to SVM

# 203.6.1 Introduction to SVM

The first post is about basics of classification and slowly we will lean into SVM with full throttle.

Support Vector Machines

Contents

• Introduction
• The decision boundary with largest margin
• SVM- The large margin classifier
• SVM algorithm
• The kernel trick
• Building SVM model
• Conclusion

### Introduction

• SVM is another black box method in Machine Learning space
• Compared to other ml algorithms, SVM totally a different approach to learning.
• The in-depth theory and mathematics of SVM needs great knowledge in vector algebra and numerical analysis
• We will try to learn the basic principal, philosophy, implementation of SVM
• SVM was first introduced by Vapnik and Chervonenkis
• Neural networks try to reduce the squared error and often suffer from overfitting.
• SVM algorithm has better generalization ability. There are many applications where SVM works better than neural networks

### The Classifier

• To understand the SVM algorithm easily, we will start with the decision boundary
• The line or margin that separates the classes
• Classification algorithms are all about finding the decision boundaries
• A good classifier is the one that generalizes well. It should work well on both training and testing data
• It need not be a straight line always

### The Margin of Classifier

Out of all the classifiers, the one that has maximum margin will generalize well. But why?

### The Best Decision Boundary

• Imagine two more data points. The classifier with maximum margin will be able to classify them more accurately.

### The Maximum Margin Classifier

• So, the best classifier has maximum margin
• The classifier that maximizes the distance between itself and the nearest training data
• In our example a,b,c are the training data points that are near to m1, and a,c,d are the training examples that are near to model m2.
• The model m1 has maximum margin
• The model m1 works well with the unseen examples
• The model m1 does good generalization
• For a given dataset, if we can find a classifier that has maximum margin, then it will assure maximum accuracy.

## 204.6.9 Digit Recognition using SVM

In this final post of this series we will put SVM into practice by solving …