Home / Predictive Modeling & Machine Learning / 203.5.11 Hidden Layers and Their Roles

203.5.11 Hidden Layers and Their Roles

In this post we will understand one of the most important part of a Neural Network : Hidden Layers. We will be able to use this information to control the parameters related to hidden layers.

Hidden layers and their role

Multi Layer Neural Network

The Role of Hidden Layers

  • The First hidden layer
  • The first layer is nothing but the liner decision boundaries
  • The simple logistic regression line outputs
  • We can see them as multiple lines on the decision space
  • The Second hidden layer
  • The Second layer combines these lines and forms simple decision boundary shapes
  • The third hidden layer forms even complex shapes within the boundaries generated by second layer.
  • You can imagine All these layers together divide the whole objective space into multiple decision boundary shapes, the cases within the shape are class-1 outside the shape are class-2

The Number of Hidden Layers

  • There is no concrete rule to choose the right number. We need to choose by trial and error validation
  • Too few hidden layers might result in imperfect models. The error rate will be high
  • High number of hidden layers might lead to over-fitting, but it can be identified by using some validation techniques
  • The final number is based on the number of predictor variables, training data size and the complexity in the target.
  • When we are in doubt, its better to go with many hidden nodes than few. It will ensure higher accuracy. The training process will be slower though
  • Cross validation and testing error can help us in determining the model with optimal hidden layers

About admin

Check Also

203.5.14 Neural Network Appendix

In this post we will discuss the math behind a few steps of Neural Network …

Leave a Reply

Your email address will not be published. Required fields are marked *