Home / Python / Predictive Modeling & Machine Learning / 204.5.9 Building a Neural Network in Python

# 204.5.9 Building a Neural Network in Python

## Building the Neural Network

The good news is…

• We don’t need to write the code for weights calculation and updating
• The gradient descent method is not very easy to understand for a non mathematics students
• Neural network tools don’t expect the user to write the code for the full length back propagation algorithm

### Building the Neural Network in Python

• We have a couple of packages available in Python
• We need to mention the dataset, input, output & number of hidden layers as input.
• Neural network calculations are very complex. The algorithm may take sometime to produce the results
• One need to be careful while setting the parameters. The runtime changed based on the input parameter values.

## Python Code Options

• Step 1 : import the neurolab
• Step 2 : We need a epouch(error pouch) `error = []`, In this we store error value of each iteration or epoach.
• Step 3 : Create a network
• Here `nl.net.newff([[0, 1],[0,1]],[4,1],transf=[nl.trans.LogSig()] * 2)` is feed forward neural network
• 1st argument is min max values of predictor variables; can be a list of list
• 2nd argument is no.of nodes in each layer i.e 4 in hidden 1 in output layer
• transf is transfer function applied in each layer
• Step 4 : Train Network
• net.train() outputs error which is appended to error variable and has a few parameters
• Step 5 : Simulate network
• net.sim(input) gives the output for the network

## Practice : Building the neural network in Python

• Build a neural network for XOR data
• Dataset: Emp_Productivity/Emp_Productivity.csv
• Draw a 2D graph between age, experience and productivity
• Build neural network algorithm to predict the productivity based on age and experience
• Plot the neural network with final weights

#### Manual Calculation of Weights XOR Dataset

In [37]:
```## Importing the dataset
xor_data.shape
```
Out[37]:
`(4, 3)`
In [38]:
```xor_data.head()
```
Out[38]:
input1 input2 output
0 1 1 0
1 1 0 1
2 0 1 1
3 0 0 0
In [39]:
```#Neural network building
#We use 'neurolab' package for implementing neural nets in python
#We need to manually downlaod and install  neurolab
import neurolab as nl
import numpy as np
import pylab as pl
```
In [40]:
```error = []        #In this we store error value of each iteration or epoach

# Create network with 1 hidden layer and random initialized
#nl.net.newff() is feed forward neural network
#1st argument is min max values of predictor variables
#2nd argument is no.of nodes in each layer i.e 4 in hidden 1 in o/p
#transf is transfer function applied in each layer
net = nl.net.newff([[0, 1],[0,1]],[4,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
```
In [41]:
```# Training network
#net.train outputs error which is appended to error variable
error.append(net.train(xor_data[["input1"]+["input2"]], xor_data[["output"]], show=0, epochs = 100,goal=0.001))
```
In [42]:
```#plotting epoches Vs error
#we can use this plot to specify the no.of epoaches in training to reduce time
pl.figure(1)
pl.plot(error[0])
pl.xlabel('Number of epochs')
pl.ylabel('Training error')
pl.grid()
pl.show()
```
In [43]:
```# Simulate network(predicting)
predicted_values = net.sim(xor_data[["input1"]+["input2"]])
```
In [44]:
```#converting predicted values into classes by using threshold
predicted_class=predicted_values
predicted_class[predicted_values>0.5]=1
predicted_class[predicted_values<=0.5]=0
```
In [45]:
```#predicted classes
predicted_class
```
Out[45]:
```array([[ 0.],
[ 1.],
[ 1.],
[ 0.]])```
In [46]:
```#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(xor_data[['output']],predicted_class)
print('ConfusionMatrix : ', ConfusionMatrix)

#accuracy
accuracy=(ConfusionMatrix[0,0]+ConfusionMatrix[1,1])/sum(sum(ConfusionMatrix))
print('Accuracy : ', accuracy)

#accuracy is 100% it classifying every training observation correctly
error=1-accuracy
print('Error :', error)
```
```ConfusionMatrix :  [[2 0]
[0 2]]
Accuracy :  1.0
Error : 0.0
```

Building the neural network in Python on EMP_Productivity datasets

In [47]:
```Emp_Productivity_raw = pd.read_csv("datasets\\Emp_Productivity\\Emp_Productivity.csv")
```
Out[47]:
Age Experience Productivity Sample_Set
0 20.0 2.3 0 1
1 16.2 2.2 0 1
2 20.2 1.8 0 1
3 18.8 1.4 0 1
4 18.9 3.2 0 1
5 16.7 3.9 0 1
6 16.3 1.4 0 1
7 20.0 1.4 0 1
8 18.0 3.6 0 1
9 21.2 4.3 0 1
In [48]:
```#Draw a scatter plot that shows Age on X axis and Experience on Y-axis. Try to distinguish the two classes with colors or shapes.
import matplotlib.pyplot as plt

fig = plt.figure()

ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==0], s=10, c='b', marker="o", label='Productivity 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==1], s=10, c='r', marker="+", label='Productivity 1')
plt.legend(loc='upper left');
plt.show()
```
In [49]:
```####Building neural net
import neurolab as nl
import numpy as np
import pylab as pl

error = []
# Create network with 1 layer and random initialized
net = nl.net.newff([[15, 60],[1,20]],[6,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
```
In [50]:
```# Train network
error.append(net.train(Emp_Productivity_raw[["Age"]+["Experience"]], Emp_Productivity_raw[["Productivity"]], show=0, epochs = 500,goal=0.02))

# Simulate network
predicted_values = net.sim(Emp_Productivity_raw[["Age"]+["Experience"]])
```
In [51]:
```#Converting Predictive values into Predected Classes
predicted_class=predicted_values
predicted_class[predicted_values>0.5]=1
predicted_class[predicted_values<=0.5]=0

#Predcited Classes
predicted_class[0:10]
```
Out[51]:
```array([[ 0.],
[ 0.],
[ 0.],
[ 0.],
[ 0.],
[ 0.],
[ 0.],
[ 0.],
[ 0.],
[ 0.]])```
In [52]:
```#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(Emp_Productivity_raw[['Productivity']],predicted_class)
print('Confusion Matrix : ', ConfusionMatrix)

#accuracy
accuracy=(ConfusionMatrix[0,0]+ConfusionMatrix[1,1])/sum(sum(ConfusionMatrix))
print('Accuracy : ', accuracy)

error=1-accuracy
print('Error : ', error)
```
```Confusion Matrix :  [[75  1]
[ 3 40]]
Accuracy :  0.966386554622
Error :  0.0336134453782
```
In [53]:
```#plotting actual and prected classes
Emp_Productivity_raw['predicted_class']=pd.DataFrame(predicted_class)

import matplotlib.pyplot as plt

fig = plt.figure()

ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==0], s=40, c='g', marker="o", label='Productivity 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==1], s=40, c='g', marker="x", label='Productivity 1')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.predicted_class==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.predicted_class==0], s=30, c='b', marker="1", label='Predicted 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.predicted_class==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.predicted_class==1], s=30, c='r', marker="2", label='Predicted 1')

plt.legend(loc='upper left');
plt.show()
```
In [54]:
```# Simulate network with 'Age': 40; 'experience':12
x = np.array([[40,12]])
net.sim(x)
```
Out[54]:
`array([[ 0.00012387]])`

With the threshold 0.5 the above output will be ‘0’.