Home / Python / Predictive Modeling & Machine Learning / 204.1.1 Correlation in Python

204.1.1 Correlation in Python

Correlation

What is need of correlation?

  • Is there any association between hours of study and grades?
  • Is there any association between number of temples in a city & murder rate?
  • What happens to sweater sales with increase in temperature? What is the strength of association between them?
  • What happens to ice-cream sales v.s temperature? What is the strength of association between them?
  • How to quantify the association?
  • Which of the above examples has very strong association?
  • Correlation

Correlation coefficient

  • It is a measure of linear association
  • r is the ratio of variance together vs product of individual variances.
Correlationcoefficient(r)=CovarianceofXYSqrt(VarianceXVarianceY)
  • Correlation 0 No linear association
  • Correlation 0 to 0.25 Negligible positive association
  • Correlation 0.25-0.5 Weak positive association
  • Correlation 0.5-0.75 Moderate positive association
  • Correlation >0.75 Very Strong positive association

Practice : Correlation Calculation

  • Dataset: AirPassengers\AirPassengers.csv
  • Find the correlation between number of passengers and promotional budget.
  • Draw a scatter plot between number of passengers and promotional budget
  • Find the correlation between number of passengers and Service_Quality_Score
In [1]:
import pandas as pd
air = pd.read_csv("datasets\\AirPassengers\\AirPassengers.csv")
air.shape
Out[1]:
(80, 9)
In [2]:
air.columns.values
Out[2]:
array(['Week_num', 'Passengers', 'Promotion_Budget',
       'Service_Quality_Score', 'Holiday_week',
       'Delayed_Cancelled_flight_ind', 'Inter_metro_flight_ratio',
       'Bad_Weather_Ind', 'Technical_issues_ind'], dtype=object)
In [3]:
#Find the correlation between number of passengers and promotional budget.
import numpy as np
np.corrcoef(air.Passengers,air.Promotion_Budget)
Out[3]:
array([[ 1.        ,  0.96585103],
       [ 0.96585103,  1.        ]])
In [4]:
#Draw a scatter plot between number of passengers and promotional budget
import matplotlib.pyplot as plt
%matplotlib inline  
plt.scatter(air.Passengers, air.Promotion_Budget)
Out[4]:
<matplotlib.collections.PathCollection at 0x8feb8d0>
In [5]:
#Find the correlation between number of passengers and Service_Quality_Score
np.corrcoef(air.Passengers,air.Service_Quality_Score)
Out[5]:
array([[ 1.        , -0.88653002],
       [-0.88653002,  1.        ]])

Beyond Pearson Correlation

  • Correlation coefficient measures for different types of data
Variable Y\X Quantitative /Continuous X Ordinal/Ranked/Discrete X Nominal/Categorical X
Quantitative Y Pearson r Biserial rb Point Biserial rpb
Ordinal/Ranked/Discrete Y Biserial rb Spearman rho/Kendall’s Rank Biserial rrb
Nominal/Categorical Y Point Biserial rpb Rank Biserial rrb Phi, Contingency Coeff, V

About admin

Check Also

204.7.6 Practice : Random Forest

Let’s implement the concept of Random Forest into practice using Python. Practice : Random Forest …

Leave a Reply

Your email address will not be published. Required fields are marked *