Selecting Significant Variable in ML Regression

Step Regression

Dinesh

3 November 2016

About Dataset:  Cars total observation : 206 , Variable : 26

Setting up the environment

getwd()

## [1] "C:/Users/ssn/Documents/Dinesh Analyst/Project"

setwd("C:/Users/ssn/Documents/Dinesh Analyst/Project")
automobile = read.csv("cars.csv",header = TRUE)
View(automobile)

To read Na values

colSums(is.na(automobile))

##         symboling normalized.losses              make         fuel.type
##                 0                41                 0                 0
##        aspiration      num.of.doors        body.style      drive.wheels
##                 0                 0                 0                 0
##   engine.location        wheel.base            length             width
##                 0                 0                 0                 0
##            height       curb.weight       engine.type  num.of.cylinders
##                 0                 0                 0                 0
##       engine.size       fuel.system              bore            stroke
##                 0                 0                 4                 4
## compression.ratio        horsepower          peak.rpm          city.mpg
##                 0                 2                 2                 0
##       highway.mpg             price
##                 0                 4

Converting Na into values by substituting the Median value of the variables in the each column.

automobile$normalized.losses[is.na(automobile$normalized.losses)]=median(automobile$normalized.losses,na.rm = T)
automobile$horsepower[is.na(automobile$horsepower)]=median(automobile$horsepower,na.rm = T)
automobile$price[is.na(automobile$price)]=median(automobile$price,na.rm = T)
automobile$bore[is.na(automobile$bore)]=median(automobile$bore,na.rm = T)
automobile$stroke[is.na(automobile$stroke)]=median(automobile$stroke,na.rm = T)
automobile$peak.rpm[is.na(automobile$peak.rpm)]=median(automobile$peak.rpm,na.rm = T)

Changing Rating Levels Into Factors

str(automobile)

## 'data.frame':    205 obs. of  26 variables:
##  $ symboling        : int  3 3 1 2 2 2 1 1 1 0 ...
##  $ normalized.losses: num  115 115 115 164 164 115 158 115 158 115 ...
##  $ make             : Factor w/ 22 levels "alfa-romero",..: 1 1 1 2 2 2 2 2 2 2 ...
##  $ fuel.type        : Factor w/ 2 levels "diesel","gas": 2 2 2 2 2 2 2 2 2 2 ...
##  $ aspiration       : Factor w/ 2 levels "std","turbo": 1 1 1 1 1 1 1 1 2 2 ...
##  $ num.of.doors     : Factor w/ 2 levels "four","two": 2 2 2 1 1 2 1 1 1 2 ...
##  $ body.style       : Factor w/ 5 levels "convertible",..: 1 1 3 4 4 4 4 5 4 3 ...
##  $ drive.wheels     : Factor w/ 3 levels "4wd","fwd","rwd": 3 3 3 2 1 2 2 2 2 1 ...
##  $ engine.location  : Factor w/ 2 levels "front","rear": 1 1 1 1 1 1 1 1 1 1 ...
##  $ wheel.base       : num  88.6 88.6 94.5 99.8 99.4 ...
##  $ length           : num  169 169 171 177 177 ...
##  $ width            : num  64.1 64.1 65.5 66.2 66.4 66.3 71.4 71.4 71.4 67.9 ...
##  $ height           : num  48.8 48.8 52.4 54.3 54.3 53.1 55.7 55.7 55.9 52 ...
##  $ curb.weight      : int  2548 2548 2823 2337 2824 2507 2844 2954 3086 3053 ...
##  $ engine.type      : Factor w/ 7 levels "dohc","dohcv",..: 1 1 6 4 4 4 4 4 4 4 ...
##  $ num.of.cylinders : Factor w/ 7 levels "eight","five",..: 3 3 4 3 2 2 2 2 2 2 ...
##  $ engine.size      : int  130 130 152 109 136 136 136 136 131 131 ...
##  $ fuel.system      : Factor w/ 8 levels "1bbl","2bbl",..: 6 6 6 6 6 6 6 6 6 6 ...
##  $ bore             : num  3.47 3.47 2.68 3.19 3.19 3.19 3.19 3.19 3.13 3.13 ...
##  $ stroke           : num  2.68 2.68 3.47 3.4 3.4 3.4 3.4 3.4 3.4 3.4 ...
##  $ compression.ratio: num  9 9 9 10 8 8.5 8.5 8.5 8.3 7 ...
##  $ horsepower       : int  111 111 154 102 115 110 110 110 140 160 ...
##  $ peak.rpm         : int  5000 5000 5000 5500 5500 5500 5500 5500 5500 5500 ...
##  $ city.mpg         : int  21 21 19 24 18 19 19 19 17 16 ...
##  $ highway.mpg      : int  27 27 26 30 22 25 25 25 20 22 ...
##  $ price            : int  13495 16500 16500 13950 17450 15250 17710 18920 23875 10295 ...

automobile$symboling <- as.factor(automobile$symboling)
levels(automobile$symboling)[1] <- "very low risk"
levels(automobile$symboling)[2] <- "low risk"
levels(automobile$symboling)[3] <- "moderate risk"
levels(automobile$symboling)[4] <- "risk"
levels(automobile$symboling)[5] <- "high risk"
levels(automobile$symboling)[6] <- "very high risk"

Creating Regression Line

full <- lm(price ~., data = automobile)

Akaike Information Criterion

AIC(full)

## [1] 3737.955

Getting the Summary value

summary(full)

##
## Call:
## lm(formula = price ~ ., data = automobile)
##
## Residuals:
##     Min      1Q  Median      3Q     Max
## -7642.2  -977.0     0.0   819.9  8761.2
##

## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1930 on 139 degrees of freedom
## Multiple R-squared:  0.9591, Adjusted R-squared:   0.94
## F-statistic: 50.15 on 65 and 139 DF,p-value: < 2.2e-16

Selecting the Best Variables to Predict the Price

model.backward <- step (full,direction = "backward", trace = 1)

## Start:  AIC=3154.19
## price ~ symboling + normalized.losses + make + fuel.type + aspiration +
##     num.of.doors + body.style + drive.wheels + engine.location +
##     wheel.base + length + width + height + curb.weight + engine.type +
##     num.of.cylinders + engine.size + fuel.system + bore + stroke +
##     compression.ratio + horsepower + peak.rpm + city.mpg + highway.mpg
##
##
## ## price ~ make + aspiration + body.style + wheel.base + length +
##     width + curb.weight + engine.type + num.of.cylinders + engine.size +
##     bore + stroke + peak.rpm
##
##                    Df Sum of Sq        RSS    AIC
## <none>                           562722530 3129.2
## - stroke            1   7316986  570039516 3129.8
## - bore              1  11655223  574377753 3131.4
## - wheel.base        1  11751599  574474129 3131.4
## - length            1  22208879  584931409 3135.1
## - curb.weight       1  29019407  591741937 3137.5
## - body.style        4  47937452  610659982 3137.9
## - aspiration        1  36577611  599300141 3140.1
## - num.of.cylinders  4  57125458  619847988 3141.0
## - peak.rpm          1  47698183  610420713 3143.9
## - width             1  70734082  633456612 3151.5
## - engine.size       1  77180510  639903040 3153.5
## - engine.type       4 278331165  841053695 3203.6
## - make             20 508414975 1071137505 3221.1

Checking the Summary of the Model

summary(model.backward)

##
## Call:
## lm(formula = price ~ make + aspiration + body.style + wheel.base +
##     length + width + curb.weight + engine.type + num.of.cylinders +
##     engine.size + bore + stroke + peak.rpm, data = automobile)
##
## Residuals:
##     Min      1Q  Median      3Q     Max
## -7152.8 -1109.7   -27.6   866.9  9213.5
##
## Coefficients: (2 not defined because of singularities)
##                          Estimate Std. Error t value Pr(>|t|)   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1875 on 160 degrees of freedom
## Multiple R-squared:  0.9556, Adjusted R-squared:  0.9433
## F-statistic:  78.2 on 44 and 160 DF,  p-value: < 2.2e-16

Considering the most significant variables to predict the price

refined<- lm (price ~stroke+bore+wheel.base+length+curb.weight+body.style+aspiration+num.of.cylinders+peak.rpm+width+engine.size+engine.type+make, data = automobile)

Checking the Akaike Information Criterion Value for the refined model

AIC(refined)

## [1] 3712.949

Conclusion :

Comparing the Akaike Information Criterion value for the full and refined model

AIC (full)

## [1] 3737.955

AIC (refined)

## [1] 3712.949

Conclusion:

1.      After optimization   refined model gives us the 3712 which has been reduced by 25 in numbers.

2.      Adjusted R Square for Full model and Refined model is the same 0.94

3.      Further considering the F-stat value

Full model give 50 and

Refined models give 78

Refined models can be considered for further analysis.