Linear Regression

Different techniques can be used to prepare or train the linear regression equation from data, the most common of which is called Ordinary Least Squares

Linear Regression is a supervised learning algorithm which assumes a linear relationship between the input variables (x) and the continuous output variable (y). In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables. When there is one input (explanatory or independent) variable, is called simple linear regression; for more than one, the process is called multiple linear regression.

Table of Content

1. Overview
2. Types of Linear Regression
3. Assumptions
4. Video
6. Overfitting in Regression (To be added)

Overview

The objective of Linear Regression is to predict a dependent variable value (y) based on a given independent variables (x) using a best fit line. We assume there is an linear relationship between dependent and independent variables; and dependent variable must be continuous in nature.

Let’s understand theory of linear regression using an example:

We want to predict the house prices (y) using an independent feature “size in square feet” (x).

To predict house price using above data set, Linear regression will help. Let’s suppose above line is the best fit line of linear regression (pink line in the above image), with a corresponding mathematical equation. Now, if we know the equation of that line, then for any given house size i.e. input (x), we can find any house price i.e. output (y).

Hypothesis Function

Lets use the hypothesis function formula as:

hθ(x)=θ01(x)

Cost Function

The vertical distance between the points and the fitted line (best fit line) are called errors. The main idea of objective function is to fit this regression line by minimizing the sum of squares of these errors or Cost Function. This is also known as principle of least squares.

minimize θ01 — J(θ01)

– where J(θ01) is the cost function.

Visit this link for regression notes: Univariate Linear Regression