Ordinary Least Square-PART 1

3 min readApr 17, 2021

Let’s get this clear, it is beautiful when you understand something clearly ..and especially if it is an algorithm. It immediately gives you confident that you can rule the world.

So here is my in-depth intuition of what linear regression is about, jotting down if so that i can revise it later :)

We have all seen something like

for observation i

Assumptions of OLS Regression

The equation is assumed to be linear in parameter. Yes, it is parameters and not independent variable x. if it is logx , e^x , x² , you can simply say these functions to be some other variable t and carry on the regression.
The independent variable x is deterministic variable as we know it beforehand and it can not change after observation is recorded. The response and explanatory variable have exact relationship, that is given x, we can predict y with 100% certainty which gives OLS estimation a deterministic relationship.
Epsilon here is a random variable which has Normal Distribution.
Expectation of Epsilon is zero.
Homoscedasticity :Variance of Epsilon is constant for all i. The below graph shows the plot which can be evaluated to check this condition, where epsilon is plotted against each observation.

Zero Autocorrelation: This means that the episilons of two observation are not correlated and are independent that is mathematically

Epsilons are independent of their corresponding independent variable values. i.e.

Number of observations are greater than number of parameters, where parameters are betas and also constant variance term of epsilon. As they completely determine the picture of the response variable.