Ordinary least squares, OLS
---------------------------
OLS is at the core of econometrics curriculum, it is easily derived and
highly practical to familiarise a learner with regression possibilites
and limitations.
The usual way to teach OLS is to present assumptions and show how to deal
with their violations as indicated below in a review chart from Kennedy's
textbook.
.. image:: _static/peter_kennedy_on_ols.png
Math:
:math:`Y = \beta X + \epsilon`, :math:`\epsilon` is iid, normal with finite variance.
Common steps:
1. specify model: select explanatory variables, transform them if needed
2. estimate coefficients
3. elaborate on model quality (the hardest part)
4. go to 1 if needed
5. know what model *does not* show (next hardeer part)
What may go wrong:
- residuals are not random
- variables are cointegrated
- multicollinearity in regressors
- residuals depend on x (heteroscedasticity)
- inference is not causality
- wrong signs, insignificant coefficients
- variable normalisation was not described
Discussion:
- why sum of squares as a loss function?
- connections to bayesian estimation
- is R2 useful or dangerous?
Implementations:
- `lm function in
R `__
- `OLS class in python
statsmodels `__
- `python scypi least
squares `__
- julia `Alistair `__, GLM.jl,
Regression.jl
- `Replication
examples `__
- check unsorted `links about OLS `__ - but it is not
better than googling on your own
Baby dragon special:
.. raw:: html