Quick thing: for linear regression you actually don't need gradient descent and you can solve it directly using linear algebra (it's called the Normal Equation in Machine Learning).
Here is a sample code I posted on twitter recently:
import numpy as np
number_of_flats = (1, 2, 3, 4, 5, 6, 7)
price_of_house = (10000, 20000, 30000, 40000, 50000, 60000, 70000)
A = np.vstack([np.ones(len(number_of_flats)), number_of_flats]).transpose()
b, a = np.linalg.pinv(A.transpose().dot(A)).dot(A.transpose()).dot(price_of_house)
model = lambda x: a*x+b
print(model(10))
That is a great point, Marcel! Thanks for contributing this explanation and code. It is always good to look at alternative ways to solve a problem.
Although gradient descent isn't necessary for linear regression, I chose to use it here, partly because gradient descent is just such a useful thing to know in machine learning, and partly because it is something I have been studying lately. 🙂
Yes, gradient descent is way more important for machine learning, and also if you have a lot of parameters the matrices get too big to efficiently invert them and you are better of with gradient descent. When I started looking into ML I was very confused that linear regression is even considered machine learning. But you can explain a lot of important things with it like gradient descent, supervised learning over/underfitting etc.
Marcel Bischoff
Quick thing: for linear regression you actually don't need gradient descent and you can solve it directly using linear algebra (it's called the Normal Equation in Machine Learning). Here is a sample code I posted on twitter recently:
import numpy as np number_of_flats = (1, 2, 3, 4, 5, 6, 7) price_of_house = (10000, 20000, 30000, 40000, 50000, 60000, 70000) A = np.vstack([np.ones(len(number_of_flats)), number_of_flats]).transpose() b, a = np.linalg.pinv(A.transpose().dot(A)).dot(A.transpose()).dot(price_of_house) model = lambda x: a*x+b print(model(10))