Yes, gradient descent is way more important for machine learning, and also if you have a lot of parameters the matrices get too big to efficiently invert them and you are better of with gradient descent. When I started looking into ML I was very confused that linear regression is even considered machine learning. But you can explain a lot of important things with it like gradient descent, supervised learning over/underfitting etc.

Quick thing: for linear regression you actually don't need gradient descent and you can solve it directly using linear algebra (it's called the Normal Equation in Machine Learning). Here is a sample code I posted on twitter recently: import numpy as np number_of_flats = ( 1 , 2 , 3 , 4 , 5 , 6 , 7 ) price_of_house = ( 10000 , 20000 , 30000 , 40000 , 50000 , 60000 , 70000 ) A = np.vstack([np.ones(len(number_of_flats)), number_of_flats]).transpose() b , a = np.linalg.pinv(A.transpose().dot(A)).dot(A.transpose()).dot(price_of_house) model = lambda x: a*x+b print (model( 10 ))