Its the problem where you should look at one way binding / reducing bindings. One of the best way to use two way binding with a huge data is virtual repeat. I have kind of same problem in my application, where i have around 10k rows with 13-18 columns which have 2-4 different components which needs two way binding. There are many virtual repeat directives if you google them, however i am using md-virtual-repeat by Angular Material. The one problem it can create for you that virtual repeat requires all item height to be equivalent, so if you have a row of 10px then all rows should be 10px. The only dark side of this, rather then this it works smoothly. I saw that google fonts have kind of same thing on their new home page which does the same thing. The logic behind virtual repeat is pretty simple, imagine you have a wrapper of 250px and have 100 items 5px each, now you have items worth 500px and viewarea is 250px what virtual repeat will do is when you scroll in your container, it only loads that items which can be visible in that 250px area ( 2/3 extra to maintain smooth scrolling )
I am not sure if the Material design one will work for your issue, however you can write a custom one which can work with dynamic heights :)