You should have a scripting off fallback for normal pagination in place; the scripting for the infinite scroll then removes that pagination or limits the infinite scroll so as to be PRACTICAL and not blow up the browsers memory footprint. It's the old saying:
If you can't make a fully functional website without JavaScript first, you likely have zero damned business adding scripting to it!
Therefore if you have no scripting off graceful degradation, you've already failed! Also depending on the type of website (government, health care, public utility, banking) you're also risking fines, civil lawsuits, and breach of contract.
Though really, being dumbass scripttardery, infinite scroll is basically telling search AND users to go plow themselves -- part of why it's the typical artsy fartsy harder to use BS I do not advocate crapping on a website with in the first place! It flips the double bird at too many users!!!
BUT IF you are going to do it, have a bloody scripting off fallback plan! Any well written accessible site should be created as if JavaScript doesn't exist FIRST, and then ENHANCED with scripting as desired. Whilst yes, massive popular sites like FB or half-twit can tell large swaths of users with accessibility needs to f*** off, it's not a great plan if you don't have that level of traction.