Let's say we take facebook as an example. The feed in facebook is infinitely scrolled. Since the indexing bot cannot perform human interactions like scroll (or at least that's my assumption here ... correct me if I'm wrong. 😛), how do I make sure that the indexing bot can see everything on the page?
You should have a scripting off fallback for normal pagination in place; the scripting for the infinite scroll then removes that pagination or limits the infinite scroll so as to be PRACTICAL and not blow up the browsers memory footprint. It's the old saying:
If you can't make a fully functional website without JavaScript first, you likely have zero damned business adding scripting to it!
Therefore if you have no scripting off graceful degradation, you've already failed! Also depending on the type of website (government, health care, public utility, banking) you're also risking fines, civil lawsuits, and breach of contract.
Though really, being dumbass scripttardery, infinite scroll is basically telling search AND users to go plow themselves -- part of why it's the typical artsy fartsy harder to use BS I do not advocate crapping on a website with in the first place! It flips the double bird at too many users!!!
BUT IF you are going to do it, have a bloody scripting off fallback plan! Any well written accessible site should be created as if JavaScript doesn't exist FIRST, and then ENHANCED with scripting as desired. Whilst yes, massive popular sites like FB or half-twit can tell large swaths of users with accessibility needs to f*** off, it's not a great plan if you don't have that level of traction.
Atul Sharma
Full Stack Developer | Cloud Native Applications
Create unique URL for every-post just like in Facebook when you click on post time (under username who posted it ) it will open that post on new page without timeline (Only the post) with its unique url in url bar.
Now add that url to sitemap.xml
That's it ! The crawler will pick url from sitemap file and will index it.
Just like I have seen this post in my timeline but it has a unique url (hashnode.com/post/if-a-page-has-infinite-scroll-h…) using which you can access post directly.
Now you can keep displaying the post in timeline of users but google has indexed it So that user can directly land on that post from Google search.