For a beginner guide the browser vs http split works as a starting point but in production most people just mix both. I run the browser once to grab tokens or the request shape, then replay that over cheap http at volume and only bring out the full browser when nothing else works.
Also a lot of js heavy retail sites hydrate from embedded state, like the data blob next.js dumps into the page, or from internal json and graphQL endpoints, so you can often skip the browser. Client still has to pass TLS fingerprint checks just to pull the html tho, and digging those APIs out takes real work and breaks silently once a private endpoint changes.
Playwright has basically taken over as default for new projects, selenium 4 is still fine for legacy code or grid. Log the context with each availability record, location, variant, fulfillment, account state, timestamp, since availability shifts around constantly
For a beginner guide the browser vs http split works as a starting point but in production most people just mix both. I run the browser once to grab tokens or the request shape, then replay that over cheap http at volume and only bring out the full browser when nothing else works. Also a lot of js heavy retail sites hydrate from embedded state, like the data blob next.js dumps into the page, or from internal json and graphQL endpoints, so you can often skip the browser. Client still has to pass TLS fingerprint checks just to pull the html tho, and digging those APIs out takes real work and breaks silently once a private endpoint changes.
Playwright has basically taken over as default for new projects, selenium 4 is still fine for legacy code or grid. Log the context with each availability record, location, variant, fulfillment, account state, timestamp, since availability shifts around constantly