Both, but it's not that easy to just take those two values into account. Basically, the page size determines how much traffic the user is creating. Example given: page size is 1kb and 1 million users accessing your website. Means your server has to deliver about 1 GB (1kb * 1000000 users) at the same time. If your server only can deliver let's say 250 MB per second it takes at least 4 seconds to deliver the whole page to all users (very simple calculation here). However, it's more important to know how many concurrent users the server itself is able to handle. Usually and with default configuration the most setups can deliver between a few hundred to a few thousand responses per second (example: help.dreamhost.com/hc/en-us/articles/215945987-We…). Let's say your server is able to serve your site to 5000 users at the same time. That means that it takes at least 200 seconds (1000000 / 5000) to server everything, because only the first 5000 users will get an answer, all others have to wait in line for the page.
Another more important metric you should take into account is the time it takes for your server to respond to a user request, also known as TTF (time to first byte). It could be described as the delay between your server getting the user request and the start of the delivery of the website. This time should be as low as possible but usually takes a few dozen to a few hundred milliseconds. This delay pushes the users waiting in line even further because for the time of the delay the server is not able to process any new user.
But, if you speak of such a high user amount you have to build a better infrastructure with load balancing and multiple servers responding to your users, as one server will not be enough.