Perhaps I can answer this from a slightly different angle, as the employer (I'm a co-founder of a startup).
So, this is still a learning curve for us, as we've only really employed interns and not full time people.
So here's what we provide. NB - By this point almost all the HR/paperwork has been complete. The only outstanding documents are those that need to be handed to the new team member to have a copy of, and to also have a physically signed copy of the contract.
As Adam said. There's no real work done in the first day, just intros and settling in.
Some people here my disagree with not fully setting up the devs computer with exactly what they need, but I feel allowing them to choose the programs they prefer provide them with the opportunity to setup a new environment (which can be difficult if not done in a while, because we all just set it up then leave it and hope it continues to work). But I also feel it gives them an additional sense of appreciation as I care about what they want to use, instead of just telling them to use something they don't prefer.