As a previous VP Eng and current CTO I can try and provide some insight.

It certainly depends on the size of the organization as I agree with Fred Wilson, early stage companies can have a single person handling both. At some point though, this becomes impossible. I think there are definitely grey areas here, as you find some very technical CTOs, or some CTOs that are great leaders but have not been close to the tech for some time and rely on a strong VPE.

I would say as the org gets to around 30-40 people you see the usual pattern of an experienced CTO come on board who usually has a VPE and a VP Operations reporting into him/her, separating out Engineering and Infrastructure/Operations.

At this point, a CTO starts to assume some of the non-technical burden that is required to run the technology organization. Working with HR/Recruiting on a recruiting strategy. Working with finance on budgets and forecasts. Making sure the teams are empowered with the right tools. At the same time however, there is the role of the over-arching technology strategy. What are the infrastructure, security and process goals of the organization ? Are there culture issues (technical or not) that need to be re-enforced ? Is the organization structured right? How are the managers performing ? Etc.

For a VPE, I think its a lot more of a balance of managing engineers, managing the overall quality of the code/infrastructure and being more ingrained in the day to day operation of the engineering organization. Things that might seem too nuanced for a CTO (like how are we handling our automated deployments, or leveraging Vagrant for local development?) is probably something more in the camp of a VPE. At this point, the CTO should be setting the overall technology strategy and vision (focusing on automation so we can move quicker) and the VPE should be more focused on the tools and execution to make it happen (Vagrant + Ansible + Jenkins + Docker, etc). Here, the CTO is less concerned about how its done and more concerned that it gets done and executed on in a way he/she is happy with. This issue becomes more magnified for a CTO as the Engineering organization grows to Software, Test, Data, Data Science, etc. He/She is going to need all these Engineering leaders to ensure they are executing under his/her vision properly and he/she would be stretched way too thin to get involved at the necessary levels without a proper VPE.

Hope that helps.

In small startups, one person can do both jobs (maybe not great, but one person can do both).

When you start looking at big companies or well-funded companies, your Vice President of Engineering is usually a glorified project manager who is also a great developer, but in his role as VP of Engineering, his main focus is managing people, keeping projects to their deadlines, building teams, keeping people happy and he barely touches code even though he's a great developer - so in short, very people orientated. In some cases, the VP of Engineering can also be managing multiple teams each with their own dev managers.

The CTO on the other hand has a black belt in Software Architecture, Infrastructure, he understands all the latest technologies, pros and cons of each of them, he has a firm understanding of business, often have some sort of business degree as well, he's a visionary and can often see trends which helps him understand which technologies to use and which to avoid.

In medium sized companies, the CTO is often the architect as well, he lays the foundation, builds the infrastructure and then leave it up to the senior developers to drive the project further. In larger companies, the CTO will work with an Architect and the CTO will drive it from a business point of view, telling the architect in technical terms what he wants, the architect will then take care of the technology constantly running it past the CTO while the CTO's core focus will be working with business to understand their needs, telling them if it can be done, then working with the Dev Manager / VP of Engineering and Architect to explain to them what needs to happen in technical terms - so in short, very business orientated.

Fred Wilson, a Venture Capitalist has this to say about the two roles:

A VP Engineering is ideally a great manager and a great team builder. He or she will be an excellent recruiter, a great communicator, and a great issue resolver. The VP Eng’s job is to make everyone in the engineering organization successful and he or she needs to fix the issues that are getting in the way of success.

A CTO is ideally the strongest technologist in the organization. He or she will be an architect, a thinker, a researcher, a tester and a tinkerer. The CTO is often the technical co-founder if there is one (and you know I think there must be one). 

Source: avc.com/2011/10/vp-engineering-vs-cto

Thanks for inviting me into this discussion, but I can't help you out here. I've worked mainly for smaller businesses, so there weren't that many layers to justify a VP or CTO... mostly lead developers and PMs ;)

I hope someone here can give you the answer though!

I like to think of it this way: The CTO makes sure the company builds the right thing. The VP of Engineering make sure the company builds the thing right.