Thanks for doing the AMA!
I am a freelance developer who has used DO exclusively for all my servers for several years now. Being able to provision servers via API has been a huge timesaver. Most everything I try to do with the platform is focused on automation as I just don't have time to manually do things. With that being the context, I am curious about the following:
Block storage has been really useful for abstracting files from the main OS, but I would like to share that between two (or more) servers for load-balancing instead of something like GlusterFS. Any plans for something like this (and I know this is a non-trivial problem)?
I just started with object storage and was planning on using it much like a CDN to offload assets and save on block storage (provisioned one private, for backups, and one public, for content). Was it designed for that in mind, or more of a straight up S3 alternative? Mostly, I'm worried about latencies, not that DO has ever really underperformed in that regard.
Any future plans for cross datacenter networking and/or easy live migration of droplets between datacenters? I would love to be able to move them around as need be, but have all the associated things come with or work (i.e. floating IP's, firewalls, block storage).
Thank you for DO, it has seriously changed my life as a developer. Most of what I do now wouldn't be possible without it. Thank you!
Hi Adam - thanks for all of the questions.
Re: block storage mounting bet. multiple servers at once, we've heard that ask before. No definitive plans around that yet. (If you don't mind sharing, what's your use case for a shared (between droplets) block volume?)
Re: object storage (Spaces), it is not a CDN and is not designed as such. That being said, we have many customers (including DO!) that use Spaces as a CDN origin.
Here are some instructions for doing just that:
- Fastly: https://docs.fastly.com/guides/integrations/digitalocean-spaces
- KeyCDN: https://www.keycdn.com/support/digitalocean-spaces-cdn/
(Note: Spaces should work with almost any CDN, these are simply the ones where I could track down vendor documentation :)
In the future we'd like to help customers automate some of the config and management of CDNs when used with Spaces. Hope to be able to share more on that front in '18.
Cross DC networking: We are investing a lot in building out our network capabilities and you should expect to hear more on this as we get into '18.
Thank you so much for answering all the questions!
My use case basically is much easier fail-over for servers. Since the storage already has high availability on your side, being able to have that style of sharing would let me have multiple front-end servers (notably for Wordpress) that target the same block storage. It lets me horizontally scale with minimal effort and cost.
Thanks for the CDN links, I was not a 100% on the use cases there so I appreciate that!
Thanks for mentioning your use-case! Designing and scaling applications can be challenging and sometimes tricky to get all the aspects right. It's also important to see what kind of use-case we are dealing as one solution might not map equally well to all problems.
I think one way to scale a typical system that has a "stateless" frontend sitting on top talking to a persistent data store, is to decouple those two pieces. The backend datastore can be mounted on a separate droplet that has block storage attached to it (and hence it gets all the high-consistency+fault-tolerance guarantees for all the valuable data it is dealing with) and there could be multiple wordpress instances that connect to it. That way you can scale wordpress instances to handle skews in traffic, and your data store volume is always protected even if your database droplet undergoes any problem (since the actual storage is remote to it and available from any new droplet you create from a snapshot for instance).
However, a lot would depend on your use-case and if wordpress (or any other blogging tool) allows such a setup or not before you decide to head down this route. :)
With spaces, DO stepped into object storage. Should we expect to see new additions like cdn, e-mail infra, etc so that companies can set up their whole infrastructure on DO?
Hi Ipseeta, yes you can expect new products and features in the coming months to help developers and businesses build, scale and manage their applications on DigitalOcean. We’ve been growing our product and engineering teams to do just that!
For CDN and email infrastructure, there are some good open source and commercial options. We’re always aiming to make open source options easy to use on DO, and when there are popular commercial products we will look to make those integrations easy as well.
Hi, DO team
Thanks for doing AMA's.
I want to know that, are you planning/building project like firebase/Firestore or real-time database?
Because mostly startups want serverless infrastructure at initial phase.
Hi @DO team! Thanks for the AMA.
Is DigitalOcean mainly only preferred by Startups and freelance developers? How do you attract corporate companies?
Hi Duane, we’ve seen a steady increase in the number of project teams that use DigitalOcean, with over 70K teams on the platform and 1+ million developers.
These teams are at startups and larger companies, for example slack and docker. The ease of using DigitalOcean is normally why we are brought into a company, usually by a developer who has used us in the past and wants to use us for a project at work. We’ve profiled a few customers on our web site and will be adding more: https://www.digitalocean.com/customers/
Hi there! With the cloud market heating up, how is DO planning to compete with AWS, Google Cloud, Alibaba Cloud, etc?
What database backend stores filestructure in digital ocean places? And how is it designed?
For both block and object storage we use Ceph (https://github.com/ceph/ceph) as our backend storage engine. The requirement of distributing copies of user data effectively while keeping them fully consistent was paramount to us. That and the fact that we like to use open-source tools throughout our stack (and contribute upstream whenever possible) led us to make this choice.
Regarding the design of Ceph, I like to point people to https://www.youtube.com/watch?v=7I9uxoEhUdY since it gives an overview of the internals very nicely. :)
Will Digital Ocean gift cards be available in time for Christmas?
Thanks for doing the AMA, DigitalOcean!
Any plans on offering a database as a service product?
Based on the recent releases it looks like it should be the next move so people can easily move all their infrastructure to DO.
What degree(s) are most relevant to this field?
Do you have any plans to offer GPUs or "serverless" functions(like AWS Lambda)? Thanks for doing the AMA!
Clearly you automate the heck out of everything you work on and I imagine you work a lot with bare metal. What challenges have you faced regarding automating bare metal resources? What tools have you used to help with those challenges?
With different hardware vendors having widely varying interfaces for automating their products, how do you manage those interfaces to provide the same end-result across the board?
Do you implement automated testing or CI of the physical provisioning process? If so, what does that look like?
What University degree(s) would you consider most valuable in this field?
Delivering both block and object storage platforms out to our users involved our very talented designers, engineers, infrastructure, operations, product, support and marketing teams, so there's no one particular "field" that you need to be best at. ;)
I can speak specific to storage engineering a little bit. I don't think there's a strong requirement of a university degree as such, but having the knowledge (and/or experience) of dealing with persistence (along with the problems that come with it) and various access patterns for large amounts of data would certainly be useful. We have built several services internally that manage information flow and persist metadata of storage clusters, that don't deal directly with the underlying user data but are critical to the control plane and having generic software engineering skills is quite useful here.