Firebase's Realtime Database essentially operates as a single threaded process for each database instance. So if you have one long-running operation on the database, that blocks the server from processing any other operations that are going on on that same database.

That's why we recommend performing maintenance in smaller operations, instead of big bulk updates. In fact, that's one of the first things our support team looks for when we are helping developers troubleshoot: are there any periodic processes running that either read the entire database, or update large sections of the database in one go. Splitting such operations up is often the fastest approach to solve the problem.

If you have an operation that needs to inspect the entire database, you can also consider enabling automatic backups of your database, and then using that backup as a read-only copy to perform the analysis. That way you can read the database, without it blocking other operations. This is how you can often optimize maintenance operations too: determine what nodes need to be updated by reading from the backup, then perform a number of (not too big) write operations to update the production database.

We've started answering questions just now :-)

Another way to order lists by a specific criterion is to add an artificial sorting key. Let's say you want to enable users to re-arrange their to-do items into the order they plan to work on them (which might not be the same as the task priority). By adding an "order" field and populating this according to the order the user arranged their tasks in, you can implement this use case with Firestore's built-in capabilities.

Adding more context, given other replies in this Q&A. We need both ability to update small chunks of data frequently and capacity to handle large sets of small chunks of data.

Each Realtime Database instance is essentially running as a single-threaded process on our servers. The load that you see in the usage chart is an indication of how busy that thead is. This means that if you hit 100% sustained load, there's not much more you can do than reduce the amount of work you make the database do.

To scale the Realtime Database beyond that, you have to shard your data over multiple instances. This is one of the big differences between Realtime Database and Firestore: the latter handles scalability automatically for you, while with Realtime Database you'll have to think of it more yourself.

Coming up with a sharding strategy is something best done early in your data modeling phase, typically around the same time when you start thinking about how to secure access to the data. If you have a specific use-case and are wondering how to shard it, I'd always recommend posting to our firebase-talk mailing list as our team loves to think along with those.

Thanks for the Firebase love! Ok I'll try to give each question a short answer:

1. The team is working hard to improve cold starts right now but they'll likely always be an issue. Cloud Run is the future here, it has min instances and concurrency.
2. We already have scheduled functions!
3. No plans to invest in GraphQL right now, although we do notice the enthusiasm.
4. Probably not that exactly but we are working on a way for Firestore developers to cache results to common queries.
5. Nothing in the works right now!
6. Probably not going to change, it's likely Firestore will always charge based on operations. I wouldn't say RTDB is always less expensive though. If you have a large data set (hundreds of GBs) you're going to pay much much less for storage on Firestore!

Nope.

Oh, you probably wanted more info, didn't you?

Okay… no, I think there will be situations where RTDB will always be a better solution. The Realtime Database is optimized for situations where you want small chunks of data that's updated frequently (perhaps to power a poker game or something), where as Cloud Firestore is optimized for larger chunks of data that are updated less frequently. (Like, say, a blog post.)

I'm not sure if there will ever be a day where one database can serve both or your needs, and we have lots of customers happily using each one of them, so I think we're happy to continue running two NoSQL databases for the foreseeable future.

This is really helpful, thanks for sharing! Samuel Stern

Thanks for sharing! Todd Kerpelman

This is really helpful, thanks for sharing! Frank van Puffelen

If you haven't used Firebase much in the recent past, you might be interested to hear we added Firebase Hosting, which provides fast and secure hosting for your web app, static and dynamic content, and microservices. You can host your JAMStack app on Firebase Hosting.

The key tenet of JAMstack is to serve as much pre-generated and CDN cacheable content as possible. Firebase Hosting's global CDN lets you do that easily with static files, or, if you do want to generate some markup on the fly, lets you easily set up a Cloud Function or Cloud Run container behind the CDN. Both Cloud Functions and Cloud Run scale down to 0 if they're not being used, but auto-scale quickly if you were to get a spike in non-cacheable requests.

Firebase also has databases like Firestore that allow you to retrieve data in realtime on the client without running your own server. The Firebase client-side JS SDKs let you easily authenticate and interact with your Firebase database and other Firebase services directly from client code, while security rules make it easy to secure your data without an additional server layer.

Yep this is pretty bad and we know it. The short answer is that there's not much you can do about it now and you may want to avoid Hosting → Run/Functions rewrites if your primary user base is far from us-central. The long answer is that we want to bring the Hosting "origin" to more places and the team is investigating how to do it!

Samuel Stern this is great news about origin! I'm Super happy, thanks for responding :)

See an answer on the "missing functionality for GeoPoint" Firestore issue:

For those arriving here, I wouldn't hold your breath. I've been waiting for Geo Queries on Cloud Datastore (which Firestore is pretty much just a new iteration of) for over 5 years now. There are much better geo-native solutions out there if you want to get it done. Anything geo built on top of Firestore / Datastore is a hack at best, and from what I've seen vastly under-performs just picking the right tool the first time.

I really wish Google Cloud would give us a nice native option for this though - I mean they certainly do have the tech for it (Google Maps etc.) but they clearly aren't open to sharing that just yet...

Really sucks to see that Google isn't doing anything for that in 6 years..

We've answered this in a few other comments (geo queries are popular, who knew?!?) but it's worth answering again.

With Firestore one of the core principles is that query speed depends on the number of results, not the size of the data set. Doing geo queries which are both accurate and fast is really, really hard and we have not yet invested in that. There are some tricks you can do with geohashes though and we're working on getting out some better documentation about that.

Samuel Stern I see. My specific use-case is to show nearby posts, so I don't need accuracy but can work in clusters/chunks with customizable radius, so using geohashes would theoretically work for me. Can't wait for that documentation

No work is currently being done on adding querying capabilities to the Cloud Storage SDK for Firebase, as it just doesn't fit very well with the core functionality of that product. I'd recommend storing the metadata you want to query on in a database (like Firestore or Realtime Database) and then performing your searches there.

Right now, probably your best option is to export your data into BigQuery and then import it from there into a tool like Looker, Tableau, or Data Studio. That said, I know the GA team is working on some ways to more easily get at your data programmatically so there might be some more clever solutions you'll be able to build in the future.

Ok there's a lot here let me see if I can address at least most of it:

1. I agree token management in FCM is hard! Feedback taken. There are many services (not from Google) built on top of FCM that will do it for you though. You may want to invest in one of those.
2. Firebase Auth is free, and one of the reasons it's offered for free is because it's not a database. If it had to support querying and arbitrary user data it would be a much different product. That said I totally agree with you about simplifying the connection between RTDB/Firestore and Auth. We could make that easier!
3. That's good feedback and we're definitely looking into ways ot make your users' Firebase Auth identity better connected to other Firebase products.
4. Honestly we messed up. We added it to the SDK/API surface because we wanted to avoid a breaking change later and way back in 2016 that seemed like a good idea. We were wrong and there is basically no good reason to use GeoPoint today, I'd avoid it and just use numeric fields. I can say though that Puf and I are working on some utilities to make geo hashing easier!
5. This video explains how queries work in Firestore and why the query you described can't be implemented and still satisfy our performance guarantees (queries must scale with the size of the result set)

Currently this is nor possible at a collection level, although you can see global read/write rate and security rule evaluations in Cloud Monitoring. We're working on some tools to make it easier to find out if your database has any performance hotspots though, as we realize developers need this insight.

That's kinda hard to say without knowing a little more about the searches you're performing, but I will say:

1. If you're performing frequent searches, expect to have frequent document reads. That's just going to be a fact of life
2. Consider adding limits to your queries and paginating your results -- make sure your client isn't retrieving more records than your users actually need. This can be especially important when the size of your collections grow.
3. I'm assuming you're asking because of cost, and my general advice here is to beware prematurely optimizing for price. I've seen a lot of really gnarly code on StackOverflow where developers have spent hours creating unstable solutions that are really hard to debug in order to save, like, 10 cents a day. Obviously, you want to avoid doing really bad things like reading in an entire collection when you don't need to, but remember that your time costs money too!

SPOILER ALERT Post contains unreleased Firebase features :-)

A few that I'm personally excited about:

1. The Authentication emulator, which should make the Emulator Suite feel much more complete
2. A re-written JS SDK that is designed to support tree-shaking so modern bundlers can only include the parts you use.
3. Firestore is getting != queries very soon (in fact it's out now if you know where to look).
4. We're working on support for Swift Package Manager, so you'll have the choice of either using CocoaPods or SPM to add Firebase's iOS SDKs into your project.
5. We're also looking into implementing support for Apple's Combine framework to bring Functional Reactive Programming to Firebase on iOS.

(Peter wrote 4 and 5, he knows about all this iOS stuff I'm just an Android/Web caveman over here)

The product teams don't like it when we spoil too much for them, so I have to be somewhat vague here.

In addition to the items Sam mentioned above, I'll also add that the Remote Config team is looking to add a number of quality-of-life improvements to make sure your RC setup is easier to manage when your project gets larger and more complicated.

I'll also add that there should be some nice improvements coming to Performance Monitoring, too, so you should considering adding that into your app if you haven't yet.