Firebase Team AMA
Hello everyone, we're on the Firebase team at Google, ask us anything!
Let's talk about:
- Firebase Authentication
- Security Rules
- Firebase Hosting
- Cloud Storage
- Cloud Firestore
- Firebase Realtime Database
- Google Analytics
- Cloud Functions
- Firebase Extensions
- SwiftUI & Firebase
- and more...
We'll be answering your questions live from September 23rd, 8:30 am PDT / 3:30 pm GMT.
Is adding full text search functionality to the Firebase product suite on the product roadmap?
Here's a non-firebase question:
Impostor syndrome is one challenge developers face, what are your individual experiences with impostor syndrome, and what advice do you have for developers facing this currently?
Thanks for this AMA Frank, Samuel, Todd and Peter.
I don't use Firebase as much as I did in the past, how does Firebase fit into the current JAMstack ecosystem?
The key tenet of JAMstack is to serve as much pre-generated and CDN cacheable content as possible. Firebase Hosting's global CDN lets you do that easily with static files, or, if you do want to generate some markup on the fly, lets you easily set up a Cloud Function or Cloud Run container behind the CDN. Both Cloud Functions and Cloud Run scale down to 0 if they're not being used, but auto-scale quickly if you were to get a spike in non-cacheable requests.
Firebase also has databases like Firestore that allow you to retrieve data in realtime on the client without running your own server. The Firebase client-side JS SDKs let you easily authenticate and interact with your Firebase database and other Firebase services directly from client code, while security rules make it easy to secure your data without an additional server layer.
Is geo-search for Firestore on the roadmap?
Next to full-text-search, geo-search is a requirement A LOT of apps have, and besides the added complexity of using a third party service (such as Algolia), they're also VERY expensive. Also, because of this not being integrated into the suite I've been fighting with race conditions a lot, using transactions to mirror my data from Firestore to Algolia - please do something about that! 🙏
We've answered this in a few other comments (geo queries are popular, who knew?!?) but it's worth answering again.
With Firestore one of the core principles is that query speed depends on the number of results, not the size of the data set. Doing geo queries which are both accurate and fast is really, really hard and we have not yet invested in that. There are some tricks you can do with geohashes though and we're working on getting out some better documentation about that.
Awesome work on Firebase and Cloud Firestore. We use Firestore at Hashnode for saving and sharing article drafts. It has been working great so far. I noticed that there is a dedicated backup tab for Firebase which makes it easy to back up the data. However, there is no easy way to back up data on Firestore. Is it in the roadmap, or maybe I am missing something 🤔?
This is one of the top requests for Firestore and we agree, it's super important. Backups are absolutely on the roadmap for Firestore, although we can't share a timeline. Right now the closest thing is managed export/import which can help provide peace of mind but definitely has limitations.
What are some new features that you're working on? Can you spoil some of them for us?
SPOILER ALERT Post contains unreleased Firebase features :-)
A few that I'm personally excited about:
- The Authentication emulator, which should make the Emulator Suite feel much more complete
- A re-written JS SDK that is designed to support tree-shaking so modern bundlers can only include the parts you use.
- Firestore is getting != queries very soon (in fact it's out now if you know where to look).
- We're working on support for Swift Package Manager, so you'll have the choice of either using CocoaPods or SPM to add Firebase's iOS SDKs into your project.
- We're also looking into implementing support for Apple's Combine framework to bring Functional Reactive Programming to Firebase on iOS.
(Peter wrote 4 and 5, he knows about all this iOS stuff I'm just an Android/Web caveman over here)
The product teams don't like it when we spoil too much for them, so I have to be somewhat vague here.
In addition to the items Sam mentioned above, I'll also add that the Remote Config team is looking to add a number of quality-of-life improvements to make sure your RC setup is easier to manage when your project gets larger and more complicated.
I'll also add that there should be some nice improvements coming to Performance Monitoring, too, so you should considering adding that into your app if you haven't yet.
Hey, folks! We gotta run; Wednesday is my "too many meetings" day, and I know Sam, Puf, and Peter had to run off to other meetings as well.
Thanks to everybody who asked us questions! Always good to see such enthusiasm from the Firebase community, and hopefully we can do this again sometime...
Why is FCM so difficult to use? The token management logic needs to be written by the developer, which is fine but the APIs aren't very friendly to use. For example, the group notification APIs do not have a DELETE group API node till date, the way it works now as per the docs is that at least one token needs to be passed in order to create the group, and essentially when all tokens are removed the group deletes internally. This is bad design generally because let's say I'm coding a backend which has messaging lists so that I can send a notification to a group of users simply by sending to that list--I need to add a minimum 1 user with a valid token in order to even create that list and store it's reference for sending messages later, which is bad design. This is also bad because sometimes a developer may LOSE a token generated in the past and can then NEVER delete that group--I faced this when I was initially designing the backend was making test requests without storing the token and only later realised how FCM manages it so I have residual groups that I cannot delete even thou I have the group's reg ID. On top of that, despite creating the group, the token updates are still to be managed by the developer i.e. when a token expires that group continues to have that token despite expiry rather than it internally updating to whatever newer token was issued.
Why doesn’t firebase auth allow extra meta data by default? I understand there is a technique to set a flag marking a user's role internally using "custom claims" but this isn't the same as setting extra custom fields such as city or t-shirt size, etc. A developer needs to create a separate database collection for managing their users and there isn't even a way to mark a reference of the database entry to the authentication table so that I can jump from auth list to this user details entry, making it difficult. Plus the whole technique is kind of counter-intuitive since I now essentially have 2 user tables.
Combining points 1 and 2, it would be nice to see FCM token management done internally by the firebase auth libraries for various platforms... since when Firebase provides Auth and FCM why not the features where they work together as well? Would help SOOOO many developers if when using both Auth and FCM, the auth library automatically could take up the fresh token during authentication process and automatically update the token internally on token expiry too. This would make the self-managed FCM to firebase-managed making life easier for so many devs.
Why does GeoPoint datatype on Firebase Firestore suck? I am unable to do comparisons as it internally only compares the latitude value alone. There was literally an issue raised in the NodeJS SDK for firebase and the final solution was to avoid using GeoPoint altogether and custom write logic for GeoPoint comparisons or use Geohashes (again custom coded by developer).
I believe the above GeoPoint limitation exists because of the same query limitation—“Queries with range filters on multiple fields are not supported.”? Why do you have such a limitation on Firestore? Any commercial or large product will have to query multiple fields so forcing single field conditions renders firestore not viable for production-level apps. Am I right?
Firstly, I just wanted to say how much I love Firebase and how grateful I am that Google hasn't sent it to the infamous graveyard where other beloved Google products have sadly gone. I've been using Firebase almost exclusively for the last four-and-a-half years.
I apologise for being greedy, but I have a few questions. Hopefully nothing too complicated that stumps your brilliant minds.
Will there ever be a solution to the massive cold start issues with Firebase Functions? I realise they use Google Cloud Functions under-the-hood, but I've found unless I continually ping my functions to keep them warm, the startup time can be brutal.
Are there any plans to add a cron feature into the Firebase interface? I know you can create crons through Google Cloud, but integrating them into Firebase can be tedious. I would love to be able to trigger cloud functions, perform actions in Firestore (clean old orphan data) and whatnot from Firebase Crons.
GraphQL. Any plans to explore the possible use of GraphQL natively within Firebase? I've written GraphQL API's using Firebase Functions before, but I'd love to be able to map my Firestore database to a GraphQL layer of some kind without having to maintain it myself.
Will you consider adding in the ability to configure caching? Right now, querying data in Firestore (even if it hasn't changed) results in being charged a read. It would be awesome if I could configure a cache strategy (document-by-document basis, even better) where I can specify content is cached for X amount of time (30 minutes or something) and any reads of cached data would not be counted towards usage.
Any plans to introduce a non-NoSQL database option, something more akin to a traditional RDBMS like Postgres? Firestore is great, but it's not a perfect fit for all use-cases.
Will pricing around Firestore ever be improved? Cost-wise, Real-Time Database is more cost-effective than Firestore. Perhaps considering a model which considers the storage size used of your documents and overall Firestore instead of going so heavy on read and write cost would be worth considering.
Thanks for the Firebase love! Ok I'll try to give each question a short answer:
- The team is working hard to improve cold starts right now but they'll likely always be an issue. Cloud Run is the future here, it has min instances and concurrency.
- We already have scheduled functions!
- No plans to invest in GraphQL right now, although we do notice the enthusiasm.
- Probably not that exactly but we are working on a way for Firestore developers to cache results to common queries.
- Nothing in the works right now!
- Probably not going to change, it's likely Firestore will always charge based on operations. I wouldn't say RTDB is always less expensive though. If you have a large data set (hundreds of GBs) you're going to pay much much less for storage on Firestore!
I am building a web app using React+Redux+Firebase. Problem is when I do logout, it doesn't remove all realtime database listeners and causes some permission warnings because the user is not allowed to read/write the database when he's not logged in.
How do I remove all listeners of the realtime database at once? Can it be a reason of memory leak?
How to minimize document reads in a collection if you have frequent multi item searches?
That's kinda hard to say without knowing a little more about the searches you're performing, but I will say:
- If you're performing frequent searches, expect to have frequent document reads. That's just going to be a fact of life
- Consider adding limits to your queries and paginating your results -- make sure your client isn't retrieving more records than your users actually need. This can be especially important when the size of your collections grow.
- I'm assuming you're asking because of cost, and my general advice here is to beware prematurely optimizing for price. I've seen a lot of really gnarly code on StackOverflow where developers have spent hours creating unstable solutions that are really hard to debug in order to save, like, 10 cents a day. Obviously, you want to avoid doing really bad things like reading in an entire collection when you don't need to, but remember that your time costs money too!
Is there any easy way to profile firestore writes/reads? At leases get a breakdown by collections?
Currently this is nor possible at a collection level, although you can see global read/write rate and security rule evaluations in Cloud Monitoring. We're working on some tools to make it easier to find out if your database has any performance hotspots though, as we realize developers need this insight.
Please share roadmaps for the following data stores:
1) Realtime Database (RTB) 2) Cloud FireStore (CFS)
We have used both in our projects, but biased towards Cloud FireStore. CFS appears to get more press & feature updates. We have also reviewed comparisons here .
Would be good to know feature roadmap on RTB and CFS and a presentation around them during the event. Would be good to outline the best use cases for each.
This will help us, and other customers make the right architecture decisions.
Ramesh | srclogix.com
First off: thanks for liking our databases. :-D Both Realtime Database and Cloud Firestore allow for a wide variety of use-cases. If you want to know which works best for a specific use-case, I'd recommend using the database recommender you already found. Oh dear, that "vs" in the URL makes it sound like a battle. :)
While it's true you'll have heard more about Firestore in recent years, that doesn't mean we're not working on Realtime Database anymore. In fact, we added an increment operator earlier this year, and have at least three new features in the works. If you'd like to stay up to date on things we're working, sign up for our alpha program.
Do you plan to support LinkedLists? In FireStore? Would be good for managing ordered lists and improve sorting.
Another way to order lists by a specific criterion is to add an artificial sorting key. Let's say you want to enable users to re-arrange their to-do items into the order they plan to work on them (which might not be the same as the task priority). By adding an "order" field and populating this according to the order the user arranged their tasks in, you can implement this use case with Firestore's built-in capabilities.
Can we host server side rendered react apps or next.js based apps on firebase hosting?
Firebase Hosting on its own can only serve static resources, and won't interpret any code in the files that it serves.
David East did a video series on this a while ago, and while it may be a bit outdated I'd still recommend checking it out.
Do you guys think about adding more searching functionalities to firebase storage, such as name or date quering?
No work is currently being done on adding querying capabilities to the Cloud Storage SDK for Firebase, as it just doesn't fit very well with the core functionality of that product. I'd recommend storing the metadata you want to query on in a database (like Firestore or Realtime Database) and then performing your searches there.
Oops! I have another one.
Question about Firebase and GCP.
How is Firebase different from GCP?
Are there some duplicate services between the two platforms?
When to choose Firebase over GCP?
The short answer is that, underneath the hood, Firebase IS a GCP product. When you create a Firebase project, Firebase creates a GCP project for you behind the scenes. So services like Cloud Firestore and Cloud Storage are powered by those same corresponding cloud products -- there really isn't a difference between them. And so if you wanted to go ahead and add in some other GCP services to that project, you can go ahead and do that.
That said, Firebase provides some additional easy-to-use client libraries to access these, along with access to other services that aren't really part of Google Cloud like Crashlytics, Remote Config, and GA.
What would be the best way show data from multiple projects on a single screen (eg. DAU, day 1 retention) ?
Right now, probably your best option is to export your data into BigQuery and then import it from there into a tool like Looker, Tableau, or Data Studio. That said, I know the GA team is working on some ways to more easily get at your data programmatically so there might be some more clever solutions you'll be able to build in the future.
Firebase RTDB can experience an extended 100% load event, lasting upwards of 15-30 minutes during intensive use. While this is occurring, its operations per second drops considerably, resulting in a backup of pending tasks, and further exacerbating the issue.
Why is Firebase slowing down while at 100% load instead of continuing to operate at max capacity?
Firebase's Realtime Database essentially operates as a single threaded process for each database instance. So if you have one long-running operation on the database, that blocks the server from processing any other operations that are going on on that same database.
That's why we recommend performing maintenance in smaller operations, instead of big bulk updates. In fact, that's one of the first things our support team looks for when we are helping developers troubleshoot: are there any periodic processes running that either read the entire database, or update large sections of the database in one go. Splitting such operations up is often the fastest approach to solve the problem.
If you have an operation that needs to inspect the entire database, you can also consider enabling automatic backups of your database, and then using that backup as a read-only copy to perform the analysis. That way you can read the database, without it blocking other operations. This is how you can often optimize maintenance operations too: determine what nodes need to be updated by reading from the backup, then perform a number of (not too big) write operations to update the production database.
Is there any plan that Firebase RTDB will be replaced by Firestore?
Oh, you probably wanted more info, didn't you?
Okay… no, I think there will be situations where RTDB will always be a better solution. The Realtime Database is optimized for situations where you want small chunks of data that's updated frequently (perhaps to power a poker game or something), where as Cloud Firestore is optimized for larger chunks of data that are updated less frequently. (Like, say, a blog post.)
I'm not sure if there will ever be a day where one database can serve both or your needs, and we have lots of customers happily using each one of them, so I think we're happy to continue running two NoSQL databases for the foreseeable future.
What is firebase's worst tech debt or design decision that you most regret in the backend?
With 18 products that's so hard to say! I can definitely say we wish some of our earlier services were deployed more globally (like Cloud Firestore). So many developers want Realtime Database or Firebase Hosting to be served from near where they live or where their users live. But we're definitely looking at ways to solve that!
Question for Google Analytics for Firebase.
The features you guys provide with Firebase analytics is just mind-blowing 🤯. However, some of the features are shallow. Example - Retention and cohort charts within GA for Firebase.
Is there something concrete on the roadmap to make GA for Firebase as good as the normal GA or even better?
And also, the whole GA thing with Firebase is so confusing. There are so many GAs now.
Well, these days, any Analytics project you hook up to Firebase is really a GA "App + Web Property" underneath the hood, so if the answer you're looking for isn't in the Firebase dashboard, I might recommend heading on over to the GA site and seeing if it's there. There's a number of reports that are exclusively available on GA, so you might be able to find what you're looking for. The team did a pretty nice video about this recently.
And also, the whole GA thing with Firebase is so confusing. There are so many GAs now.
As it so happens, I wrote a blog post all about this! (Sorry it's not on hashnode)
about a month ago or so, fireSQL was launched with limited queries allowed. my first inclination was that with this new possibilities/feature some SQL power will be present ; is there any plan to address the count thing for collection anytime soon?
How has it been working at firebase after getting acquired? Did anything [culture etc] change?
Firebase has always been built on a combination of technology, team, and community. That hasn't changed since we founded the company, but of course the scale has changed a bit in that time. :)
We've always had unusual holidays and traditions at Firebase, and luckily that is still possible at Google. We still celebrate Bring Your Pineapple To Work Day each year, which has turned from a 4-person inside joke, into a global annual holiday. And some things we do today would not have been possible before we joined Google. I'm of course thinking of the Firebase Summit there, but I'm quite sure everyone on our team has their own favorite Firebase moment and tradition.
Thanks for asking! 🔥🎉
Having trouble deploying react apps that use the webpacks and DIST folder. Can you guide me through the steps once the app is built and runs locally? I'm just not sure how to deploy the project (github, heroku, doesn't matter as long as its free) using the generated DIST folder contents. THANKS IN ADVANCE and see you on the 23rd.
Think this has probably been asked before, but I think it would drastically improve adoption of both Flutter and Firebase as a whole.
My questions may be dump but will like to help me more on structuring:
The limitation for Index entries for each documents being 40,000 limits us with the number of fields we can have limitation on the number of fields we can have in a document, what should we do in that scenario?
How does firebase distribute user's DB data across multiple servers to achieve scalability? LevelDB? Or Is it using sharded mysql with some raft/paxos consensus similar to dynamo? Or is it some internal proprietary solution?
In dealing with a firebase RTDB issue, where we experience frequent 100% load spikes, we have exhausted all the recommended debugging / resolutions strategies, including utilizing the profiler, Cloud Monitoring, DB sharding, query optimization, etc..
How can we scale firebase RTDB to meet our needs, or is it possible that we should migrate to another service? What are the hard limits on scalability for firebase?
Each Realtime Database instance is essentially running as a single-threaded process on our servers. The load that you see in the usage chart is an indication of how busy that thead is. This means that if you hit 100% sustained load, there's not much more you can do than reduce the amount of work you make the database do.
To scale the Realtime Database beyond that, you have to shard your data over multiple instances. This is one of the big differences between Realtime Database and Firestore: the latter handles scalability automatically for you, while with Realtime Database you'll have to think of it more yourself.
Coming up with a sharding strategy is something best done early in your data modeling phase, typically around the same time when you start thinking about how to secure access to the data. If you have a specific use-case and are wondering how to shard it, I'd always recommend posting to our firebase-talk mailing list as our team loves to think along with those.
This question is regarding Firebase Rewrite for CloudRun and Architetcing Firebase hosting with cloud run region with Firestore region (while discussing with Cloud run developeres I got this) :
we understand that Firebase Hosting originates in us-central1 and CDN available across all regions, the problem is how to architect if we are based in India location
firebase hosting(locked to us-central1 ) +cloud run(which region we should select?)+Firestore(FB app created in MUMBAI (asia-south1))?
Request path Scenario 1:
user/you(India)→India(CDN)→USA(Firebase us-central1)→ Singapore/Tokyo(CloudRun+aync call to Firestore India)→USA→CDN→you
which is REALLY BAD in terms of latency how to deal with it?
*I can't move or change / recreate my firebase app to us-central1 location.
1 possible solution is to use replace cloud run and build static sites by replacing firebase firestore SDK to cloud function with caching with a static site but we really need a SDK level firestore CDN solution to avoid this locked us-central1 firebase hosting us-central1 lock
another option is we need Firebase Hosting to originate in all 3 zones such that each zone have a separate firebase instance for using firebase hosting with rewrites to cloud run! otherwise cloud run users have to move to another local region hosting providers or move out of cloud run :(
Also cloud run SINGAPORE (asia-southeast1) region doesn't have rewrite access to firebase hosting please resolve this
Credits: @ahmetb for explaining Request path Scenario :)
Yep this is pretty bad and we know it. The short answer is that there's not much you can do about it now and you may want to avoid Hosting → Run/Functions rewrites if your primary user base is far from us-central. The long answer is that we want to bring the Hosting "origin" to more places and the team is investigating how to do it!