Being a Software Developer feels great, but every software developer is scared of a Production bug.
Production bug will keep haunting you until you resolve it.
I am an Android Developer and I would like to share my production mistake story where a little tiny bug made a huge impact.
Our company has developed an Internet Speed Test applications (for Mobile and TV). I got this project for further development and to improve the speed test algorithm. I started converting all Java classes into Kotlin, restructuring the app, and optimizing the algorithm for the 5G network speed test. After a few days app was ready for testing. Testing was successful with a few minor bugs which I resolved eventually and then we decided to update the new version on the play store. I was excited to see this app live.
After it went live 1st day was normal. No velocity bugs in firebase but on the 2nd day we received a mail from the user about this weird bug. "when I clicked on the START TEST button nothing happens and the speedometer stays quiet, it's not moving". Got some similar comments from other users. I checked the Firebase Crahlaytics but no sign of this weird bug and on the play store the negative comments started to increase.
This is the moment where the developer will go mad when there is no clue of the bug source and it is already in production. I knew I have made a critical mistake so I started the Root Cause Analysis. After running the app on several devices, several different Android OS versions, over VPN with Wifi / Cellular data connection... NO LUCK. The app was working well in my testing.
still tracking the bug...Now it's been 4 days already and I still didn't manage to track this bug. I left the Android Studio window hopelessly and started to read users' negative comments. (It was very disappointing) but while reading the comments I found the pattern that most of the negative comments are from European countries e.g. Germany, France. I started to think why would users from only these countries face this problem. After some analysis and my code review I got the answer, they follow the different decimal separators i.e. comma ','.In my country (in India) we use the dot "." as a decimal separator.
e.g. fifty six-point thirty
- India - 56.30
- Germany - 56,30
The Bug:
- Exception occurred while formatting double from string because no Locale was specified. If the user sets his/her native locale to the device and if it is following decimal separator as comma then an exception will arise
- Why exception was not caught by Crashlytics? because it was handled and printed in logcat (stupid practice).
The Solution: All I had to do is change the following code
fun Double.roundToTwoDecimal() = "%.${2}f".format(this).toDouble()
to this
fun Double.roundToTwoDecimal() = "%.${2}f".format(Locale.US, this).toDouble()
I need to specify the Locale.US while formatting the number.
I know you might say what a dumb mistake it was. Yes, it is. But it was an eye-opener for me how these dumb or tiny mistakes can cost you. Hope you found this story interesting and if you also made some dumb mistakes in your production then let me know in the comments.
The apps I mentioned in the story are
- for mobile: 5G Speed Test – Internet Speed Testing
- for TV: Internet Speed Test-FiberTest for Android Smart TV
This is my first article so kindly enlighten me with your feedback. Thank You.