Feed
Beautiful article Victor. Well-done. There is a small question I have with your design though. And that is: why did you use lambda for processing a CSV, especially one which seems like it was heavy. Didn't Lambda timeout after some seconds? Why not use EC2 and a CRON job and stream this file from S3. You could have used eventbridge to start and stop this instance if you are worried about it running all the time and racking up cost.
Emenike Jane
Even as a non-techie, I could feel the stress of that “too many connections” moment! I love how you broke everything down so clearly (and with humor too). It’s amazing to see how much thought goes into making these systems work smoothly behind the scenes. Proud of you for always finding smart solutions and for helping others avoid the same headaches!