Kamran Aliatech.guide·Aug 28, 2024From Spark to Ray: Amazon's $120MM/year Cost-Saving JourneyAmazon Business Data Technologies (BDT) Team moved EXABYTE scale jobs from Spark to this tech. 🤔 Cost saving of $120MM/year on EC2 on-demand R5 instance charges 🔥 Background It started with migration from Oracle To decouple storage and compute, th...1 like·65 readsLearn System Design from Industryspark
yash bhaskaryash9439.hashnode.dev·Mar 13, 2024Accelerating Document Embedding Generation with Ray, FastEmbed, and QdrantFor medium and large businesses, extracting meaningful insights from large volumes of unstructured data, such as text documents, is crucial. However, the traditional approach of sequentially processing documents for embedding generation can take time...10 likesDocument Embedding
Kevin SuforFlyte Blogacidic-committee-improve-52.hashnode.dev·Aug 25, 2022Ray and Flyte: Distributed Computing and OrchestrationWhether it is asking, “Why do 87% of data science projects never make it into production?” or trying to make sense of a “hot mess” called MLOps, debates around machine learning in production have never really subsided. Machine Learning research is bo...20 likes·1.6K readsdistributed system
