Mehul Kansalmehulkansal.hashnode.dev·Oct 23, 2024Week 20: Real-Time Data Processing with Databricks Autoloader ⏳Hey data enthusiasts! 👋 Spark Structured Streaming provides an efficient framework for processing streaming data in real time, while Databricks Autoloader simplifies the process of ingesting streaming data from external sources. In this blog, we wil...Databricks
Rohit Paulrohitpaulscodism.hashnode.dev·Sep 24, 2024FeaturedMastering Node.js Streams with TypeScript: A Complete GuideStreams are one of the core concepts in Node.js, enabling efficient reading and writing of large datasets. Instead of loading the entire content into memory, streams allow handling data chunk by chunk, making them especially useful for working with l...40 likes·550 readsNode.jsNode.js
Freda Victorlearndataengineering.hashnode.dev·Sep 19, 2024Building a Recommender System on AWS: Batch and Streaming PipelinesIntroduction Recommender systems have become essential to e-commerce, helping customers find products they’re interested in, ultimately improving the customer experience and driving sales. In this blog, we’ll guide you through implementing your recom...AWS
Bobur UmurzokovforGlassFlow Blogglassflowblog.hashnode.dev·Aug 22, 2024Real-time Generative Feedback Loop Automation with GlassFlow, Supabase, and WeaviateGenerative Feedback Loops (GFL) are a powerful technique where the results generated from language models like GPT are vectorized and saved back into a vector database. This blog post explores how using GlassFlow, Supabase, and Weaviate can improve h...393 readsUse CasesAI
Priyanshu Vermapriyanshuverma.hashnode.dev·Aug 20, 2024Fluvio - An open-source In-motion data framework.Hello enthusiasts, Today we are going to learn about Fluvio work with this technology. This post is not about theory it's a practical based article. You can checkout Fluvio's own blogs to learn about fluvio deep. Why to read this article? Yes, you ca...1 likeFluvio
Piyush Kumarpiyushcv.hashnode.dev·Aug 7, 2024Streaming Data in Chunks: A Comprehensive GuideStreaming data in chunks is a powerful technique for managing long-running processes and delivering real-time updates to clients. By sending data incrementally while maintaining an open connection, you can keep users informed and engaged without wait...Data Chunking
Harshit Tripathiharshit11.hashnode.dev·Jul 25, 2024Kakfa quick start using DockerApache Kafka, a distributed streaming platform, is a powerful tool for building real-time data pipelines and streaming applications. In this tutorial we will be using docker to run kafka cluster and create consumer to write. Prerequisites Make sure d...kafka
Mahidhar Mullapuditutorialq.hashnode.dev·Jun 28, 2024Building Scalable Applications with Kafka and Reactive ProgrammingIntroduction In today's digital world, the ability to process data in real-time and scale applications efficiently is crucial for success. Kafka, a distributed event streaming platform, combined with reactive programming, offers powerful tools to ach...8 likesReactive Programming
Cloud Tunedcloudtuned.hashnode.dev·May 22, 2024Key Questions to Ask When Selecting a Data Streaming Solution for a New ProjectKey Questions to Ask When Selecting a Data Streaming Solution for a New Project Selecting the right streaming solution for a new project is essential for handling real-time data processing, analytics, and event-driven architectures effectively. From ...data streaming
Sumit Mondalsumit007.hashnode.dev·Mar 23, 2024A Beginner's Guide: Implementing Amazon MSK in AWSIntroduction: Amazon Managed Streaming for Apache Kafka, commonly known as Amazon MSK, is a fully managed service that makes it easy to set up, operate, and scale Apache Kafka clusters in the AWS Cloud. If you're new to Amazon MSK and looking for a s...AWS - HandsOnAmazonMSK
