© 2023 Hashnode
#kafka
EXECUTIVE SUMMARY Data engineers build pipelines that help companies collect, merge, and transform data to facilitate seamless analytics. They oversee the creation of an infrastructure design that enables modern data analytics. A Data engin…
Introduction Many articles talk about the benefits of using AWS SQS, and rightly so because AWS SQS is a great service! It's a highly reliable and scalable messaging service that's used by many organi…
EMQX is the most scalable MQTT broker for IoT, IIoT, and connected vehicles. It is also highly extendable to integrate with various databases and streaming platforms. EMQX Enterprise’s Rule-Engine pro…
Many of you might have come across a scenario where you wanted to install a package with a consistent version across all nodes in cloudera cluster, for example, if you want a higher python version on …
Introduction In this article, we'll demonstrate creating a simple streaming data pipeline using a small micro-batching tool and CockroachDB's CDC Kafka sink. Setup Prerequisites: CockroachDB, with a …
For the first post in the paper notes series, I will go through a paper published in 2011 about the Kafka distributed messaging system. In a bit more than 10 years, Kafka has become the de-facto infra…
What does it mean to optimize Kafka for High Availability? This means that the Kafka applications should recover as quickly as possible from failures to ensure reads and writes are always available fr…
Sometimes, when writing microservices, we often come across a dilema: Whether to use gRPC and message queues. I, literaly, have been biting the bullet making a decision about which tech to use. After …
Apache Kafka is a powerful and widely used event streaming platform that allows for the efficient exchange of large amounts of data in real time. One popular way to work with Kafka is to use the Go pr…
Kafka is a distributed streaming platform that is used for building real-time data pipelines and streaming apps. It is a powerful tool that can handle high volumes of data and allows for the storage a…
