---
title: "Discussion on \"Building a Generic Data Integration Pipeline in Scala: The Data Plumber - Part 2\" | Hashnode"
description: "Discussion on \"Building a Generic Data Integration Pipeline in Scala: The Data Plumber - Part 2\". Building Resumable Data Pipelines with Redis-backed Offset Tracking and Batch Processing"
url: "https://hashnode.com/posts/building-a-generic-data-integration-pipeline-in-scala-the-data-plumber-part-2/67d8735d9cf82b4956a1169d"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Building a Generic Data Integration Pipeline in Scala: The Data Plumber - Part 2](https://yadukrishnan.live/building-a-generic-data-integration-pipeline-in-scala-the-data-plumber-part-2?utm_source=hashnode&utm_medium=feed)

Building Resumable Data Pipelines with Redis-backed Offset Tracking and Batch Processing

Mar 17, 2025 · 6 min read · yadukrishnan.live  
Read the article

-   [#scala](https://hashnode.com/tag/scala)
-   [#dataintegration](https://hashnode.com/tag/dataintegration)

## Written by

[Yadukrishnan](https://hashnode.com/@yadukrishnan)

@yadukrishnan · Hamburg, Germany

  
[View profile](https://hashnode.com/@yadukrishnan)

Software Developer - Scala

Travel | Movies | History | Nature I am a software developer. Started my career as a Java developer, but later switched to Scala. I wish to write Scala articles which are easier for newbies to follow. I love to travel, prefer to be alone or in very small group. Love to watch movies/series.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.