--- title: "AATISH SINGH (@aatishintodata) | Hashnode" description: "Bigdata Developer| DataEngineer| Hadoop| Spark | Mapreduce| Hive| Sqoop| Sql | Scala| 4+ years of Experience| Amdocs" url: "https://hashnode.com/@aatishintodata" --- @aatishintodata # AATISH SINGH @aatishintodata ยท Pune,India ยท Joined July 2023 Bigdata Developer| DataEngineer| Hadoop| Spark | Mapreduce| Hive| Sqoop| Sql | Scala| 4+ years of Experience| Amdocs ### [๐’๐ญ๐จ๐ฉ ๐ญ๐ซ๐ž๐š๐ญ๐ข๐ง๐  ๐‡๐ฎ๐๐ข ๐ฏ๐ฌ. ๐ˆ๐œ๐ž๐›๐ž๐ซ๐  ๐š๐ฌ ๐š๐ง '๐ž๐ข๐ญ๐ก๐ž๐ซ/๐จ๐ซ' ๐›๐š๐ญ๐ญ๐ฅ๐ž.](https://aatishintodata.hashnode.dev/lakehouse?utm_source=hashnode&utm_medium=feed) In production architectures, they solve two entirely different bottlenecks: โšก ๐ˆ๐ง๐ ๐ž๐ฌ๐ญ๐ข๐จ๐ง & ๐‚๐ƒ๐‚ (๐‡๐ฎ๐๐ข) ๐ŸงŠ ๐‹๐š๐ซ๐ ๐ž-๐ฌ๐œ๐š๐ฅ๐ž ๐Ž๐‹๐€๐ & ๐๐ˆ (๐ˆ๐œ๐ž๐›๐ž๐ซ๐ ) Here is how they fit toge 2 min read ยท [aatishintodata.hashnode.dev](https://aatishintodata.hashnode.dev/lakehouse?utm_source=hashnode&utm_medium=feed) [Aug 25](https://hashnode.com/posts/lakehouse/6a8dc7b4a358852c4974da59) ### [๐–๐ก๐š๐ญ ๐ข๐ฌ ๐’๐ญ๐ซ๐ฎ๐œ๐ญ๐ฎ๐ซ๐ž๐ ๐’๐ญ๐ซ๐ž๐š๐ฆ๐ข๐ง๐ ?](https://aatishintodata.hashnode.dev/spark-streaming?utm_source=hashnode&utm_medium=feed) ๐–๐ก๐š๐ญ ๐ข๐ฌ ๐’๐ญ๐ซ๐ฎ๐œ๐ญ๐ฎ๐ซ๐ž๐ ๐’๐ญ๐ซ๐ž๐š๐ฆ๐ข๐ง๐ ? Think of Structured Streaming as a way to handle data that's constantly arriving, like a river of information. Spark processes this data in small 3 min read ยท [aatishintodata.hashnode.dev](https://aatishintodata.hashnode.dev/spark-streaming?utm_source=hashnode&utm_medium=feed) [Aug 24](https://hashnode.com/posts/spark-streaming/6a8c777d449ed4a2a1a4c9d7) ### [๐Ÿ๐Ÿ ๐๐š๐ญ๐š๐›๐š๐ฌ๐ž ๐ค๐ž๐ฒ๐ฌ we all need to know](https://aatishintodata.hashnode.dev/we-all-need-to-know?utm_source=hashnode&utm_medium=feed) โ†˜๏ธ ๐Œ๐จ๐ฌ๐ญ ๐๐ž๐ฏ๐ž๐ฅ๐จ๐ฉ๐ž๐ซ๐ฌ ๐ฆ๐ข๐ฑ ๐ญ๐ก๐ž๐ฌ๐ž ๐ฎ๐ฉ ๐๐ฎ๐ซ๐ข๐ง๐  ๐ข๐ง๐ญ๐ž๐ซ๐ฏ๐ข๐ž๐ฐ๐ฌ. ๐Ÿš€ Here is a quick cheat sheet of all ๐Ÿ๐Ÿ ๐๐š๐ญ๐š๐›๐š๐ฌ๐ž ๐ค๐ž๐ฒ๐ฌ we all need to know: Core Identification 3 min read ยท [aatishintodata.hashnode.dev](https://aatishintodata.hashnode.dev/we-all-need-to-know?utm_source=hashnode&utm_medium=feed) [Aug 23](https://hashnode.com/posts/we-all-need-to-know/6a8b2530a1f48ce5dfda1471) ### [๐„๐ง๐-๐ญ๐จ-๐„๐ง๐ ๐ƒ๐š๐ญ๐š ๐„๐ง๐ ๐ข๐ง๐ž๐ž๐ซ๐ข๐ง๐  ๐๐ข๐ฉ๐ž๐ฅ๐ข๐ง๐ž ๐จ๐ง ๐ƒ๐š๐ญ๐š๐›๐ซ๐ข๐œ๐ค๐ฌ (๐…๐Œ๐‚๐† ๐ƒ๐จ๐ฆ๐š๐ข๐ง)!](https://aatishintodata.hashnode.dev/end-to-end-data-engineering-pipeline?utm_source=hashnode&utm_medium=feed) ๐‰๐ฎ๐ฌ๐ญ ๐›๐ฎ๐ข๐ฅ๐ญ ๐š๐ง ๐„๐ง๐-๐ญ๐จ-๐„๐ง๐ ๐ƒ๐š๐ญ๐š ๐„๐ง๐ ๐ข๐ง๐ž๐ž๐ซ๐ข๐ง๐  ๐๐ข๐ฉ๐ž๐ฅ๐ข๐ง๐ž ๐จ๐ง ๐ƒ๐š๐ญ๐š๐›๐ซ๐ข๐œ๐ค๐ฌ (๐…๐Œ๐‚๐† ๐ƒ๐จ๐ฆ๐š๐ข๐ง)! When two companies merge, unifying their messy, disparat 2 min read ยท [aatishintodata.hashnode.dev](https://aatishintodata.hashnode.dev/end-to-end-data-engineering-pipeline?utm_source=hashnode&utm_medium=feed) [Aug 22](https://hashnode.com/posts/end-to-end-data-engineering-pipeline/6a89e1dcb40f57d8ac78e6fb) ### [๐Ÿ“Œ ๐‡๐จ๐ฐ ๐ฐ๐จ๐ฎ๐ฅ๐ ๐ฒ๐จ๐ฎ ๐ก๐š๐ง๐๐ฅ๐ž ๐š ๐ฌ๐ข๐ญ๐ฎ๐š๐ญ๐ข๐จ๐ง ๐ฐ๐ก๐ž๐ซ๐ž ๐œ๐ซ๐ข๐ญ๐ข๐œ๐š๐ฅ ๐„๐“๐‹ ๐ฉ๐ข๐ฉ๐ž๐ฅ๐ข๐ง๐ž ๐ก๐š๐ฌ ๐Ÿ๐š๐ข๐ฅ๐ž๐ ๐ข๐ง ๐ฉ๐ซ๐จ๐](https://aatishintodata.hashnode.dev/8jtjcdwnzch8j2qqpcdklag8j2qspcdkkjwnzcu8j2qpfcdkj0g8j2qsvcdkkjwnzcuipcdkkhwnzca8j2qpcdkj3wnzcl8j2qnidwnzcaipcdkkzwnzci8j2qrfcdkk7wnzca8j2qrfcdkklwnzco8j2qpydwnzcw8j2qofcdkj7wnzcr8j2qnidwnzcc8j2qqcdkklwnzct8j2qovcdkjzwnzca8j2qpsdwnzce8j2qkcdkisg8j2qqfcdkklwnzcp8j2qnvcdkkxwnzci8j2qpcdkj4g8j2qofcdkjrwnzcsipcdkjwnzca8j2qovcdkkxwnzce8j2qnsdwnzci8j2qpydwnzcp8j2qqcdkkjwnzcd?utm_source=hashnode&utm_medium=feed) โœ”๏ธ According to me, these are the few steps you could take:-โœ… Access the failure by checking logs, error messages, and monitoring alerts to identify the root cause, like a schema changeโœ… Check how much data has been affected and also if the downstrea... 1 min read ยท [aatishintodata.hashnode.dev](https://aatishintodata.hashnode.dev/8jtjcdwnzch8j2qqpcdklag8j2qspcdkkjwnzcu8j2qpfcdkj0g8j2qsvcdkkjwnzcuipcdkkhwnzca8j2qpcdkj3wnzcl8j2qnidwnzcaipcdkkzwnzci8j2qrfcdkk7wnzca8j2qrfcdkklwnzco8j2qpydwnzcw8j2qofcdkj7wnzcr8j2qnidwnzcc8j2qqcdkklwnzct8j2qovcdkjzwnzca8j2qpsdwnzce8j2qkcdkisg8j2qqfcdkklwnzcp8j2qnvcdkkxwnzci8j2qpcdkj4g8j2qofcdkjrwnzcsipcdkjwnzca8j2qovcdkkxwnzce8j2qnsdwnzci8j2qpydwnzcp8j2qqcdkkjwnzcd?utm_source=hashnode&utm_medium=feed) [Aug 27, 2025](https://hashnode.com/posts/8jtjcdwnzch8j2qqpcdklag8j2qspcdkkjwnzcu8j2qpfcdkj0g8j2qsvcdkkjwnzcuipcdkkhwnzca8j2qpcdkj3wnzcl8j2qnidwnzcaipcdkkzwnzci8j2qrfcdkk7wnzca8j2qrfcdkklwnzco8j2qpydwnzcw8j2qofcdkj7wnzcr8j2qnidwnzcc8j2qqcdkklwnzct8j2qovcdkjzwnzca8j2qpsdwnzce8j2qkcdkisg8j2qqfcdkklwnzcp8j2qnvcdkkxwnzci8j2qpcdkj4g8j2qofcdkjrwnzcsipcdkjwnzca8j2qovcdkkxwnzce8j2qnsdwnzci8j2qpydwnzcp8j2qqcdkkjwnzcd/68af632abed5d14651d35b49)