Sharath Kumar Thungathurthisharaththungathurthi.hashnode.dev·Nov 14, 2024Managed vs External TablesIn an interview, questions about managed vs. external tables in PySpark are likely to focus on concepts, practical applications, and potential scenarios where one is preferable over the other. Here are some areas to prepare for: 1. Definition and Dif...DiscussPySpark
KAPUPA HAAMBAYIdatasmithery.hashnode.dev·Nov 5, 2024Proactive Manufacturing with Data VisualisationAs a data engineer, I see data visualisation not as a stand-alone solution but as a vital part of data engineering, where raw data is transformed into actionable insights. This is especially true in manufacturing, where efficiency, speed, and accurac...Discuss#manufacturing
Alex Mercedalexmerced.hashnode.dev·Oct 31, 2024Hands-on with Apache Iceberg & Dremio on Your Laptop within 10 MinutesFree Copy of Apache Iceberg the Definitive Guide Free Apache Iceberg Crash Course Iceberg Lakehouse Engineering Video Playlist Efficiently managing and analyzing data is essential for business success, and the data lakehouse architecture is leading ...Discussdataengineering
KAPUPA HAAMBAYIdatasmithery.hashnode.dev·Oct 29, 2024Digital Transformation in Energy: A Blueprint for Precision with Azure Data EngineeringI know what you’re thinking: What do oil refining and manufacturing have in common beyond large plants and heavy machinery? You might also only think of manufacturing as conveyor belts and assembly lines with identical items moving from machine to ma...DiscussAzure
Arpit Tyagidataminds.hashnode.dev·Oct 24, 2024Important Business Use Case - Solved by ADF (Azure Data Factory)“Use Case”: Whenever we have more than 100 records in the customer table, we copy the customer data to another table customer_copy within the SQL DB. However, whenever we do this copy, we first truncate the table 'customer_copy' and then copy the dat...Discuss·10 likesAzure Data FactoryAzure
Shreyash Banteshreyash27.hashnode.dev·Oct 23, 2024Understanding Hierarchical and Network Data Models: Structure, Benefits, and Use CasesHierarchical Data Model Overview: The Hierarchical Data Model organizes data in a tree-like structure where records have a parent-child relationship. This model is best suited for situations where data naturally fits into a hierarchy, such as organiz...DiscussData Science
KAPUPA HAAMBAYIdatasmithery.hashnode.dev·Oct 22, 2024A Day In The Life of a Super Azure Data EngineerWhen I was starting out my data engineering journey, I often imagined what it would be like to work as a great data engineer, especially in a fast-paced, data-driven environment like manufacturing. Fun fact*: I actually worked for a manufacturing com...Discuss#techinmanufacturing
Ekemini Thompsonekeminithompson.hashnode.dev·Oct 13, 2024Day 1: Tasks for Aspiring Data Scientist, Data Engineer, and Cloud EngineerDay 1 for Aspiring Data Scientist: Introduction to Data Science and Python Setup Objective: Kick off your data science journey by understanding the basics of data science and setting up Python, your primary tool for data manipulation and analysis. ...Discuss·11 likes·40 reads30 DAYS of Internshipdatascience
Ekemini Thompsonekeminithompson.hashnode.dev·Oct 13, 2024Day 8: Tasks for Aspiring Data Scientist, Data Engineer, and Cloud EngineerDay 8 for Aspiring Data Scientist: Feature Engineering Objective: Learn about feature engineering and its importance in improving the performance of machine learning models. You will explore techniques for creating and selecting features from existi...Discuss30 DAYS of InternshipData Science
Ekemini Thompsonekeminithompson.hashnode.dev·Oct 13, 2024Day 7: Tasks for Aspiring Data Scientist, Data Engineer, and Cloud EngineerDay 7 for Aspiring Data Scientist: Data Preprocessing and Cleaning Objective: Learn how to preprocess and clean data to prepare it for analysis or machine learning models. You will use Python libraries like Pandas and NumPy to handle missing data, d...Discuss30 DAYS of InternshipData Science
