James NguyenforCollate Blogblog.getcollate.io·Jan 21, 2025Managed OpenMetadata from Collate: 2024 Year in ReviewThroughout 2024, Collate made incredible progress bringing new capabilities to our customers and to the OpenMetadata open source community. We’ve shipped new features and improvements to accelerate AI automation for data discovery, observability, and...data discovery
Stephen David-Williamsstephendavidwilliams.com·Dec 30, 2024Use data contracts to automate data workflows - part 2Preface📖 In part 1, we explained what a data contract is why we need them, and what a typical one contains In this blog, we dive into a demo to explore how they actually work so we can process data safer, faster and effectively. Goal🎯 The data...data-engineering
pallavi chauhaninnovateitworld.hashnode.dev·Dec 27, 2024Essential Skills for Aspiring Data ScientistsIn the modern data-centric landscape, data science has become one of the most in-demand professions. Organizations across various sectors are utilizing data to make smarter decisions, streamline operations, and stay ahead of the competition. Conseque...Data Science
pallavi chauhaninnovateitworld.hashnode.dev·Dec 25, 2024The Impact of Data Science on Supply Chain ManagementIn today’s fast-paced business environment, the integration of data science into supply chain management has proven to be transformative. Organizations across various industries are harnessing data-driven insights to enhance efficiency, reduce costs,...Data Science
Stephen David-Williamsstephendavidwilliams.com·Dec 24, 2024Use data contracts to automate data workflows - part 1Preface 📚 I’ve actually written a post on data contracts before, so have a quick scan here if you want to see a project I created on them using Python, AWS S3 and other libraries (like Selenium and Soda). What is a contract? 🤔 Let’s first talk abou...28 readsdata-engineering
Anuj Syalanujsyal.com·Dec 8, 2024FeaturedMastering Data Quality in ETL Pipelines with Great ExpectationsIn the world of data engineering, ensuring data quality is paramount. From business analysts relying on dashboards to C-level executives making strategic decisions, and data scientists training machine learning models — everyone depends on the qualit...4 likes·77 readsData Science
Raghuveer Sriramanraghuveer.me·Nov 24, 2024Writing data quality tests in DataformReliable, accurate data is the foundation of data-driven decision-making. Poor data quality can lead to incorrect insights, a lack of trust in data and data products. It really is quite an obvious statement to make, but what’s not obvious is how exac...Practical guide to build data pipelines with Dataformdataform
Piotr Czarnasdqops.hashnode.dev·Nov 23, 2024Data Architecture for Data QualityThe purpose of data quality validation Data quality validation is the process of ensuring that data is accurate, complete, and suitable for its intended use. Just as a baker checks the freshness and quantity of ingredients before baking a cake, busin...data-quality
BuzzGKbuzzgk.hashnode.dev·Nov 7, 2024Data Quality Checks TechniquesEnsuring the accuracy, completeness, and reliability of data is crucial for making informed decisions and maintaining the trust of stakeholders. Implementing data quality checks using SQL and orchestrating them with tools like Apache Airflow can help...data-quality
Vipinvipinmp.hashnode.dev·Oct 26, 2024Building a Data Quality Testing Framework Using Snowflake and SQLIn the era of data-driven decision-making, maintaining high data quality is crucial. Poor data quality can lead to incorrect insights, impacting business decisions. This blog post will guide you through building a robust data quality testing framewor...37 readsE2E Projectsdata-quality
