Anix Lynchgozeroshot.dev·Dec 6, 2024Python Automation #2: 🗳️ Data Transformation w/polars, pyjanitor, pandas, polars1. Convert Column Names to Snake Case (pyjanitor.clean_names) import pandas as pd import janitor # Sample DataFrame df = pd.DataFrame({"Column Name 1": [1, 2], "AnotherColumn": [3, 4]}) # Convert column names to snake_case df = janitor.clean_names(...pyjanitor
Arpit Tyagidataminds.hashnode.dev·Dec 2, 2024Mastering Slowly Changing Dimensions (SCD) "Type 2" with Azure Data Factory: A Step-by-Step GuideIntroduction to Slowly Changing Dimensions (SCD) Type 2 Slowly Changing Dimensions (SCD) Type 2 is a data warehousing technique used to track historical changes in dimension data over time. Unlike SCD Type 1, which overwrites old data, Type 2 preserv...Azure Data FactoryAzure
Arpit Tyagidataminds.hashnode.dev·Dec 2, 2024Mastering Slowly Changing Dimensions (SCD) Type 1 with Azure Data Factory: A Step-by-Step Guide(SCD Type 1 implementation via ADF) Step 1: Setting Up Your Azure SQL Database for SCD Type 1. Create the emp_scdtype1 table in Azure SQL Database. Step 2: Populating Your Table: Adding Initial Data Entries. Step 3: Visualizing Data: Confirming Tab...8 likesAzure Data FactoryAzure
Arpit Tyagidataminds.hashnode.dev·Dec 2, 2024Mastering DataFlow Techniques in Azure Data Factory with a Data Transformation example:Step 1: Exploring the Data Lake: Initial File Inspection Step 2: Dataflow Blueprint: A Snapshot of the Transformation Process Step 3: Connecting the Dots: Linking to Your Data Source Step 4: Filtering the Blues: Excluding Specific Data Entries St...5 likesAzure Data FactoryAzure
Arpit Tyagidataminds.hashnode.dev·Dec 2, 2024Azure Data Factory: "Join" 2 or more CSV Files and Convert to JSON FormatStep 1: Inspecting the CSV Files in Data Lake: Your First Step to Data Optimization Step 2: Configuring the Data Flow Sources: Pointing to the Customer.CSV Files and use Join tool after that. Step 3: Use Join on Customer id as that is the common fi...5 likesAzure Data FactoryADF
Anastasia Zaharievawhenmathmetdata.hashnode.dev·Nov 27, 2024Day 10: Data TransformationWelcome to Day 10! Today, we’re diving into data transformation, an essential step to prepare raw data for analysis and machine learning. Data transformation includes scaling, normalizing, encoding, and reshaping data, ensuring it’s in the optimal fo...30 Days Data Science ChallengePython
Lucy ZhangforNocoBase's blognocobase.hashnode.dev·Nov 5, 2024The Ultimate Guide to Data Transformation ToolsOriginally published at The Ultimate Guide to Data Transformation Tools - NocoBase. In the era of big data, businesses and organizations face the challenge of handling vast amounts of data. As applications become more complex and user needs evolve, d...Open Source
Md.Rejoyan Islamrejoyan.hashnode.dev·Oct 24, 2024Unlocking the Power of JSON: Tips and Tricks for DevelopersWhat is JSON? Definition: JSON is a format for representing structured data as text, using key-value pairs. Origins: Derived from JavaScript, but now language-independent. File Extension: .json Media Type: application/json History of JSON Ameri...JavaScript
Pronod Bharatiyadata-intelligence.hashnode.dev·Oct 18, 2024Discrete and Continuous Models in Machine Learning: Understanding CDF as a BridgeMachine learning (ML) models are often categorized as either discrete or continuous, based on the nature of the data they handle. Discrete models work with distinct, countable values, while continuous models operate on variables from a continuous ran...69 readsDiscrete Models
Sandhya Kondmarekloudsan.hashnode.dev·Oct 6, 2024Feature Engineering: A Key Step in Machine Learning"feature" ka matlab dataset mein woh columns hote hain jo specific information ko represent karte hain. Yani, feature kisi bhi characteristic ya attribute ko define karta hai jo aapke analysis ke liye important hai. Example ke liye: Agar aap ek datas...DataPreprocessing
