SKShivam Kumarinsavvyshivam.hashnode.dev·Dec 4, 2024 · 2 min readAWS Glue TroubleshootingNotes: Read each line of problem statement S3-Glue-Redshift Setting Up Redshift cluster and connection in AWS Glue create cluster → Node type ( dc2.large) → No of nodes (1) Database Configurations- Manually add admin password → awsuser , Awsuser1 Clu...00
SKShivam Kumarinsavvyshivam.hashnode.dev·Dec 1, 2024 · 2 min readML Pie chartimport boto3 import pandas as pd import matplotlib.pyplot as plt import seaborn as sns import joblib import tempfile from sklearn.compose import ColumnTransformer from sklearn.preprocessing import OneHotEncoder, StandardScaler from sklearn.feature_se...00
SKShivam Kumarinsavvyshivam.hashnode.dev·Dec 1, 2024 · 2 min readtypes of feature engineering1. One-Hot Encoding import pandas as pd # Sample DataFrame df = pd.DataFrame({'Animal': ['Cat', 'Dog', 'Fish']}) # One-Hot Encoding df_one_hot = pd.get_dummies(df, columns=['Animal'], dtype=int) print(df_one_hot) Output: Animal_Cat Animal_Dog...00
SKShivam Kumarinsavvyshivam.hashnode.dev·Dec 1, 2024 · 1 min readML troubleshooting# Cloud driven Loan Defalut predictor using machine learning ## Task 1. Launching an Amazon sage make instance # Task 2. upload note book into jupiter # Task 3. Data Loading import boto3 import pandas as pd import numpy as np from sklearn.model...00
SKShivam Kumarinsavvyshivam.hashnode.dev·Dec 1, 2024 · 3 min readconfigure python file in vs codeRead data from pyspark.sql import SparkSession from pyspark.sql.types import StructType, StructField, StringType, IntegerType, FloatType def read_data(spark, customSchema): Step 1: Define the S3 bucket and file location bucket_name = " loa...00
