Eklemis Santo Nduneknotes.app·Nov 28, 2024Streamlining Data Flow: Building Efficient Data PipelinesIn today's digital age, the term "data" is everywhere. Businesses collect information about customers, scientists gather research data, and even our smartphones generate data about our daily activities. But have you ever wondered how all this raw inf...Discussdata
Varas Vishwanadhulasparkcache.hashnode.dev·Nov 27, 2024Maximizing Spark Performance: When, Where, and How to Use Caching TechniquesCaching is a technique of storing intermediate results in memory or disk. Computing the whole data again is not needed if we are using it again in further data processing. In SPARK we do cache the DataFrame so we can use the result in next tranforma...Discuss#persist
Urvish Suhagiyaurvishsuhagiya.hashnode.dev·Nov 25, 2024Your CI/CD Pipeline on AWS - Part 3What is AWS CodeDeploy? AWS CodeDeploy is a deployment service that automates application deployments to various compute services such as Amazon EC2 instances, on-premises instances, serverless Lambda functions, or Amazon ECS services. CodeDeploy sup...Discuss·3 likesDevops
Dhruv Moradiyadhruvmoradiya.hashnode.dev·Nov 25, 2024Day 51: Your CI/CD Pipeline on AWS - Part 2 🚀☁Welcome to Part 2 of building your CI/CD pipeline on AWS! Yesterday, you set up AWS CodeCommit as your version control system. Today, we’ll dive into AWS CodeBuild, a powerful tool to compile your code, run tests, and prepare deployment artifacts. Le...DiscussAWS
Nalaka Wanniarachchibidiaries.com·Nov 24, 2024Understanding the Difference: "Wait on Completion" vs. "Parallel Execution" in Microsoft Fabric Pipelines / ADFWhen designing pipelines in Microsoft Fabric or Azure Data Factory (ADF), understanding the execution flow of activities is critical. One setting that often causes confusion is "Wait on Completion"—especially in how it differs from parallel execution...Discuss·26 readsFabricADF
Avdhesh Varshneyavdhesh-blogs.hashnode.dev·Nov 23, 2024Text Representation in NLPIntroduction What is Feature Extraction from text? To text representation To text recognition Why do we need it? Why is it difficult? What is the core idea? What are the techniques? OHE (One Hot Encoding) BOW (Bag of Words) ngrams TfIdf ...Discussnlp
Ali Iqbaljenkins-route.hashnode.dev·Nov 23, 2024Jenkins Mega ProjectBuilding a project having state of art technologies with 6 different stages and code having more the 15 components. Data Flow Diagram Tech stack used in this project: GitHub (Code) Docker (Containerization) Jenkins (CI) OWASP (Dependency check)...DiscussJenkins
Urvish Suhagiyaurvishsuhagiya.hashnode.dev·Nov 23, 2024Setting Up Your CI/CD Pipeline on AWS - Part 1 : Introduction to CodeCommitWhat is CodeCommit? CodeCommit is a managed source control service provided by AWS. It allows developers to store, manage, and version their source code and artifacts securely and at scale. Here are some key features of CodeCommit: Git Support: Code...Discuss·2 likes#90daysofdevops
Snigdha Chaudharisnigdhachaudhari.hashnode.dev·Nov 21, 2024Building a Complete CI/CD Pipeline for EKS Deployment with Jenkins and DockerModern application delivery requires a seamless CI/CD pipeline to ensure code moves from development to production with minimal manual intervention. In this blog, I'll walk you through a DevOps project I recently worked on, where we implemented a mul...Discuss·101 readsmultibranch
Amit singh deoraamitsinghs.hashnode.dev·Nov 21, 2024Day 26: Jenkins Declarative Pipeline Task – Completed! 🎉🚀 Today’s Task: Exploring Jenkins Declarative Pipeline – the backbone of modern CI/CD practices. What is a Pipeline? A Pipeline in Jenkins is a collection of steps or jobs, arranged sequentially to automate building, testing, and deploying your app...DiscussPipeline
