Harsh Ranjanharsh666.hashnode.dev·Jul 11, 2024Explanation of 'Ops' extending into specialized fieldsIn the ever-changing world of IT operations, 'Ops' has expanded into various specialized fields. Lets Explore how DevOps, DataOps, MLOps, and AIOps each play a distinct role in shaping how technology integrates and improves efficiency: a) DevOps seam...Discuss·1 likeDevops
Jesus DiazforDataShell team's blogelyisu.hashnode.dev·Jan 10, 2024Why DataShell?For several years, Matt Turck has been compiling the Machine Learning, Artificial Intelligence, and Data (MAD) landscape in an effort to make sense of this vibrant space. The image above represents the most recent version of the landscape. The divers...Discuss·400 readsdata analytics
Harshita Chaudharyharshita.hashnode.dev·Oct 7, 2023PySpark Job Optimization Techniques - Part IApache Spark stands out as one of the most widely adopted cluster computing frameworks for efficiently processing large volumes of complex data. It empowers organizations to swiftly handle intricate data processing tasks. In this discussion, we will ...Discuss·28 readsdataops
Enov8enov8.hashnode.dev·Jul 18, 2023Unlocking the Power of DataOps: A Step-by-Step Guide to Supercharge Your Data TestingIn the age of big data, organizations rely heavily on accurate and reliable data to drive their decision-making processes. However, ensuring the quality and integrity of data can be a complex task. This is where a DataOps platform comes into play. ...Discuss·1 likedataops
Mike Logaciukmlog.hashnode.dev·Jul 6, 2023DevOps: My all-in-one & must have listGit To install git type: sudo apt install git && git config --global credential.helper store git config --global user.name "your username" git config --global user.password "your password" Zsh In order to install zsh on Debian or its deratives, plea...Discuss·157 readsDevops
Andrew Sharifikiaalireza-sharifikia.hashnode.dev·Apr 28, 2023DataOps: Apache Spark - Intermediate (Part 1)Overview In a previous article, we covered the basics of Apache Spark. Now that the foundational concepts and workflow architecture of the spark are covered, we'll be exploring PySpark and its conventional practices, and implementations. PySpark has ...Discuss·38 readsDataOpsdataops
Andrew Sharifikiaalireza-sharifikia.hashnode.dev·Mar 17, 2023DataOps: Apache Airflow - BasicIntroduction Apache Airflow is an open-source platform for authoring, scheduling, and monitoring data and computing workflows. It was developed by Airbnb and is now under the Apache Software Foundation.It uses Python to create workflows that can be e...Discuss·97 readsDataOpsairflow
Josue Luzardo Gebrimjosueluzardogebrim.hashnode.dev·Mar 13, 2023Cube: Creating a Semantic Data Layer!Hello, Nowadays, it is very common to have a data environment with several solutions; in this post on Medium, I will summarize how to implement a data layer to unify access safely weighing on better governance, cost reduction, and compliance with LGP...Discuss·32 readsDatabases
Andrew Sharifikiaalireza-sharifikia.hashnode.dev·Mar 13, 2023DataOps: Apache Kafka - BasicIntroduction Technically speaking, event streaming is the practice of capturing data in real-time from event sources like databases, sensors, mobile devices, cloud services, and software applications in the form of streams of events; storing these ...Discuss·46 readsDataOpskafka
Andrew Sharifikiaalireza-sharifikia.hashnode.dev·Mar 9, 2023DataOps: Data Build Tool (DBT)Background Three key concepts of maintaining data engineering projects are: Versioning: You must always keep a history of the lineage of the data sources and data models that you use. Testing: Like every other aspect of software development, data m...Discuss·58 readsDataOpsdbt
