Manuel Schmidbauerdeltaload.hashnode.dev·May 24, 2024Building a poor man's data lake for Shopify dataInspired by a recent blog post, I decided to experiment with various technologies and build a small data lake for Shopify data. In this project, the following technologies are used: Data Ingestion:dlthub I use the dlt connector to push data from the ...DiscussPoor Man's Data LakeData-lake
Cloud Tunedcloudtuned.hashnode.dev·May 19, 2024Daily Hack #day46 - DuckDB Minimalistic Analytics DatabaseDuckDB is an open-source, lightweight database engine optimized for analytical workloads. It is characterized by its exceptional query performance, minimal resource requirements, and ease of integration into existing applications. DuckDB is particula...DiscussDaily HacksduckDB
Tobias Müllertobilg.com·Apr 16, 2024Chat with a DuckA while ago I published sql-workbench.com and the accompanying blog post called "Using DuckDB-WASM for in-browser Data Engineering". The SQL Workbench enables its users to analyze local or remote data directly in the browser. This lowers the bar rega...Discuss·519 readsduckDB
Fritz LarcoforSling Data Blogblog.slingdata.io·Apr 12, 2024Export Data From StarRocks into DuckDBIntroduction Let's look at how we can easily export data from StarRocks into a local DuckDB database with Sling, a flexible command-line interface (CLI) data integration tool that enables rapid extraction and loading of data directly from the termina...Discuss·98 readsduckDB
Constantin Lungudatawise.dev·Apr 5, 2024A portable Data Analytics stack using Docker, Mage, dbt-core, DuckDB and SupersetJust wanted to share a small learning-by-doing project of mine. It's a containerized Data Analytics suite, covering end-to-end analytics process for a small (imaginary) company. We're talking about:- generating example data in parquet files using Pyt...Discuss·266 readsLearning JourneyDocker
Tobias Müllertobilg.com·Jan 27, 2024Using DuckDB-WASM for in-browser Data EngineeringIntroduction DuckDB, the in-process DBMS specialized in OLAP workloads, had a very rapid growth during the last year, both in functionality, but also popularity amongst its users, but also with developers that contribute many projects to the Open Sou...Fabio Chen and 6 others are discussing this7 people are discussing thisDiscuss·5 likes·15.4K readsdata-engineering
Manas Singhcode.manas.me·Dec 2, 2023Compare Forex Rates using PythonCurrency Exchange Rates When we transfer foreign currency, we look for the best exchange rates. Now, this rate changes daily and varies across banks. It is hard to predict the currency exchange rate but you can select the bank based on that day's rat...Discuss·34 readsPython
Constantin Lungudatawise.dev·Sep 25, 2023Hands-on with DuckDBThe right tool for the right job, right? When developing a basic application or testing a concept, an option could be SQLite. It's a self-contained database system that offers a SQL interface and conveniently stores data in a file on your device. Per...Discuss·352 readsLearning JourneyduckDB
Tobias Müllertobilg.com·Apr 26, 2023Gathering and analyzing public cloud provider IP address data with DuckDB & ObserverableAs organizations increasingly adopt the public cloud, managing the networking and security aspects of cloud computing becomes more complex. One of the challenges that cloud administrators face is, especially in a hybrid cloud environment, keeping tra...Discuss·1 like·1.2K readsduckDB
Tobias Müllertobilg.com·Apr 24, 2023Casual data engineering, or: A poor man's Data Lake in the cloud - Part IIn the age of big data, organizations of all sizes are collecting vast amounts of information about their operations, customers, and markets. To make sense of this data, many are turning to data lakes - centralized repositories that store and manage ...Discuss·11 likes·4.3K readsA poor man's Data Lake in the cloudAWS
