DQDerrick Qininderrickqin.com·Sep 24, 2025 · 5 min readIntegrates DuckDB with Google BigQueryBackground In today's data-driven world, the ability to efficiently query and manage large datasets is crucial for businesses. DuckDB, an in-process SQL OLAP database management system, offers a powerful solution for handling complex analytical queri...00
DQDerrick Qininderrickqin.com·Aug 29, 2025 · 3 min readMissing DAGs in Airflow UI: A Cloud Composer Filter BugI recently assisted a customer in troubleshooting an issue with their Google Cloud Composer environment and found myself delving deep to uncover the problem. They had 22 DAGs, but only 11 were visible in the UI. It turns out this is a known bug that ...00
DQDerrick Qininderrickqin.com·Mar 28, 2025 · 2 min readEffective Use of Google Cloud IAM Deny Policies for BigQuery Data SecurityBackground Recently, I assisted a customer who required protection for sensitive data in BigQuery, ensuring access was restricted to a small group of individuals. They wanted to prevent any accidental access by employees. To address this, I recommend...00
DQDerrick Qininderrickqin.com·Feb 1, 2025 · 2 min readHow to Implement IAM Authentication in Cloud SQL PostgreSQL: A Step-by-Step TutorialBackground IAM Authentication in Cloud SQL for PostgreSQL lets you manage database user access using Google Cloud Identity and Access Management (IAM) identities instead of traditional PostgreSQL usernames and passwords. This integrates database acce...00
DQDerrick Qininderrickqin.com·Jan 1, 2025 · 3 min readHow to trigger the DAG in Amazon Managed Workflows for Apache Airflow (MWAA)Background Directed Acyclic Graphs (DAGs) in Amazon Managed Workflows for Apache Airflow (MWAA) can be triggered in several ways, depending on how much automation and integration you need. While you can manually trigger DAGs using the Airflow UI, we ...00