Tag: data-engineering
All the articles with the tag "data-engineering".
-
Data This Week #18
Supabase Multigres for Postgres scaling, Netflix's dynamic Cassandra partition splitting, dbt Core v2 on Rust, SQL/PGQ graph queries in Postgres 19, and IAM fundamentals for data engineers.
-
Data This Week #17
Pulsar 5.0 Scalable Topics, Iceberg multi-engine query routing, Netflix's high-throughput graph abstraction, Iceberg partition evolution, AWS Glue automation, and community thoughts on Databricks' BI migration tool.
-
Data This Week #16
Databricks as a unified platform, PostgreSQL pluggable storage engines, Iceberg 1.11.0 highlights, database egress costs, JDBC query caching, and dbt column-level lineage.
-
Data This Week #15
Spark Declarative Pipelines for financial lakehouses, ten AWS Glue & Iceberg fixes, MOR as an architectural shift, DuckDB's Quack protocol, SQL fraud patterns, Kafka checkpoint patterns, and the LLM-for-validation debate.
-
Data This Week #14
Flink CDC streaming ELT from MySQL to Kafka, the LLM engineer's stack map, Ursa's diskless Kafka fork, Iceberg write mechanics, Instacart's billion-product search, Jikkou 1.0, and the AI knowledge-base debate.
-
Data This Week #13
Spark memory tuning, row-level validation tiers, Postgres RLS pitfalls, Stripe's sharding at 5M QPS, Aurora DSQL vs Postgres, Velero joins CNCF, and SQLGlot 5x faster with mypyc.
-
Data This Week #11
Iceberg cross-account migrations, DuckLake 1.0 metadata, IaC for data engineers, Redshift Iceberg writes, agent-data patterns, LARQL for LLM graph queries, and Dagster pricing debate.
-
Data This Week #10
Data product lifecycle, semantic context layer for LLM agents, Netflix's Druid interval caching, Ursa Kafka storage engine, Iceberg v3 VARIANT type, and Ministack vs LocalStack.