I think there no one size fit for all and for very large scale data processing it best to use distributed data processing tool Apache Spark and Apache Flink both support SQL which is very well structured and organized approach to build the ETL , support time slices joins and TTL bases materialized view and you can choose b/w sql/java/python to write your code . So one tools works with all.