Incremental Load in Data bricks part -1
Use cases
Suitable if the pipeline runs infrequently.
Assume the scenario if files are loaded in same directory everyday. If not used processed and yet to process folder.
Steps to implement
List the current files in directory
create database if...
notes123.hashnode.dev1 min read