Read historical data in the first run
I have a pipeline that extracts data from Oracle table and write data to destination. Oracle select snap will read current day’s date based on a time field. When I am migrating this pipeline to PROD ...
Forum Discussion
thank you @Supratim, as you have suggested that’s doable in DEV. I am concerned how do I achieve this in PROD environment where I won’t be able to modify the file or snap settings . I am looking for some configuration/design that doesn’t need manual intervention.
You don’t need to change the manually for every single execution.
When you migrate to prod first time also move this file with some old dates , rest of the time pipeline will update the latest time stamp on this file.
