I create a pipeline to create a target table based on source table, the steps in the pipeline are: 1.DROP the target table if exists → 2.Create the table if not exists -->3. Modify some column length in the target table → 4.Read data from source table → 5. Mapper and then Bulk load to target table.
I do NOT think any of these steps would get table lock, however, when execute this pipeline from a parent pipeline, it will get deadlocked issue sometimes, and it will always success after 2nd or 3rd time re-run.
Anyone has idea about this?