Forum Discussion

Senthilnaga91's avatar
Senthilnaga91
New Contributor III
3 years ago
Solved

Scheduling Tasks Dynamically

Hello All, I have a use case where I have to schedule my pipeline dynamically based on entries from an input table? What’s the best way to implement this? For Example: If I have a pipeline loadSale...
  • Abhishek_Soni37's avatar
    Abhishek_Soni37
    3 years ago

    @Senthilnaga91 ,

    As @bojanvelevski mentioned you can use the Metadata snap pack to achieve it,

    Solution Approach:

    1. Create all tasks that you want to run dynamically as Scheduled.
    2. Read data from the source table with info on when to run a pipeline.
    3. Read the specific Task with the help of the Metadata snap pack.
    4. Assign new values and update the Task with the Metadata snap pack.

    Set values below in the pipeline parameter:

    source_proj: orgName/ProjectPath
    target_proj: orgName/ProjectPath
    taskToUpdate: Name of the task

    Sample pipeline:
    UpdateTaskInfo.slp (3.7 KB)

    Let us know if this worked.

    Cheers 🙂

Senthilnaga91's avatar
Senthilnaga91
New Contributor III
3 years ago
Copy LinkReport

I have provided more details in my previous reply. Please refer.

Abhishek_Soni37's avatar
Abhishek_Soni37
Contributor
3 years ago
Copy LinkReport

@Senthilnaga91 ,

As @bojanvelevski mentioned you can use the Metadata snap pack to achieve it,

Solution Approach:

  1. Create all tasks that you want to run dynamically as Scheduled.
  2. Read data from the source table with info on when to run a pipeline.
  3. Read the specific Task with the help of the Metadata snap pack.
  4. Assign new values and update the Task with the Metadata snap pack.

Set values below in the pipeline parameter:

source_proj: orgName/ProjectPath
target_proj: orgName/ProjectPath
taskToUpdate: Name of the task

Sample pipeline:
UpdateTaskInfo.slp (3.7 KB)

Let us know if this worked.

Cheers 🙂