cancel
Showing results for 
Search instead for 
Did you mean: 

Pipeline To Output one File per Date

GV2019
New Contributor II

I need to get Distinct update_dt and write one file per date from a oracle table sufixing the filename with date value. can you provide a sample pipeline which has similar functionality
I have created 2 pipelines one with

  1. Oracle execute – ( Select trunc(update_dt) from inv_lines_stage group by trunc(update_dt)
  2. mapper
  3. For Each
    Pipleline 2 with
  4. Oracle Select – (Select * from inv_lines_stage where trunc(update_dt) == $_updateDt)
    _upsateDT defined as parameter in pipeline settings
  5. mapper
  6. CSVFormater
  7. FileWriter
    My problem : I am not able to join these pipelines and pass the update_dt as parameter to 2nd pipeline
3 REPLIES 3

tlikarish
Employee
Employee

Hey welcome to the community, GV2019.

I’d recommend that you consider using a PipeExec instead of the deprecated ForEach snap. The basic idea is to have a pipe parameter that is passed into your second pipeline, which can be used for setting the file name. The document will also be passed in, which can be used to do the query. This makes it a little easier so you don’t have to plumb the update_dt through to the FileWriter.

Here’s a simple example that shows how to pass a parameter and use it in the a child pipeline. The pass-param-pipeline will pass in a document and set a pipeline parameter on use-param-example, which takes the document and writes it out using the update_dt field. Hope this is helpful.

pass-param-pipeline_2019_10_28.slp (4.2 KB)

use-param-example_2019_10_28.slp (4.8 KB)

GV2019
New Contributor II

Hi tlikarish the samples helped in understanding how to create and pass parameters
Thank you
GV

lokey
New Contributor II

@GV2019 You may make use of parameters and headers to configure pipeline execute snap. If there are more don’t configure anything and pull them as $ in child.