Forum Discussion

GV2019's avatar
GV2019
New Contributor II
7 years ago

Pipeline To Output one File per Date

I need to get Distinct update_dt and write one file per date from a oracle table sufixing the filename with date value. can you provide a sample pipeline which has similar functionality
I have created 2 pipelines one with

  1. Oracle execute – ( Select trunc(update_dt) from inv_lines_stage group by trunc(update_dt)
  2. mapper
  3. For Each
    Pipleline 2 with
  4. Oracle Select – (Select * from inv_lines_stage where trunc(update_dt) == $_updateDt)
    _upsateDT defined as parameter in pipeline settings
  5. mapper
  6. CSVFormater
  7. FileWriter
    My problem : I am not able to join these pipelines and pass the update_dt as parameter to 2nd pipeline

3 Replies

  • GV2019's avatar
    GV2019
    New Contributor II
    7 years ago

    Hi tlikarish the samples helped in understanding how to create and pass parameters
    Thank you
    GV

    • lokey's avatar
      lokey
      New Contributor II
      6 years ago

      @GV2019 You may make use of parameters and headers to configure pipeline execute snap. If there are more don’t configure anything and pull them as $ in child.

  • tlikarish's avatar
    tlikarish
    Employee
    7 years ago

    Hey welcome to the community, GV2019.

    I’d recommend that you consider using a PipeExec instead of the deprecated ForEach snap. The basic idea is to have a pipe parameter that is passed into your second pipeline, which can be used for setting the file name. The document will also be passed in, which can be used to do the query. This makes it a little easier so you don’t have to plumb the update_dt through to the FileWriter.

    Here’s a simple example that shows how to pass a parameter and use it in the a child pipeline. The pass-param-pipeline will pass in a document and set a pipeline parameter on use-param-example, which takes the document and writes it out using the update_dt field. Hope this is helpful.

    pass-param-pipeline_2019_10_28.slp (4.2 KB)

    use-param-example_2019_10_28.slp (4.8 KB)