CSV with header, detail, and trailer records on S3
I need to create a pipe delimited text file. The file needs to have a single header type record row with two columns, infinite number of detail record rows, and a single trailer type record row with three columns. 00|File Type Description 01|Data1|Data2|Data3|Data4|Data5|Data6|Data7|Data8|Data9|Data10 99|1|EOF I thought I could use multiple CSV parsers + File writer snaps and then just APPEND the detail records and the trailer record, but it looks like the APPEND function isn’t supported on S3. Any ideas on how to accomplish this?
Writing flat file
Hi Team For my current use case, I need to write a flat-file. I am using the following pipeline to write csv. Input to JSON formatter is a document of arrays each having different no of properties. Can you please suggest a way to generate CSV file ignoring all null values and writing only what is present? Note - Each input document to CSV formatter has different set of properties. Thanks in advancePass parameter from child pipeline to parent pipeline
I have a parent pipeline that executes a child pipeline. The child pipeline’s sole purpose to write a single row to a logging table. After writing to the logging table, the child pipeline runs a Select Snap to get the newly inserted log_id . My question is: is it possible to pass the log_id from the child pipeline back to the parent pipeline as a parameter? I know it’s possible to pass it as a document (and join with other documents in my parent pipeline) but then that causes all sorts of other issues. Main question, is it possible to set a parent’s parameter from a child pipeline?
