cancel
Showing results for 
Search instead for 
Did you mean: 

Transpose column into row csv file

fernandezso
New Contributor II

I have a pipeline that reads two csv files from different servers, i want to put together the data of both csv files into one. I used a union snap to put the data together but if one server is down it’s only going to bring me the data of only one server. I put a flag in each section of the file reader, if that flag its true after the join it will check that both flags are true, if yes then write one csv file if not don’t write. But after I configure my join snap i notice that the final csv has more columns, new column names with the prefix “input1_ColumnName”. How can I make that data move from the right to the bottom of the csv file?
image
image

1 ACCEPTED SOLUTION

koryknick
Employee
Employee

Here is a sample pipeline using Gate and Router to verify the existence of data from both input streams. Note that I’m not doing the email off the read fail, but it will still work the same way.

Example_Missing_Input_2020_07_09.slp (17.4 KB)

Input file used: directory.txt (2.1 KB)

View solution in original post

4 REPLIES 4

koryknick
Employee
Employee

Try adding an Exit snap after the Emailer with threshold at 0 to fail the pipeline if either source fails. You may also have to set the batch size property in the Emailer to 1 to prevent record batching.

This should allow you to continue to use the Union snap and avoid the “added columns” issue.

I just put the exit snap but since one server stills up it is sending the information to the file writers, and creating the csv file with half of the information:
image

image

koryknick
Employee
Employee

Here is a sample pipeline using Gate and Router to verify the existence of data from both input streams. Note that I’m not doing the email off the read fail, but it will still work the same way.

Example_Missing_Input_2020_07_09.slp (17.4 KB)

Input file used: directory.txt (2.1 KB)

It worked!! Thanks!
image

This is the final pipeline 😀