New element in mapper
I have a pipeline in which I am reading a file and parse it with CSV parser.
Name|Age|Gender
David| 18| M
Julie|20|F
I am able to parse the file but at the end of each record, I would like to...
Forum Discussion
Hi,
Please someone advice on this one.
Thanks
Hi @pranil2k2
You can try by opening the error view for Oracle Merge Snap(meaning that all the error rows will be routed in that view). In that way you will isolate the pipeline from failing.
As an output of the Oracle Merge Snap you will get all successful records.
To count the records from output you can use :
- snap.out.totalCount - you will get total number of documents that have passed through.
- Use Aggregate Snap(function Count) on some unique column(field).
Regards,
Spiro
Thanks I was able to get done with child pipeline here.
I am wondering why parser looses the header such as $[“content-location”]
If we can have a flag on parser to preserve those headers, we would be able to accomplish all in one pipeline ?
You can also add Binary to document to the FileWriter, after that a copy snap, then document to binary and csv parser. After the csv parser you will add a join snap, where you will join the output of the copy snap and the output of the csv parser where you will have the $[‘content-location’] and the output of the parser.
But join is an expensive operation, so I am not sure with how many records are you dealing with in this pipeline. This is just a suggestion of a possible way how to implement it in a single pipeline.
