File Writer - Append to Excel

[sdoscher]

Hoping someone can help. I have a pipeline that executes a SQL, uses a file writer to put the result set in an excel (call it excel #1) and then emails Excel #1 as an attachment. All works good.

WHAT I NEED TO DO: have another excel file (call it excel #2) that keeps a history of what excel #1 has/had. The key is that I would like the final excel #1 to NOT include anything if it already pulled in an earlier file. My thought was to strip out of Excel #1 anything that is in Excel #2.

There very well could be a better way to do it, but I CAN’T do it in SQL because I can’t build any table in that environment.

Any help would be greatly appreciated.

Scott

[Supratim]

sdoscher:

very well could be a better way to do it, but I CAN’T do it in SQL because I can’t build any table in that environment.

@sdoscher I am not 100% sure if I understand your requirement correctly, but as per my understanding you can read execl#2 parallelly then parse it .After that do a join (inner/outer or left outer) just before your mapper by your key element from both execl1 and excel 2. Then use mapper for creating new set of data whatever is required.

[sdoscher]

Thank you so much @Supratim. Let me try that approach. I appreciate your feedback.

[mina]

Hi Scott and @Supratim I’d think our Diff snap would also work. Here is the sample pipeline. What do you think?

[sdoscher]

Oh wow…let me take a look at that…thank you!