File Writer - Append to Excel

Hoping someone can help. I have a pipeline that executes a SQL, uses a file writer to put the result set in an excel (call it excel #1) and then emails Excel #1 as an attachment. All works good.

WHAT I NEED TO DO: have another excel file (call it excel #2) that keeps a history of what excel #1 has/had. The key is that I would like the final excel #1 to NOT include anything if it already pulled in an earlier file. My thought was to strip out of Excel #1 anything that is in Excel #2.

There very well could be a better way to do it, but I CAN’T do it in SQL because I can’t build any table in that environment.

Any help would be greatly appreciated.


@sdoscher I am not 100% sure if I understand your requirement correctly, but as per my understanding you can read execl#2 parallelly then parse it .After that do a join (inner/outer or left outer) just before your mapper by your key element from both execl1 and excel 2. Then use mapper for creating new set of data whatever is required.

Thank you so much @Supratim. Let me try that approach. I appreciate your feedback.

Hi Scott and @Supratim I’d think our Diff snap would also work. Here is the sample pipeline. What do you think?

1 Like

Oh wow…let me take a look at that…thank you!