File Writer - Append to Excel

Hoping someone can help. I have a pipeline that executes a SQL, uses a file writer to put the result set in an excel (call it excel #1) and then emails Excel #1 as an attachment. All works good.

WHAT I NEED TO DO: have another excel file (call it excel #2) that keeps a history of what excel #1 has/had. The key is that I would like the final excel #1 to NOT include anything if it already pulled in an earlier file. My thought was to strip out of Excel #1 anything that is in Excel #2.

There very well could be a better way to do it, but I CAN’T do it in SQL because I can’t build any table in that environment.

Any help would be greatly appreciated.


@sdoscher I am not 100% sure if I understand your requirement correctly, but as per my understanding you can read execl#2 parallelly then parse it .After that do a join (inner/outer or left outer) just before your mapper by your key element from both execl1 and excel 2. Then use mapper for creating new set of data whatever is required.