cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

File Writer - Append to Excel

sdoscher
New Contributor III

Hoping someone can help. I have a pipeline that executes a SQL, uses a file writer to put the result set in an excel (call it excel #1) and then emails Excel #1 as an attachment. All works good.

snap

WHAT I NEED TO DO: have another excel file (call it excel #2) that keeps a history of what excel #1 has/had. The key is that I would like the final excel #1 to NOT include anything if it already pulled in an earlier file. My thought was to strip out of Excel #1 anything that is in Excel #2.

There very well could be a better way to do it, but I CANโ€™T do it in SQL because I canโ€™t build any table in that environment.

Any help would be greatly appreciated.

Scott

4 REPLIES 4

Supratim
Contributor III

@sdoscher I am not 100% sure if I understand your requirement correctly, but as per my understanding you can read execl#2 parallelly then parse it .After that do a join (inner/outer or left outer) just before your mapper by your key element from both execl1 and excel 2. Then use mapper for creating new set of data whatever is required.

sdoscher
New Contributor III

Thank you so much @Supratim. Let me try that approach. I appreciate your feedback.

Hi Scott and @Supratim Iโ€™d think our Diff snap would also work. Here is the sample pipeline. What do you think?

sdoscher
New Contributor III

Oh wowโ€ฆlet me take a look at thatโ€ฆthank you!