Submitted by @stodoroska from Interworks
Based on the source data, we are creating action flags that later will be used in the Action stage pipeline.
Screenshot of pipeline
Configuration
Because we are working with a large scale of data, the target records are grouped in the chunks of 2000 records. This makes the pipeline faster and less memory is used.
Sources: CSV file
Targets: CSV file and PGP-encrypted file
Snaps used: Mapper, Join, Group by N, Splitter, CSV Parser, File Writer, File Reader, Union, Copy, PGP Encrypt
Downloads
Attach pipelines and any necessary resources such as expression libraries or source files.
Diane Miller
Community Manager
