How to copy logs from /opt/snaplogic/run/log through a pipeline without losing data or data duplication?

Hi Smitkumar,

Good day, base from the scenario that you post it looks like you need a file watcher in the pipelines view there’s no such thing unless you schedule the pipeline to run every 15 secs

A quick snaps that comes to my head is use a directory browser snap, copy snap, sort snap and a file operation snap and a delete snap for the 10 file mark depending where you are comfortable you can add a sequence snap

first line of pipeline is the scenario (1)

Directory browser to browse the log source path use a filter then filter only the files with size >= 100MB then feed the path to file operation use move action so that it will also be deleted from the source dir

second line of pipeline is the scenario (2)
Directory browser to browse the target path put a sort snap sort it by update date then the sequence snap to label the files then filter snap filter the files that are labelled > 10 then delete snap

🙂 hope this helps