Used ranges For example:
Date Records processed Execution time
17-Nov-2026 51,113 3 min 21 sec
17-Nov-2026 31,868 2 min 12 sec
17-Nov-2026 18,946 1 min 26 sec
17-Nov-2026 32,167 2 min 20 sec
18-Nov-2026 9,683 57 min 50 sec
18-Nov-2026 9,683 52 min 59 secHi Monsih - I checked SnapLogic documentation via SnapGPT and here are some things to consider: A 51K-record run in about 3 minutes versus a 9.7K-record run in 53-58 minutes, with the same pipeline and configuration, usually points to something external to the pipeline logic. The usual causes are waiting on a downstream system, resource contention on the node, or a per-document operation that got slower. The checks below should show which one it is.
Start with Snap statistics for both runsOpen both executions in Monitor → Pipeline executions and compare the Snap statistics tab side by side.o Look at document rates and input time together. An upstream Snap with a low output rate, feeding a downstream Snap with a high rate and high input time, means the upstream Snap is the bottleneck. The downstream Snap is just waiting for documents.o Find the Snap whose time changed most between the two days. If your database write or lookup Snap (Insert, Update, Merge, Select, REST, etc.) went from fast to very slow, the bottleneck is downstream, not in SnapLogic.o Check for a very low docs/sec rate on one Snap. A rate that low on a small dataset usually means row-by-row calls to an external system, such as a lookup per record or a small batch size.o Check the Pipeline logs tab (use the All filter) for retries, timeouts, connection re-establishment, or long gaps between log entries. You can download the logs for both runs and compare them.
Check Groundplex node metrics for the time windowsGo to Monitor → Infrastructure, select the node, and open the metrics graphs for 17 Nov and 18 Nov. These are the graphs to compare, with the thresholds SnapLogic recommends:
Sustained usage above ~80% suggests contention from other processes. SnapLogic service use of node CPU (%) Sustained usage above ~70% suggests pipeline workload is heavy. Last minute system load Values above ~0.7 are a concern. Consistently above 1.0 means the CPU is saturated and tasks are waiting. Service use of heap memory Post-GC heap usage above ~75% can cause long garbage-collection pauses and slowdowns. Combined committed memory Usage approaching the node’s physical RAM can cause swapping or instability Active pipelines (count) Compare against 17 Nov to see whether other pipelines ran concurrently on 18 Nov.Active threads (count) Keep this below ~80% of the thread pool maximum. Network data sent/received Low throughput on the slow day can point to a network or endpoint bottleneck. File descriptor use (%) Above ~70% can cause connection problems. Total disk use (%) Above ~80% can slow temp-file and spill operations. You can also open the Node diagnostics table (Infrastructure → click node → Additional details) to see current, recommended, and average and peak RAM values.
Check concurrency and slot availabilityo Concurrent executions: Filter the Pipeline executions table to the 18 Nov window and the same Snaplex. Look for other heavy pipelines or scheduled tasks overlapping with yours.o Slots: Each node has a slot count (default 4000, roughly one per active thread). If the node is near its limit, executions can queue. The pipeline’s Extra Details tab shows its estimated slot requirement.o Queued time: Check whether the run’s start was delayed. Queuing only applies to scheduled-task executions, so compare the scheduled time against the actual start time.o Load distribution: In a multi-node Snaplex, the scheduler picks the least-loaded node, prioritising memory. Check whether the slow run landed on a different node than the fast one.
Factors that can make a smaller dataset slower
Downstream database slowness. Locks, blocking sessions, index or statistics changes, a competing batch job, or a degraded DB server can slow every insert or update. Your pipeline would be fast on 17 Nov and crawl on 18 Nov with no change on the SnapLogic side.
Row-by-row operations. Lookups, Select Snaps, REST calls per document, or single-row inserts depend entirely on endpoint latency. A 10x latency increase turns 3 minutes into 30 or more.
Different data shape. Records with larger payloads, nested JSON, big blobs, or more child rows can cost far more per document than raw record counts suggest.
Join, Sort, or Group By Snaps on unsorted data. These are memory-heavy. If heap is under pressure, they slow down significantly. Sorted joins stream better.
Heap pressure and GC pauses. Other pipelines on the node can push heap high enough that yours is repeatedly paused.
Network latency or throttling. Firewalls, VPN or proxy issues, or API rate limiting and 429 retries from the target system.
Connection or account issues. Slow authentication, token refresh retries, or connection-pool exhaustion.
Child pipelines (Pipeline Execute). Pool size or reuse settings can change how much parallelism you actually get.
Isolating where the bottleneck isWork through these in order:
Identify the slowest Snap on 18 Nov using Snap statistics, and compare it to the same Snap on 17 Nov.
Classify that Snap:
It is a database or API Snap (Insert, Update, Select, REST, etc.). Suspect the downstream system.
It is a Mapper, Join, Sort, or Aggregate. Suspect pipeline logic or memory.
It is a Snap that is fast but shows high input time. The bottleneck is upstream.
Correlate with node metrics. If CPU, load, and heap were normal during the slow window, the Groundplex is likely not the cause. If they were pegged, check what else was running.
Ask your DBA to check the database for the same window: long-running queries, locks or blocking, wait events, CPU or I/O saturation, and any maintenance, backup, or ETL jobs.
Run a controlled test. Re-run the pipeline with a small sample (for example 1,000 records) during a quiet period and again during the problem window. If it’s fast when quiet and slow at the problem time, the cause is environmental.
Isolate the endpoint. Temporarily replace the downstream Snap with a Trash or Copy-to-file Snap. If the pipeline then runs quickly, the downstream system is confirmed as the bottleneck.
Check batch settings. Review the batch or page size on the database Snaps. Small batches amplify per-call latency.
Longer-term recommendation Configure Snaplex alerts and notifications for CPU, memory, disk, and network thresholds so you catch contention as it happens.
Run Snaplex diagnostics and review the reports periodically.
Use pipeline parameters and logging (for example start and end timestamps around the DB Snaps) to make future comparisons easier.
If the node is consistently near its limits, consider scaling out to more nodes or reducing concurrent scheduled pipelines.