ServiceNow Query snap Performance

[krishnapriyam25]

My usecase is to ingest data from ServiceNow into Snowflake, with no tranformation. I’m creating simple pipeline with 2 snaps - ServiceNow Query and Snowflake BulkLoad

The service now table has 2.5 M rows, it is taking about 7hrs to read this data from service now .

How to improve the performance??

[rdill]

instead of doing it in a single pipeline, use multiple pipelines and partition the reads. Depending on the performance of their API you could spin up 10 or 20 pipelines, have them read at the same time and get it done much faster.