cancel
Showing results for 
Search instead for 
Did you mean: 

ServiceNow Query snap Performance

krishnapriyam25
New Contributor

My usecase is to ingest data from ServiceNow into Snowflake, with no tranformation. I’m creating simple pipeline with 2 snaps - ServiceNow Query and Snowflake BulkLoad

The service now table has 2.5 M rows, it is taking about 7hrs to read this data from service now .

How to improve the performance??

image

1 REPLY 1

rdill
Employee
Employee

instead of doing it in a single pipeline, use multiple pipelines and partition the reads. Depending on the performance of their API you could spin up 10 or 20 pipelines, have them read at the same time and get it done much faster.