This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

ServiceNow Query snap Performance

krishnapriyam25
New Contributor

‎08-20-2021 09:20 AM

My usecase is to ingest data from ServiceNow into Snowflake, with no tranformation. I’m creating simple pipeline with 2 snaps - ServiceNow Query and Snowflake BulkLoad

The service now table has 2.5 M rows, it is taking about 7hrs to read this data from service now .

How to improve the performance??

image

0 Kudos
1 REPLY 1

rdill
Employee
Employee

‎08-20-2021 01:16 PM

instead of doing it in a single pipeline, use multiple pipelines and partition the reads. Depending on the performance of their API you could spin up 10 or 20 pipelines, have them read at the same time and get it done much faster.

0 Kudos