Inserting large data in servicenow

Question

Hello Team,&nbsp;I am developing a pipeline in SnapLogic where there are 6000000 records coming from snowflake and i have designed my pipeline like this:&nbsp;Parent pipeline: snowflake execute -&gt; mapper where i have mapped one to one field -&gt; group by n with 10000 group size -&gt; pipeline execute where&nbsp;Pool size is 5&nbsp;and in child pipeline i have used json spliter and service now insert ?&nbsp;what can i do to optimize the performance and make it execute faster in snaplogic, currently it takes much time to execute ?&nbsp;Can someone assist in this regards?&nbsp;Thanks in advance.&nbsp;

koryknick · Answer

deepanshu_1&nbsp;- You can relieve some of the memory requirement off your snaplex nodes by removing the Group By N and JSON Splitter snaps and use the Batch Size option in the Pipeline Execute snap instead.&nbsp; This accomplishes exactly the same result without the memory consumption of combining large sets of records into a single document as an array.
If you are still experiencing slowness, you can follow-up with your ServiceNow admins to see if anything can be done on that side.&nbsp; I believe that ServiceNow is not really meant for batch operations, so inserting millions of records into ServiceNow is probably your bottleneck.

ssapa · Answer

deepanshu_1 The ServiceNow Insert Snap supports batching. Did you use the Page Size option to set the batch size? Please try increasing the Page Size value and see if that helps.

deepanshu_1 · Answer

Thanks&nbsp;koryknick&nbsp;and&nbsp;ssapa&nbsp;for your support.

Forum Discussion

Inserting large data in servicenow

Recent Discussions

Array of Objects manipulation

How to convert a MS Word file to PDF

Need Guidance on Dynamic Excel File Generation and Email Integration

Platform Memory Alerts & Priority Notifications for Resource Failures

SnapLogic Execution Mode Confusion: LOCAL_SNAPLEX vs SNAPLEX_WITH_PATH with pipe.plexPath