Forum Discussion

Parex122334's avatar
Parex122334
New Contributor III
3 years ago
Solved

Unable to insert 50000 records

Hi all, I want to insert whole 50000 records from CSV File  to Sql Server,but its inserting only 2000 records only in a table. How to resolve this issue or how to insert whole data from CSV to  Sql ...
  • ptaylor's avatar
    ptaylor
    3 years ago

    Try executing the pipeline instead of just validating it. Then use the Pipeline Execution Statistics to see how many records were processed.

  • darshthakkar's avatar
    darshthakkar
    3 years ago

    I would recommend going ahead with Parent-Child Pipeline and define a pool size so that there are no too many calls on the server.
    50k hitting at the same time is a lot and may break the downstream. I've had more than ~3m rows going from snapLogic and I would say for sure that you wouldn't get any errors and agreed with ptaylor - please execute the pipeline instead.

    You may want to change your settings from 2000 to 50, sharing some screenshots for your references...

     

    Thank you.

     

    Regards,

    Darsh

darshthakkar's avatar
darshthakkar
Valued Contributor
3 years ago

I would recommend going ahead with Parent-Child Pipeline and define a pool size so that there are no too many calls on the server.
50k hitting at the same time is a lot and may break the downstream. I've had more than ~3m rows going from snapLogic and I would say for sure that you wouldn't get any errors and agreed with ptaylor - please execute the pipeline instead.

You may want to change your settings from 2000 to 50, sharing some screenshots for your references...

 

Thank you.

 

Regards,

Darsh