Set a local file path for use in SQL Bulk Load Snap

We’ve attempted to use the SQL Server bulk load snap (37mm rows of data/documents) and consistently fill up the memory allocated to JCC on the snap plex node. It seems that since BCP is a file based operation, a file is written first, then the BCP command is called as expected. What isn’t expected is that that file is apparently written into memory. This seem like a volatile approach, especially if we have many large file pipelines executing at the same time.

Disk space is far more abundant and easy to allocate. Could we get an option on the SQL Server Bulk Load snap to specify an smb file location to hold the temporary file until the BCP operation is fired? Aside from that, it may also be helpful to allow a batch size in that snap to let is create smaller temporary files before committing them via BCP.

Thanks in advance,
Brett

1 Like