msalter
New Contributor II
Joined 5 years ago
User Profile
User Widgets
Contributions
Lost contact with Snaplex node
I have a very small pipeline (3 snaps) that I’m reading from a SQL table and writing the results to a new table in the same DB. I keep getting an error: Lost contact with Snaplex node while the pipeline was running. The select statement is pulling 250+M records and I’m using the Azure bulk insert to write. In order to avoid this error I keep having to reduce the batch size, from 20K to 10K to now 8K. Any thoughts on what could be causing the error?Solved12KViews0likes18CommentsSQL Server Insert snap really slow
I’m writing ~ 200M records into a SQL DB but at the current rate (56,000 records over 2 hrs) I’ll be here a long time before it finishes. Wondering if I’m missing something in the setup or if there is something that needs to be tweaked elsewhere.3.7KViews0likes3Comments