MS SQL Bulk Insert
is there a way to perform a bulk insert operation to insert data into MS SQL? currently we have a pipeline that has about 3.5M records / documents. It takes a long time to insert all of these into SQ...
Forum Discussion
Thank you both for your quick responses. š
Iām dealing with a small set of data 498 records.
Iāve been using āShift click Validateā as a fall back when snaps are not behaving as I thought they should or I change the underlying file or database structure.
in order to simplify my issue (pretty sure this is new user growing pains) Iāve filtered both New and old data sources by id=271967386.
First run performs as expected:
Second run not so much (expected 1 record in the āUnmodifiedā output)
I suspect it is due to data type? could anyone possibly confirm?
Oracle (new) data preview before Diff Snap:
SQL (old/existing) data preview before Diff Snap
Iām going to try using the mapper snap or another transform to solve.
Thank you all again for your time.
Update:
I cut down the data to only the ID field and recreated the destination SQL table, so 1 record 1 field and the issue was still present.
Adding a mapping snap which changed the ID(Oracle Source) from number to string seems to resolve it, but when I try to add other fields (still 1 record) the record is routed to the āModifiedā output.
Iāve used notepad++ compare plugin on the two Diff snapās inputs and they are a match, this has really confused me
Mapper Snap Details:
Pipeline with mapper:
any advice appreciated
Try parsing the date into the specific format:
Date.parse('2022-03-17 05:10:10','yyyy-MM-dd HH:mm:ss').toLocaleDateTimeString({'format':'yyyy/MM/dd HH:mm:ss'})Thank you, bojanvelenski.
Is there any snap for DB2 bulkupload which is similar to MySQL
when migrating the data from DB2 MySQL , i have used Bulkload snap and it mapped the columns automatically.
when I am doing the revese sync mapping , I dont see the Bulk load snap for DB2 ā¦and compain about th other columns which are are part of the table definition present in the Mapper.
is there any snap which is equivalent to MySQL bulk upload for DB2.
Frankly, Iāve never worked with DB2. Which snaps did you use for reading from DB2 ?
Generic JDBC snaps
