Forum Discussion
@fmdf - I’m sorry you find the documentation confusing. You mentioned you were using the Bulk API which is why I mentioned the “Match datatype” option, which should preserve the numeric, boolean, etc. datatypes of the scalar elements (i.e. matching what is the returned when using the REST API mode).
When using the “Create table if not exists” option, keep in mind that it creates the table based on the first input document. When pulling from a source like Salesforce, there are many optional values that may not be present in all incoming data. I would recommend that you determine which elements from the source object you want to load to the target and manually create the target table prior to running your pipeline to load it.
I am extracting data for the purposes of loading a data warehouse using the bulk api as stated in my original post.
The match data type documentation is not clear. It only applies in certain circumstances. I have checked and unchecked and run the pipeline through a dozen times and it is not behaving consistently.
If I have to manually create the tables to get the data types to be correct, it eliminates the usefulness of the create table if not present checkbox.
I do not have a way to generate a create table script from Salesforce.
I can’t be the only person that has experienced this issue.