I am working on building a pipeline to pull data from an OAUTH 2.0 API and load the data into snowflake using snowflake bulk load. I observed that for each output from REST Get snap, the bulk load is triggering a COPY INTO command in snowflake. So for 20 outputs from Rest Get triggers 20 COPY INTO command in snowflake. This is not efficient. How can i make sure that the bulk load moves data using a single COPY INTO command? Thanks!
Can you pls share your pipeline design to have more clarity ?