12-07-2021 02:46 PM
Hi,
I have a scenario to pull millions of records with many fields for each using API call in REST GET snap. Unfortunately, pipeline failing due to more volumn of records trying to get. All I need is only few key fields.
Is there a way to restrict number of field(same set of fields) for all the records, that way volumn will be much lesser and no read time out failure can happen during execution in REST GET?
Ex. I have 10 fields in API response, all I need to pull is only 3 fields.
Please help me with your suggestions.
Regards,
Amardeep Shanmugam.
12-09-2021 03:36 PM
It is all depends on your pipeline design. Can you share your pipeline for better suggestions?