We are trying to run a triggered task through a CURL call and the pipeline runs for more than 15 minutes. The CURL call dies after 15 minutes.
We found that the Cloud URL has a 15 minute connection limit whether a connection is idle or not. Is there a way around this or to maximise the connection limit?
We have a few options including batching up data in order to run the pipeline in less that 15 minutes but was wondering if we could maximise the connection limit somehow?
Any help is really appreciated. Thanks!