Forum Discussion
rdill
5 years agoAdmin
instead of doing it in a single pipeline, use multiple pipelines and partition the reads. Depending on the performance of their API you could spin up 10 or 20 pipelines, have them read at the same time and get it done much faster.