Order of reponses when using pagination with http client

Thanks koryknick 

Good to have the confirmation on thesynchronous nature of HTTP Client and REST POST snaps.

However I would be surprised if sort collation was the problem. One example is where 3210448-USM18 came before 3210448-NYR6 in a supposedly ascending sort. That would suggest a sort collation that puts U before N which seems unlikely.

As I mentioned above, the model I am using is the one given here https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/3097822264/Pagination. (Because as per that scenario I need to send sort criteria in the body of the POST for each call.)

That model does not really use the pagination functionality of the HTTP Client snap. Instead it creates a list of parameters to send, sends that list to an HTTP Client snap, and then combines the responses using a Union snap.

I note here https://community.snaplogic.com/t5/designing-and-running-pipelines/union-preserve-source-data-sequence/m-p/20480 that the Union snap doesn't guarantee preservation of order. Is it possible that in my case the responses are coming back from the API in an order different to the sending order (perfectly possible because of varying latency), and then the response are being presented in that order?

If that is the case, then I think the proposed model has a serious flaw. The idea as stated is to be able to specify e.g. sort criteria in the POST body, but if the use of the union means the results aren't necessarily sorted then that's no good.

In my case the whole point of specifying the sort in the API criteria is so that I can avoid using a Sort in the pipeline. I need to do a join later on a large volume and I run into memory issues if I have to sort.

Any thoughts ? Thanks for your input - much appreciated !

The error I had was this :

importId is the value I have fetched from the API (in ascending order) and I am joining with other data using inner join and ascending sort order. I interpret this to mean that the error was thrown because the importId of 3210448-USM18 came before 3210448-NYR6 in the join.

I have just twice fetched only the importIds from the API (without doing the join) and both times 3210448-NYR6 came before 3210448-USM18.

In addition, every single value of importId takes the form of 7 numbers followed by a hyphen followed by some other combination of letters and characters. My understanding of sort collation may not be sufficiently advanced so apologies if I'm on the wrong track - given that format I'm not sure how sort collation could have an impact. Even if one collation ignores the hyphen it's the same for every single value?

Thanks again for your thoughts !