Showing results for 
Search instead for 
Did you mean: 

Can't paginate using Post HTTP CLIENT Snap

New Contributor III

I am trying to paginate using a POST HTTP CLIENT Snap.  I don't know how to accomplish this with the information provided in the data returned.  I am extracting data from HubSpot using the 'search' endpoint (  The data returned looks like this:


How do I use this information to configure the snap to paginate? 

Please Help!


New Contributor III

@koryknick,   Thank you for your help.  I was provided a solution by a coworker.

It's essentially that pipeline solution you provided HTTP Client Pagination  with a few tweaks.  I modified the HTTP Client as follows: Removed pagination configurations and the parameters configurations for both http snaps and added to the body of the 2nd HTTP snap   "after" : $splitValue, "limit" : 100    Then I was getting subsequent data.   But the data was duplicating.  So I modified the map for pagination as follows:  

sl.range(1, parseInt($, 100)  -I also changed the 1st http snap to just give me 1 record just so that I get the total  and I process from page 1 to the end in the rest of the pipeline so I don't do a union.  
And it worked!   I am now getting subsequent data and it the records are not duplicating.  



View solution in original post


New Contributor II


I had the exactly same problem and could not find a way to do that. The documentation is poor. What I had to do was develop a python script to achive this to load lots of historical data with pagination because the api allowed me only to retrieve 1000 records at a time.

Thank you for taking the time to reply @fabriciocarboni.   Did you create the python script in snaplogic?   If so did you have to install any python packages to get it to execute?

Hi @snapation6713,

No, our Snaplogic subscription does not have RPE. So I had to develop the script and run it from my local machine.



Ah!  Ok thank you so much for your time!

Enjoy your day.