Forum Discussion

rpc1235x's avatar
rpc1235x
New Contributor II
2 years ago
Solved

Dynamic range of dates for REST

My current pipeline works with pipeline parameters defining the range of dates However my goal is to automatize this for each month. My idea is to schedule a task for the first of every month ...
  • Aleksandar_A's avatar
    Aleksandar_A
    2 years ago

    Hello again,

    As I mentioned above, this is not how a typical pagination would look like using the REST GET / HTTP Client Snap, since I'm not aware about any information in the response that can be used to set the Has Next and Next URL properties. 

    And since that you have the total number of pages in the info request, this approach will generate as many documents (requests) as you have pages.

    If your parameters are static you can map them in the Map - Pages Mapper Snap and pass them in the JSON Splitter in the Include Paths section so you can use them in the REST GET Snap.

    Please refer to the attached pipeline and let me know if this helps you.

    Regards,

    Aleksandar.

fabriciocarboni's avatar
fabriciocarboni
New Contributor II
2 years ago

Hi,

I had the exactly same problem and could not find a way to do that. The documentation is poor. What I had to do was develop a python script to achive this to load lots of historical data with pagination because the api allowed me only to retrieve 1000 records at a time.

snapation6713's avatar
snapation6713
New Contributor III
2 years ago

Thank you for taking the time to reply fabriciocarboni.   Did you create the python script in snaplogic?   If so did you have to install any python packages to get it to execute?

  • rpc1235x's avatar
    rpc1235x
    New Contributor II
    2 years ago

    Thank you for the response. 

    I have imported the expression library files to the Project Manager section. However, I have this error in the mapper connected to the REST Get. Is there an additional step to enable the expression library?