This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Best way to process millions of records in REST GET API call without Read time out error

amardeep2021
New Contributor III

‎12-08-2021 09:19 AM

Hi,

I am trying to process millions of records by using REST GET snap API call. I am getting Read time out error. Is there a best way to bring in millions of records in REST call without facing Read time out failure during execution? Currently I am using Mapper–>REST GET–>Mapper for further processing.

Please let me know with your tips.

Regards,
Amar

Reply
10 REPLIES 10

del
Contributor III
In response to amardeep2021

‎12-13-2021 12:04 PM

@amardeep2021, it looks like you’re calling a Workday RaaS (web-enabled) report. These Workday reports do not natively support pagination. The Workday Community site has multiple Brain Storm requests for a pagination feature for the reports, but Workday has not addressed these requests.

With that said, you may can work with your Workday Report team/developers to creatively develop a report with pagination/chunking capabilities.

For example, if the report data source records have timestamps or effective dating, the report team could provide prompt fields whereby you can include start and end times in the URL query string which would allow you to chunk the data by time ranges. Depending on the data you’re pulling in, there may be other creative ways to chunk the data to provide pagination capabilities.

Likewise, if this post is related to your other post about reducing the columns pulled in from the API using REST GET, the report team can reduce the number of columns in the report output.

0 Kudos
Reply