cancel
Showing results for 
Search instead for 
Did you mean: 

RPE only returns small proportion of documents

JmPearl
New Contributor II

I have a RPE running with the snaplogic recommended data science docker image. When i pass documents to the remote python script (hundreds, sometimes thousands) I only get around 1% of the documents back from the snap. Even when i literally pass the documents through the snap without any transforms, I get only 1% back.

whats going on here?
Any help with this is appreciated

1 ACCEPTED SOLUTION

JmPearl
New Contributor II

Ultimately i figured it out by digging into the log files produced by my RPE. The text being sent to the remote python code snap was not decoded properly, and therefor the RPE would break at the nth document which had faulty text. The problem was fixed by setting up a mapper snap that used HTML.decode on the text prior to the RPE snap.

View solution in original post

4 REPLIES 4

JmPearl
New Contributor II

More detail: this seems to only happen when i include a text field in the documents passed to the RPE

apranay
Employee
Employee

I tried RPE snap but I got all documents in snap output.
image

Can I know is there any specific data or fields.

JmPearl
New Contributor II

Ultimately i figured it out by digging into the log files produced by my RPE. The text being sent to the remote python code snap was not decoded properly, and therefor the RPE would break at the nth document which had faulty text. The problem was fixed by setting up a mapper snap that used HTML.decode on the text prior to the RPE snap.

koryknick
Employee
Employee

@JmPearl - this may have been a faster diagnosis if you enable both the error view and second output view of the Remote Python Script snap. But checking your script logs was a great choice.

Expected output:

  • First output view: The output from the Python script’s execution.
  • Second output view: The STDOUT redirected from the RPE.