RPE only returns small proportion of documents

[JmPearl]

I have a RPE running with the snaplogic recommended data science docker image. When i pass documents to the remote python script (hundreds, sometimes thousands) I only get around 1% of the documents back from the snap. Even when i literally pass the documents through the snap without any transforms, I get only 1% back.

whats going on here?
Any help with this is appreciated

[JmPearl]

More detail: this seems to only happen when i include a text field in the documents passed to the RPE

[apranay]

I tried RPE snap but I got all documents in snap output.

Can I know is there any specific data or fields.

[JmPearl · Accepted solution]

Ultimately i figured it out by digging into the log files produced by my RPE. The text being sent to the remote python code snap was not decoded properly, and therefor the RPE would break at the nth document which had faulty text. The problem was fixed by setting up a mapper snap that used HTML.decode on the text prior to the RPE snap.

[koryknick]

@JmPearl - this may have been a faster diagnosis if you enable both the error view and second output view of the Remote Python Script snap. But checking your script logs was a great choice.

Expected output:

• First output view: The output from the Python script’s execution.
• Second output view: The STDOUT redirected from the RPE.