Salesforce - Output field limit

[del]

We have a use case to pull all fields from wide objects in Salesforce. I keep running into “field limit” errors: .
Does anyone have any logic or best practice for pulling in all fields from wide Salesforce objects?

[sriram · Accepted solution]

del:

We have a use case to pull all fields from wide objects in Salesforce. I keep running into “field limit” errors:

Are you using the “Salesforce Read” and/or “Salesforce SOQL” snap to query wide objects in Salesforce? If yes, internally the read operation would be translated to a SOQL query equivalent for which there is a character count limitation from Salesforce (currently it is 20000 characters) and is documented here:

Reference:

developer.salesforce.com

Salesforce Developers

Salesforce Developer Website

del:

Does anyone have any logic or best practice for pulling in all fields from wide Salesforce objects?

If there is a need to query a wide Salesforce object which in turn causes the SOQL query translation/equivalent to exceed the query character count threshold, it would be best to leverage the “Output field limit” and “Output field offset” properties available on the “Salesforce Read” snap.

Reference:
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439267/Salesforce+Read

For example:
Modify a pipeline using a single “Salesforce Read” snap querying a wide object into ‘N’ number of “Salesforce Read” snaps and specify the required limit/offset by ensuring that the equivalent SOQL query will be within the maximum allowed character limit. Document streams from the above branches can then be Union’d and/or Joined for downstream processing.

[del]

Thank you @sriram, that’s very helpful. I would want to capture all this recursively via a pipeline, but I don’t necessarily know how many columns on any given object I have to tackle. As a bonus question 🙂, do you know of a way to determine the column count from a Salesforce object? (My Google searches keep leading me to APEX code)

[sriram]

del:

do you know of a way to determine the column count from a Salesforce object?

Looks like Salesforce does not expose a SOQL query-like capability to determine field count for an object, besides letting users do it in programmatic way that you have come across already. To handle this limitation / situation where in we are unable to determine the number of fields non-programatically, it would best to break the pipeline to include multiple “Salesforce Read” snaps (with Output field / Output field offset specified) to ensure that the query is well within the 20,000 character count threshold.