Forum Discussion

del's avatar
del
Contributor III
8 years ago
Solved

Salesforce - Output field limit

We have a use case to pull all fields from wide objects in Salesforce. I keep running into “field limit” errors: .
Does anyone have any logic or best practice for pulling in all fields from wide Salesforce objects?

  • sriram's avatar
    sriram
    8 years ago

    Are you using the “Salesforce Read” and/or “Salesforce SOQL” snap to query wide objects in Salesforce? If yes, internally the read operation would be translated to a SOQL query equivalent for which there is a character count limitation from Salesforce (currently it is 20000 characters) and is documented here:

    Reference:

    If there is a need to query a wide Salesforce object which in turn causes the SOQL query translation/equivalent to exceed the query character count threshold, it would be best to leverage the “Output field limit” and “Output field offset” properties available on the “Salesforce Read” snap.

    Reference:
    https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439267/Salesforce+Read

    For example:
    Modify a pipeline using a single “Salesforce Read” snap querying a wide object into ‘N’ number of “Salesforce Read” snaps and specify the required limit/offset by ensuring that the equivalent SOQL query will be within the maximum allowed character limit. Document streams from the above branches can then be Union’d and/or Joined for downstream processing.

3 Replies

  • sriram's avatar
    sriram
    Former Employee
    8 years ago

    Are you using the “Salesforce Read” and/or “Salesforce SOQL” snap to query wide objects in Salesforce? If yes, internally the read operation would be translated to a SOQL query equivalent for which there is a character count limitation from Salesforce (currently it is 20000 characters) and is documented here:

    Reference:

    If there is a need to query a wide Salesforce object which in turn causes the SOQL query translation/equivalent to exceed the query character count threshold, it would be best to leverage the “Output field limit” and “Output field offset” properties available on the “Salesforce Read” snap.

    Reference:
    https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439267/Salesforce+Read

    For example:
    Modify a pipeline using a single “Salesforce Read” snap querying a wide object into ‘N’ number of “Salesforce Read” snaps and specify the required limit/offset by ensuring that the equivalent SOQL query will be within the maximum allowed character limit. Document streams from the above branches can then be Union’d and/or Joined for downstream processing.

    • del's avatar
      del
      Contributor III
      8 years ago

      Thank you @sriram, that’s very helpful. I would want to capture all this recursively via a pipeline, but I don’t necessarily know how many columns on any given object I have to tackle. As a bonus question 🙂, do you know of a way to determine the column count from a Salesforce object? (My Google searches keep leading me to APEX code)

      • sriram's avatar
        sriram
        Former Employee
        8 years ago

        Looks like Salesforce does not expose a SOQL query-like capability to determine field count for an object, besides letting users do it in programmatic way that you have come across already. To handle this limitation / situation where in we are unable to determine the number of fields non-programatically, it would best to break the pipeline to include multiple “Salesforce Read” snaps (with Output field / Output field offset specified) to ensure that the query is well within the 20,000 character count threshold.