Configuring the Script Snap to use a configured HTTP proxy environment variable

[robin]

HTTP-compatible Snap Packs can leverage an HTTP proxy configured in the Snaplex’s Network Proxies configuration tab within the SnapLogic Manager web application.

However, the Script Snap is different because you can write Scripts to call external processes (e.g. curl) and these will not be aware of any proxy configuration set within the SnapLogic application.

curl can be configured to use a proxy directly via the --proxy argument, but if you wished to enforce that proxy usage across all usages of the Script Snap, you can set the http_proxy and/or https_proxy environment variables within a special file - /etc/sysconfig/jcc.

Environment variables declared within this file will be visible to the Snaplex application (OS-level env vars will not be).

This file (and directory) may not exist in your Snaplex, so you may have to create them (similar to the instructions on the Configuring a custom JRE version page):

sudo mkdir -p /etc/sysconfig; sudo sh -c "echo 'export http_proxy=username:password@proxy-ip-address:port' >> /etc/sysconfig/jcc"

substituting the equivalent values for username/password (if authentication is required), proxy-ip-address, and port (you may also want to add https_proxy too).

Once this file is created, restart the Snaplex application (/opt/snaplogic/bin/jcc.sh restart or c:\opt\snaplogic\bin\jcc.bat restart) and the http_proxy/https_proxy environment variable will now be active within the SnapLogic product.

Assuming your proxy is correctly configured, you can then run your Script that calls the external process and, if the process supports using a proxy, it will respect the setting.

For example, the following Script Snap (Python) uses the subprocess library to execute curl and adds the response body to the output document.

# Import the interface required by the Script snap.
from com.snaplogic.scripting.language import ScriptHook
import subprocess

class TransformScript(ScriptHook):
    def __init__(self, input, output, error, log):
        self.input = input
        self.output = output
        self.error = error
        self.log = log

    # The "execute()" method is called once when the pipeline is started
    # and allowed to process its inputs or just send data to its outputs.
    def execute(self):
        self.log.info("Executing Transform script")
        while self.input.hasNext():
            try:
                # Read the next input document, store it in a new dictionary, and write this as an output document.
                inDoc = self.input.next()
                proc = subprocess.Popen(['curl','https://www.snaplogic.com'], stdout=subprocess.PIPE)
                (out, err) = proc.communicate()
                outDoc = {
                    'original' : out
                }
                self.output.write(inDoc, outDoc)
            except Exception as e:
                errDoc = {
                    'error' : str(e)
                }
                self.log.error("Error in python script")
                self.error.write(errDoc)

        self.log.info("Script executed")

    # The "cleanup()" method is called after the snap has exited the execute() method
    def cleanup(self):
        self.log.info("Cleaning up")

# The Script Snap will look for a ScriptHook object in the "hook"
# variable.  The snap will then call the hook's "execute" method.
hook = TransformScript(input, output, error, log)

On execution, the proxy access log should show the request being routed through the proxy.

[stewartbulloch]

@robin I’ve not looked at configuring this but your guide covers restarting the application in a Windows environment but not where the equivalent /etc/sysconfig/jcc entries for the env variables should be stored. I’m assuming it might be in kcc.bat but it isn’t clear.

[robin]

@stewartbulloch yes it would be in the jcc.bat file but it’s a bit more complicated.

In that file you will see a line like:

%CYGRUNSRV% -I %SERVICE_NAME% -1 %SNAPLEX_LOG% -2 %SNAPLEX_ERR% -d %SERVICE_DISPLAY% -p %JAVA_EXE% -c %SNAP_HOME% -f "SnapLogic Snaplex (Java)" -a '%START_OPTIONS%' -o -V -e 'SL_ROOT=%SL_ROOT%' -e 'PATH=%PATH%' -e 'JAVA_HOME=%JAVA_HOME%'

I believe here you can add a new -e 'http_proxy=VALUE'/'https_proxy=VALUE' pair and, after the service is reinstalled with jcc.bat install_service; jcc.bat start, the environment variable should be available now in the Windows environment.

If indeed http_proxy (and its HTTPS equivalent) is the environment variable a certain Windows utility uses for proxy configuration, then it should work.

I’m less familiar with Windows OS though for this use case so I’d be interested to hear from anyone if this worked for them.