12-04-2023 02:34 PM - edited 03-05-2024 02:25 PM
This document is a reference manual for common administrative and management tasks on the SnapLogic platform. This document is for SnapLogic Environment Administrators (Org Administrators) and users involved in supporting or managing the platform components.
Author:
Ram Bysani
SnapLogic Enterprise Architecture team
There are two reserved groups in SnapLogic:
admins: Users in this group have full access to all projects in the Org.
members: Users in this group have access to projects that they create, or to which they are granted access. Users are automatically added to this group when you create them, and they must be a part of the members group to have any privileges within that Org.
There are two user roles:
Environment admins are Org users who can manage the Org. Environment admins are part of the admins group, and this role is named “Org Admin” in the classic Manager.
Basic user. All non-admin users. Within an Org, basic users can create projects and work with assets in the Project spaces to which they have been granted permission. To gain Org administrator privileges, a Basic user can be added to the admins group.
The below table lists the various tasks under the different categories that an Environment admin user can perform:
Task |
Comments |
USER MANAGEMENT |
|
|
When a user is removed from an Org, the administrator that removes the user becomes the owner of that user's assets. |
MANAGER |
|
|
|
ALERTS and NOTIFICATIONS |
|
|
Reference: |
SNAPLEX and ORG |
|
|
Update or revert a Snaplex version. |
APIM |
|
|
Reference: |
AutoSync |
|
|
Reference: |
The classic IIP monitoring Dashboard interface can be launched from the Dashboard tab after logging on to SnapLogic IIP. The Dashboard has multiple tabs that displays information about various Snaplogic assets.
The new, enhanced Monitor interface can be launched from the Apps (Waffle) menu located on the top right corner of the page.
Note: Presently, only some of the features of the classic Dashboard are available in the new Monitor App, and all features will eventually be ported over. We strongly recommend to use the Monitor app over the classic Dashboard for any functionality that is currently available in the Monitor as listed in the table below.
The following table lists some common administration and monitoring tasks (grouped by category) for both the classic Dashboard and the Monitor App.
Task |
Classic Dashboard |
Monitor App |
ORG |
||
|
Dashboard -> Insights Wall
Note that the Insights tab is only accessible to Org admins. |
N/A |
PIPELINES |
||
|
Dashboard -> Health wall -> Pipeline Health wall |
Monitor -> Execution tab -> Execution Overview -> Pipeline |
|
Dashboard -> Health wall -> Pipeline Health wall Dashboard -> Pipeline wall |
Monitor -> Execution tab -> Execution Overview -> Pipeline |
|
Dashboard -> Pipeline wall |
N/A |
|
Dashboard -> Insights wall |
N/A |
TASKS |
||
|
Dashboard -> Task |
Monitor -> Execution tab -> Execution Overview -> Task |
|
Dashboard -> Task |
N/A |
SNAPLEX |
||
|
Dashboard -> Snaplex |
Monitor -> Infrastructure -> <node> -> Metrics and Monitor -> <node> -> Metrics |
|
Dashboard -> Health wall |
Monitor -> Infrastructure -> <node> -> Alerts |
|
Dashboard -> Health wall -> Snaplex -> <node> |
Monitor -> Infrastructure -> <node> |
API |
||
|
Dashboard -> API Insights wall |
Monitor -> Execution -> Filter by Invocation Type = API calls The Monitor only allows to view the API execution details for a specific time period. |
Additional reference:
Insights
Health
Pipeline
Snaplex
API
Monitor Interface
The jcc script is a command-line tool that provides a set of commands to manage the Snaplex nodes. This utility is installed in the /opt/snaplogic/bin directory of the Groundplex node. The below table lists the commonly used arguments for the jcc script (jcc.sh on Linux and jcc.bat on Windows). Note that the command would list other arguments (for example, try-restart). However, those are mainly included for backward compatibility and not frequently used.
$SNAPLOGIC refers to the /opt/snaplogic directory on Linux or the <Windows drive>:\opt\snaplogic directory on Windows servers. Run these commands as the root user on Linux and as an Administrator on Windows.
Example:
sudo /opt/snaplogic/bin/jcc.sh restart or c:\snaplogic\bin\jcc.bat restart
Argument |
Description |
Comments |
status |
Returns the Snaplex status. The response string would indicate if the Snaplex Java process is running. |
|
start |
Starts the Snaplex process on the node. |
|
stop |
Stops the Snaplex process on the node. |
|
restart |
Stops and restarts the Snaplex process on the node. |
Restarts both the monitor and the Snaplex processes. |
diagnostic |
Generates the diagnostic report for the Snaplex node. The HTML output file is generated in the $SNAPLOGIC/run/log directory. |
Resolve any warnings from the report to ensure normal operations. |
clearcache |
Clears the cache files from the node. This command must be executed when the JCC is stopped. |
|
addDataKey |
Generates a new key pair and appends it to the keystore in the /etc/snaplogic folder with the specified alias. This command is used to rotate the private keys for Enhanced Account Encryption. |
Doc reference: |
The following options are available for a Groundplex on Windows server. |
||
install_service remove_service |
The jcc.bat install_service command installs the Snaplex as a Windows service. The jcc.bat remove_service command removes the installed Windows service. |
Run these commands as an Administrator user. |
Custom log file configuration is occasionally required due to internal logging specifications or to troubleshoot problems with specific Snaps.
In the following example, we illustrate the steps to configure the log level of ‘Debug’ for the Azure SQL Snap pack. The log level can be customized for each node of the Groundplex where the related pipelines are executed, and will be effective for all pipelines that use any of the Azure SQL Snaps (for example, Azure SQL - Execute, Azure SQL - Update, etc.). Note that Debug logging can affect pipeline performance so this configuration must only be used for debugging purposes.
Note: You can perform Step 2 by adding the property key and value under the Global Properties section.
Example:
Key: jcc.jvm_options
Value: -Dlog4j.configurationFile=/opt/snaplogic/logconfig/log4j2-jcc.xml
The Snaplex node must be restarted for the change to take effect. Refer to the commands in Table 3.0.
b. Edit the log4j2-jcc.xml file configured in Step a.
c. Add a new RollingRandomAccessFile element under <Appenders>. In this example, the element is referenced with a unique name JCC_AZURE. It also has a log size and rollover policy defined. The policy would enable generation of up to 10 log files of 1 MB each. These values can be adjusted depending on your requirements.
<RollingRandomAccessFile name="JCC_AZURE"
fileName="${env:SL_ROOT}/run/log/${sys:log.file_prefix}jcc_azure.json" immediateFlush="true" append="true"
filePattern="${env:SL_ROOT}/run/log/jcc_azure-log-%d{yyyy-MM-dd-HH-mm}.json”
ignoreExceptions="false">
<JsonLogLayout properties="true"/>
<Policies>
<SizeBasedTriggeringPolicy size="1 MB"/>
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingRandomAccessFile>
…
…
</Appenders>
d. The next step is to configure a Logger that references the Appender defined in step #c. This is done by adding a new <Logger> element. In this example, the Logger is defined with log level = Debug.
<Logger name="com.snaplogic.snaps.azuresql" level="debug" includeLocation="true" additivity="false">
<AppenderRef ref="JCC_AZURE" />
</Logger>
..
..
<Root>
…
</Root
</Loggers>
</Configuration>
The value for the name attribute is derived from the Class FQID value of the associated Snap.
The changes to log4j2-jcc.xml are marked by the highlighted text in steps c and d. The complete XML file is also attached for reference. You can refer to the Log4j documentation for more details on the attributes or for additional customization.
Additional debug log messages will be printed to the pipeline execution logs for any pipeline with Azure SQL Snaps. These logs can be retrieved from Dashboard.
Example:
{"ts": "2023-11-30T20:21:33.490Z", "lvl": "DEBUG", "fi": "JdbcDataSourceRegistryImpl.java:369", "msg": "JDBC URL: jdbc:sqlserver://sltapdb.database.windows.net:1433;database=SL.TAP;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;authentication=sqlPassword;loginTimeout=30;connectRetryCount=3;connectRetryInterval=5;applicationName=SnapLogic (main23721) - pid-113e3955-1969-4541-9c9c-e3e0c897cccd, database server: Microsoft SQL Server(12.00.2531), driver: Microsoft JDBC Driver 11.2 for SQL Server(11.2.0.0)", "snlb": "Azure+SQL+-+Update", "snrd": "5c06e157-81c7-497f-babb-edc7274fa4f6", "plrd": "5410a1bdc8c71346894494a2_f319696c-6053-46af-9251-b50a8a874ff9", "prc": "Azure SQL -
The updated log configuration would also write the custom JCC logs (for all pipelines that have executed the Azure SQL Snaps) to disk under the /opt/snaplogic/run/log directory. The file size for each log file and the number of files would depend on the configuration in the log4j2-jcc.xml file.
The changes to log4j2-jcc.xml can be reverted if the additional custom logging is no longer required.
The default log level for a new Snaplex is ‘Debug.’ This value can be updated to ‘Info’ in Production Orgs as a best practice.
The available values are:
Reference: Snaplex logging