Automated Deployment (CICD) of SnapLogic assets with GitHub
Introduction This guide is a reference document for the deployment of SnapLogic assets to a GitHub repository. It also includes sample YAML code for a GitHub Actions workflow which can be used to automate the deployment of assets across orgs (Dev -> Stg / Stg -> Prod, etc.) This guide is targeted towards SnapLogic Environment Administrators (Org Administrators) and users who are responsible for the deployment of SnapLogic assets / Release management operations. Section B covers automated deployment with GitHub Actions, and Section A illustrates a manual deployment flow using the Manager interface. Author: Ram Bysani SnapLogic Enterprise Architecture team SnapLogic Git Integration Git Integration allows you to track, update, and manage versions of SnapLogic assets using the graphical interface or the public APIs. The following asset types can be tracked in a GitHub repository: Accounts Files Pipelines Tasks Git model A) Asset deployment across environments - an example The example in this document illustrates a sample deployment of SnapLogic assets from the Dev environment (org) to the Prod environment. A similar methodology can be adopted to deploy assets from Dev -> Stg -> Prod environments. The environments should be configured for Git integration with GitHub. Please refer to the steps in the documentation. Git Integration Git operations The assets in this example are tracked at a project space level, i.e. one Project Space in Dev is associated with a single branch in the GitHub repository. A single GitHub repository is used to maintain the branches for Dev, Stg, Prod, etc. Repository branches can also be deleted and re-created for specific deployment needs. New / Modified Assets in the Dev Org Project Space: Dev_Integration_Space with the below project folders having SnapLogic assets. Integration_Project_1, Integration_Project_2, share Prod Environment We have already defined an empty project space named Prod_GH_Integration in the Prod org. This step can also be done by using the SnapLogic public API. Project APIs. Define branches in the GitHub repository Create individual branches in the GitHub repository for the Dev and Prod project space assets. You can choose the main branch as the default branch while creating Dev_GH_Space. Choose the Dev_GH_Space branch as the source when creating the Prod_GH_Space branch. Each branch in the GitHub repository corresponds to a Project Space in SnapLogic. e.g.: Dev_GH_Space, Prod_GH_Space Commit Dev assets to GitHub Connect to the Dev (source) environment in the SnapLogic Manager interface, and navigate to the project space named Dev_GH_Integration_Space. Right click and select Git Repository Checkout. Choose the Git repository branch Dev_GH_Space. You can see that the Git status has changed to Tracked for all assets under the child projects. Note that some assets appear with status Untracked as these were already existing in the main branch. These assets would not be committed to the Git repository. Notice the tracking message with the branch name and commit id next to the project space name: Tracked with Git repository: byaniram/RB_Snaprepo/heads/Dev_GH_Space, commit: 9a22ac8 Connect to the GitHub repository and verify the commit status for the branch Dev_GH_Space. Create Pull Request in GitHub At this step, you would need to create a Pull Request in GitHub. Choose Prod_GH_Space as the base branch, and Dev_GH_Space as the compare branch, and create the Pull request. This action would merge the assets contained in the Dev_GH_Space branch into the Prod_GH_Space branch. Connect to the GitHub repository and verify the commit status for the branch Prod_GH_Space. The assets have now been committed to the Prod environment and are tracked in the GitHub repository under the branch - Prod_GH_Space. It is also possible to merge and pull from additional branch(es) into a single Prod_GH_Space if you have a need for it. You would need to repeat the Pull / Merge process as above with the base branch being Prod_GH_Space, and the compare branch being one of Dev_GH_Space, Dev_GH_Space_1, or Dev_GH_Space_2. Pulling / Committing assets into the Prod Org Connect to the Prod (target) environment in the SnapLogic Manager interface, and navigate to the project space named Prod_GH_Integration_Space. Right click and select Git Repository Checkout. Choose the Git repository branch Prod_GH_Space. Choose Git Pull to pull the assets into the Project space. The assets from the Dev_Integration_Space project space of the Dev environment are deployed to the Prod_Integration_Space project space of the Prod environment. Notice the tracking message with the branch name and commit id next to the project space name: Tracked with Git repository: byaniram/RB_Snaprepo/heads/Prod_GH_Space, commit: ce0c368 For subsequent deployments of changed assets, you would first do a Commit to Git for the project space in the SnapLogic Dev environment, followed by the above steps. Changed assets would be visible with a Git status of ‘Tracked, Modified locally’ in the SnapLogic Manager. B) Deployment Automation using a GitHub Actions Workflow Actions workflow YAML sample A GitHub Actions workflow can be used to automate the deployment of assets across SnapLogic environments (such as Dev to Stg, Stg to Prod, etc.). A workflow is a configurable automated process made up of one or more jobs. You must create a YAML file to define your workflow configuration. Here’s a complete YAML file for the Dev -> Prod deployment example described in Section A above. The complete YAML file is attached for your reference. Please create a new Workflow from the Actions tab, and paste the contents of the file in the workflow editor and commit changes. # Actions workflow for automated deployment of SnapLogic assets name: SnapLogic CICD Sample on: push: branches: - Dev_GH_Space # Uncomment the below line if you need to execute the workflow manually. # workflow_dispatch: jobs: pull_merge_branches: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v4 - name: Merge Dev to Prod uses: devmasx/merge-branch@master with: type: now from_branch: Dev_GH_Space target_branch: Prod_GH_Space github_token: ${{ secrets.ACTIONS_TOKEN }} - name: Checkout project assets to Prod project space run: | curl -s -X POST \${{vars.SNAP_URL}}/api/1/rest/public/project/pull/${{vars.SNAP_ORG}}/${{vars.PROJECT_SPACE}} \ -H "Content-Type:application/json" -H "Authorization:Basic ${{secrets.BASE64_TOKEN}}" \ -d '{"use_theirs":"true"}' Please refer to the GitHub documentation for information related to Workflow usage and syntax: GitHub Workflows Workflow syntax The following table provides clarification on certain aspects of the sample workflow for better understanding. Section Comments runs-on: ubuntu-latest runs-on defines the runner (type of machine) to use to run the job. ubuntu-latest specifies a GitHub hosted runner image. GitHub hosted runners uses: actions/checkout@v4 checkout is an action which is available in the GitHub marketplace. This action checks out the repository for use. v4 is the version number of the action. https://github.com/marketplace/actions/checkout uses: devmasx/merge-branch@master merge-branch is an action from the GitHub marketplace. This action runs a Git merge operation. https://github.com/marketplace/actions/merge-branch It also requires you to define a personal access token (classic) under Developer Settings -> Personal access tokens. Select both the repo and workflow checkboxes. curl -s -X POST \ ${{vars.SNAP_URL}}/api/1/rest/public/project/pull/${{vars.SNAP_ORG}}/${{vars.PROJECT_SPACE}} \ -H "Content-Type:application/json" -H "Authorization:Basic ${{secrets.BASE64_TOKEN}}" \ -d '{"use_theirs":"true"}' This is a CURL command that executes the SnapLogic public API to pull the latest project files from Git. See Pull the latest project files from Git. The referenced variables are defined on the GitHub repository under Settings -> Secrets and variables -> Actions. The vars context is used to reference those variables. (e.g. SNAP_ORG, PROJECT_SPACE) You can also define encrypted Secrets for sensitive data and reference them using the secrets context as in the example. (e.g. BASE64_TOKEN has the base64 encoded string for username and password). Workflow Variables Table 1.0 - Workflow Actions Workflow execution The above Actions workflow will be automatically executed whenever there is a “Push” / “Git Commit” operation to the Dev_GH_Space branch. i.e. whenever a commit is done from the Dev SnapLogic environment project space. The workflow will execute the pull-merge operation to the Prod_GH_Space branch, and pull the latest project assets into the Prod SnapLogic environment. The YAML file must be created under the .github/workflows folder of the Dev_GH_Space branch in the GitHub repository. The workflow run status will be visible under the Actions tab. Note: If you wish to manually execute the pull-merge post code review, then you can uncomment the two lines in the script to enable workflow_dispatch, and execute the Actions workflow manually from the Actions tab on GitHub. # Uncomment the below line if you need to execute the workflow manually. # workflow_dispatch: You can edit and modify the YAML file as per your requirements. Subsequent commits and deployments from Dev->Prod can be automated similarly. Action Comments Developer commits new code or updates assets in the Dev org to the GitHub repository SnapLogic Dev org Manager Interface Asset -> Add to repository. Ensure status shows Tracked Project Space -> Commit to Git Create and merge Pull Request Create a new Pull Request on GitHub, and merge the newly committed assets by choosing the Prod branch as the base, and the Dev branch as the compare branch. Pull the updated assets into the Prod org SnapLogic Prod org Manager Interface Project Space -> Git Pull Table 2.0 - Steps for subsequent / future asset deployment Deployment flow (Dev->Test->Prod) Note: Future versions of this document will cover additional deployment scenarios. Please post your comments on the article.
