2021 Enterprise Automation Award Nomination: Pitney Bowes: Building an Enterprise Data Warehouse and Data Lake

What were the underlying reasons or business implications for the need to automate business processes? Describe the business and technical challenges the company had.

Following were some of the challenges that we had which implicated the need to automate our business process: a)Duplication and Redundancy in the process related to Development, Monitoring and Troubleshooting, b)Different Logic/Platform/Environment to gather the same set of data resulting in erroneous results, c)No common documentation of the process, d) no single Team/Owner of the process e)Delay in Time to completion

Describe your automation use case(s) and business process(es).

Pitney Bowes wanted to centrally get all of their enterprise data so that it will ensure that individual teams/users don’t need to get the data on their own, they don’t need to develop separate pipeline to get the data and the data is fully governed so that people have full trust on the data. For that PB decided to create an Enterprise Data Lake in Amazon S3. In order to achieve this Objective PB selected SnapLogic Integration Platform after reviewing at least 3 other ETL tools. Using SnapLogic PB was able to achieve it’s Data Lake Objective to centrally get all of their Enterprise Data within a period of 6 months from the time of getting the initially License of the Product. Following were some of the other use cases: a)Curating Data from S3 in the Enterprise Data warehouse - Snowflake b)Aggregating Data from Salesforce and SAP to create create a full order cycle process c)Gathering data from Third Party APIs related to social media events d)Creating various backup processes to have the data moved from Relation Databases into a less costly storage which is S3 thereby reducing cost and also maintain active data in Relational databases and Historical data into S3.

Describe how you implemented your automations and processes. Include details such as what SnapLogic features and Snaps were used to build out the automations and processes.

In order to achieve our Centralized Governed Data Lake Objective we used SnapLogic Automation Integration Platform to move data from various sources like, Salesforce, SAP_HANA, GUAM, OKTA, Oracle, SQL Server, Mongo, MySQL, REST API, SumoLogic, etc which made used of the Core Snaps like REST, JDBC and Specialized Data Snap like Salesforce, Oracle, SQL Server, SAP_HANA, Mongo DB, SumoLogic into our S3 (AWS S3 Data Lake) and then eventually move this data into the Enterprise Data warehouse Snowflake.

What were the business results after executing the strategy? Include any measurable metrics (ie. productivity improvement, speed improvement, % reduced manual inefficiencies, etc.)

Using SnapLogic the whole cycle from Development to Time to Completion improved by nearly 90%. The flexibility, speed and ease of use of the SnapLogic Platform allowed us to remove the time based bottleneck of getting the data into the hands of the Analyst and Data consumers resulting in increase in productivity by more than 70% Also process automation reduced the error percentage down to very negligible number

Who was and how were they involved in building out the solution? (Please include the # of FTEs, any partners or SnapLogic professional services who were involved on the implementation)

Myself along with 2 of my colleagues handled the initial Implementation who basically were responsible in developing and migrating the existing business process. There was assistance provided by SnapLogic Support team. We created various automated reusable templates which allowed teams even today to move their data from any source to any destination. We created configuration templates that these automated template pipeline uses to perform the Data movement. Error Trapping and automated Correction including Automated Retry were built into the templates. Today Pitney Bowes has nearly 250-300 users creating solution using the SnapLogic Platform.

Anything else you would like to add?

PB Investment in SnapLogic Integration Platform which is a true Enterprise Platform in the cloud has reaped tremendous benefits in speeding up their Business Objectives related to Data Movement and Data Availability resulting in increase growth and productivity.