Forum Discussion

debra_paponette's avatar
debra_paponette
New Contributor
5 years ago

Duplicate data using SQL offset

We are attempting to load transactional tables on a pipeline we created

Issue faced - While loading data from a transactional table by leveraging the help of limit and offset in SQL SERVER select snap (partition and loading), we are getting duplicate records on the destination table.

Cause expected – While loading data, since the source table is getting updated(insertion, deletion and updates) limit and offset values may change with the data mapping, resulting in duplication of reads.

We looked at the documentation to understand how the offset feature works but not much data there. Can someone share some further insight on this feature and how it works that may be contributing to the above issue

  • Hi maheswara ,

    The issue might be the JDBC Driver. From the screenshot you shared, I am not sure if you have configured the JDBC JARs Driver, but if you didn't, you need to download it and upload it in SnapLogic and then under the JDBC JARs configuration, upload/select the .jar file of the driver. You can download the latest driver or any version you need.

    You can download it from the official docs: https://jdbc.postgresql.org/download/

     

    • maheswara's avatar
      maheswara
      New Contributor II

      Hi j_angelevski,

      The above solution what you have given is working fine, but some time i am getting fatal error. Not able to connect that PostgreSQL.

      • j_angelevski's avatar
        j_angelevski
        Contributor III

        Hi maheswara ,

        Could you please provide more details about the error? It's a bit vague and could refer to various issues. Would you mind sharing the specific error message for clarification?