I am trying to set up a slowly changing dimension process in Snaplogic using the Snowflake SCD2 snap. This process will read a file on a daily basis and provide historical data based on the status the record is currently in for that day’s run (I.E. Open, Reviewed, Pending, Closed, etc) so we can determine the number of days the record is in each of the statuses.
I have added the columns the SCD process needs to store the historical information. I have the Snaplogic pipe line set up using the Snowflake SCD snap connecting to the Snowflake Azure DB account.
When I process the first file, everything works and the data is loaded to the table as I expect.
When I try to process another file, I am receiving the below error message and the pipeline will not validate. Does anybody know what this error is trying to tell me? I set up the SCD2 snap based on Snaplogic documentation.
The Start and End Date calculation is just current date that I am converting to Eastern time.
In reviewing the error message, I am not seeing any indication on the Start and End Time using any components of the joda time. The Auto Historization query is based on the LAST_CASE_UPDATE which is the date the status changes. This field is defined as Timestamp_NTZ(0).
ERROR:
Failure: Failed to close database cursor connection., Reason: class java.lang.String cannot be cast to class org.joda.time.DateTime (java.lang.String is in module java.base of loader ‘bootstrap’; org.joda.time.DateTime is in unnamed module of loader java.net.FactoryURLClassLoader @701fc37a), Resolution: Please ensure the database can still be accessed.
FAILED VALIDATION:
SCD FIELDS:
Snowflake Table
Start and End Date field
Date.now().toLocaleDateTimeString({“timeZone”:“US/Eastern”,“format”:“yyyy-MM-dd hh:mm:ss”})
