arvindnsn
Contributor
Joined 8 years ago
User Profile
User Widgets
Contributions
ELT Load Snap Failure
Hello, I have a process to load million of data from one snowflake table to another snowflake table (possibly from different Databases within Snowflake due to security and sensitive data). Both the source and the target table has almost 300 columns. Since it has millions of data, I am using ELT Snaps to keep the process Native as target execution will be done directly in Snowflake. I have set the Load action to "Drop and Create table" in the ELT Load Snap settings as this is similar to Truncate and Load. I am using ELT Load Snap to load from source to Target, but I am getting the below error as provided in the scrennshot as "Failure: Invalid Source Location, Reason: Source Location in Account seems None, Resolution: Provide a valid Source Location" Why is the ELT Load snap is failing and possible solution for that. Below is the screeshot of the Pipeline. I could have Truncate the target table process outside the ELT process and used the "ELT MERGE INTO" Snap, but the reason for not using that is that i have to mention all the 300+ columns in the "ELT MERGE INTO" snap and it is very tedious and not sustainable as the source and target table could grow in columns and this will lead to change in the pipeline for ELT MERGE INTO snap everytime there is a change in columns. Thanks Aravind NParsing CSV file
Hello, Below is the requirement where I have to read and parse a CSV file and load each line based on the first column's name. The file will have thousands and thousands of lines. I have the sample csv file below and 1st line will be Header and will have 7 columns and the main body will have 13 columns and there will be trailer line at the end and that will have 3 columns. "HEADER","20250219","053545","108","GRN","AX","4.01" "SUMMARY","101","","A19","220","USD","19"," 3868","3868"," 102"," 1","0"," P" "SUBMISSION","101","","A19","220","USD","124","217","217","170126","USD","0500167400","00000010" "TRANSACTN","102","","A19","220","USD","124","217","217","170126","USD","0500167400","00000010" "TXNPRICING","103","","A19","220","USD","124","217","217","170126","USD","0500167400","00000010" "TRAILER","0000000108","0000058371" I have to parse the above csv file and load each line into separate table based on the name of the first column. For example if the first column is "HEADER", the line has to be parsed and based on each comma separated value, it has to be loaded into Header table. If the fist column is "SUMMARY", then the each comma separated value of that line has to be parased and loaded into Summary table. I am trying the CSV parser but getting the error as below "Reason: The number of CSV data fields is greater than the number of header columns in the input data: number of CSV data fields = 19, number of header columns = 7, Resolution: Please make sure the number of CSV data fields is not greater than the number of header columns." How to parse this and what would be the best way to achieve this? Thanks in advance. Thanks Aravind NData Validator Pattern error
Hello, I am building an API that used PUT method to update a set of JSON data into my taget database. One of the contents of the PUT method is a boolean type that would receive ether a "true" or "false" and is a mandatory entity. I am using a Data Validator Snap with the "Pattern" constraint, but i am getting the following error. Below is the error, I am getting Do i have to do any conversion of the incoming into a string? How do i resolve this issue? Thanks Aravind NHow to find if the entity exists in an array or not
Hello, I have a scenario where i use the Gate Snap and the output of the snap would be a empty array or list of entities in the array as shown below. If the array is empty, it would give output with "original" values ( i need this to take the values in the forward snaps) and if the array is not empty, it would have the below output. One of the entity in the array is "Name" and i am trying to use "hasPath" function to check if the array is empty or not but in vain. Since "Original" values are coming along, even if the array is empty, it is giving 1 as an output if i use the "length" function. I would like to find a solution to check if the array is empty or not by checking the existence of one of the entities that the array returns. Thanks Aravind N
