I am building a generic error pipeline which will load error occuring in the source pipeline and write them/load them into a database. It will load data such as label, user, etc. on which the error takes place. In source pipeline, within within the p...
Hi,I first do an SQL Server - Execute and get this weird {_snaptype_localdatetime: "..."} for the CreationDate and LastUpdate fields. Then, using a Mapper snap I do toString() to just get the date fields in string form. Then, I do a Group By on some ...
Hello,My input is S3, where we are receiving files of different objects. Our target is to read these files and insert data into postgres DB where we have created target tables for all objects. How can we dynamically map all the objects using one con...
Hi, I need to use null and empty condition in router. curretnly i am using:- $ZSDATA.IDOC.EDL2.EDL4.BEL != “” but i feel i am not handling it correctly. Any inputs?
Hi, I have a REST API where the page index is passed in the request body. There is no indication of total records, so you must increment the page index in the request body until no records are returned. I cannot see how to do this via the pagination ...
I have a use case to load some large CSV files (in GBs) from windows network drive to Snowflake. Used a CSV parser and loading it using Snowflake bulk load. Pipeline is running without any issues. However, it takes a lot of time to load the data. Is ...
Hi Everyone,I created a user with administrator privileges to invoke pipelines in a project using a username and password. However, I now want to reduce the user's role to a lower level.Before making this change, I want to ensure it won't disrupt any...
I have a situation something like this: Employee IDEffective DateOffice11/1/2023San Francisco11/1/2024Oakland26/1/2023Sydney212/1/2023Tokyo I need to add an order column and end date of the previous record column. Can you please suggest any ways of ...
