hina_walia
New Contributor
Joined 6 years ago
User Profile
User Widgets
Contributions
Same column names in dat file
Hi, I have pipe limited dat file. I am using csv parser to get columns. The dat file has same column name(description) more than once but value is different eg- id name company cd description job description department description orgunit description The issue is when i do csv parser and preview data, i get description column only once (which is first in place that is after company cd) . The description column after department and org unit does not appear while doing preview. Can you help here. If i try to convert dat manually in csv(pipe delimited) , i get all columns but pipeline somehow skipping it.Please advise4KViews0likes3CommentsSelect count not working base in variable
Hi, I want to pass the filename from parameter(pipeline-parameter) and check if that file is already loaded or not. If not loaded, then we go ahead to load it otherwise we do nothing. Actually i have parent job and then this is child job , so pipeline parameter is from parent, it works fine without this file check step, so with file check step below is the issue:- Below is the sql in snowflake execute SELECT count(*) as cnt FROM FDR_SOURCE.KRONOS_LABORtest WHERE SOURCE_FILENAME = _Name If i pass some fixed value, instead of _Name, then pipeline work fine. if i use param (_Name), it throws below error. PLease see attached screenshot Param set in “pipeline parameter”3KViews0likes3CommentsRe: Get latest file from sftp and process one at a time
If i hardcode value in child pipeline it works, but variable doesn’t work. Please see attached screenshot, first is child pipeline set parameter and second is reading variable from file reader in child pipeline . Can you please check if syntax is fine. Thanks in advance.792Views0likes0Comments