How to remove blank rows from an input flat file?

[darshthakkar]

Hi Team,

How can I remove all the blank rows from an input flat file? I cannot use a condition restricted on a single column as the other records are needed. All I want to do is filter out ALL ROWS that doesn’t have any data (like Row 4, 5, 7, 8 shown in the screenshot below)

Desired Output:

Thus, final output should be:

Thanking in advance.

Regards,
DT

[pmancevski]

Hi @darshthakkar,

I think also it will do the trick

$.entries() != []

Thanks,
Pero M.

[darshthakkar]

pmancevski:

$.entries() != []

Thank you @pmancevski for your suggestion and assistance on this one. I tested with your solution and unfortunately it didn’t work.

[darshthakkar]

Thank you @ddellsperger, I will definitely give this a try.
Before posting this to community, I tried with $.filter((v,k)=> v != null) but didn’t get any success.

Will keep you posted with my findings. Thanks again.

Regards,
Darsh

[darshthakkar]

ddellsperger:

$.entries().filter(v => v[1] != “”).length > 0

@ddellsperger: Voila, it worked! Thank you for your assistance. I will now remove my conventional filter which has the condition of ! = null on each column in the file (it was never a good design but my only resort until we could find a best solution).

Thanks again.

BR,
Darsh

[darshthakkar]

@ddellsperger: Is it a safe assumption that the below solution provided by you is similar to the conventional filter I used (which was putting a NOT NULL condition on each column?) Q-1

I believe my conventional filter would be a lot of overhead on the system + this time, I only had to deal with 10 columns (so 20 conditions, one as !=null and another as !=“”) however if I had to work on 350+ columns (which is my ideal day scenario), conventional filter would have busted.

I want to take this opportunity to learn from an expert like you that what would be the overhead (time & space complexity) while using $.entries().filter(v => v[1] != “”).length > 0? Q-2

Is the above a recommended approach and the ONLY approach to filter out blank rows from a flat file? Q-3

Looking forward to hearing from you.

BR,
Darsh