Forum Discussion
Another option is to use the matches
operator that can check a value against a pattern. The nice thing about this approach is that the pattern looks roughly like the value being matched and the type of the value is implicitly checked. So, an error won’t be raised if the value has an unexpected type.
For your first example:
The object pattern, { MessageId }
, would only match objects that contain the property MessageId
. The full expression would look like this:
$entity matches { MessageId }
For the second example:
A string pattern that uses the ...
wildcard can check for a value with a prefix. The full expression would look like:
$entity matches "ERROR:"...
(Unfortunately, while there is documentation for the match
operator, the documentation for matches
seems to be missing at the moment. We’ll get that fixed)
- Aleksandar_A3 years agoContributor III
Hi @darshthakkar,
After parsing the file try using a filter with the following expression
$ != {}
.Let me know if this helps,
BR,
Aleksandar.- darshthakkar3 years agoValued Contributor
Sure @AleksandarAngelevski , I will give it a try,
I tried with isEmpty() both on filter and router but it didn’t work, was planning to put a condition on all the columns as the last resort (but that is not a good design to be honest). - darshthakkar3 years agoValued Contributor
This is not fetching the desired results.
- ddellsperger3 years agoAdmin
This is maybe a little bit difficult, but you can put the following filter after your parser
The expression is:$.entries().filter(v => v[1] != "").length > 0
which will find any instance where all of the values for the given input document are all empty and filter them out.
- darshthakkar3 years agoValued Contributor
@ddellsperger: Voila, it worked! Thank you for your assistance. I will now remove my conventional filter which has the condition of ! = null on each column in the file (it was never a good design but my only resort until we could find a best solution).
Thanks again.
BR,
Darsh- darshthakkar3 years agoValued Contributor
@ddellsperger: Is it a safe assumption that the below solution provided by you is similar to the conventional filter I used (which was putting a NOT NULL condition on each column?) Q-1
I believe my conventional filter would be a lot of overhead on the system + this time, I only had to deal with 10 columns (so 20 conditions, one as !=null and another as !=“”) however if I had to work on 350+ columns (which is my ideal day scenario), conventional filter would have busted.
I want to take this opportunity to learn from an expert like you that what would be the overhead (time & space complexity) while using $.entries().filter(v => v[1] != “”).length > 0? Q-2
Is the above a recommended approach and the ONLY approach to filter out blank rows from a flat file? Q-3
Looking forward to hearing from you.
BR,
Darsh
- pmancevski3 years agoNew Contributor III
- darshthakkar3 years agoValued Contributor
Thank you @pmancevski for your suggestion and assistance on this one. I tested with your solution and unfortunately it didn’t work.
- darshthakkar3 years agoValued Contributor
Thank you @ddellsperger, I will definitely give this a try.
Before posting this to community, I tried with$.filter((v,k)=> v != null)
but didn’t get any success.Will keep you posted with my findings. Thanks again.
Regards,
Darsh
- smudassir3 years agoEmployee
Will JSON.stringify($) != “{}” works? I feel this is simpler.
- darshthakkar3 years agoValued Contributor
@smudassir: I haven’t tried this but can surely give a try and keep you posted. Should the input stream be in any specific format?
BR,
Darsh- smudassir3 years agoEmployee
Just try on the data you had earlier.