Forum Discussion

LanreOyewole's avatar
LanreOyewole
New Contributor II
2 years ago
Solved

Using the Filter Snap on nested array

I have a JSON payload with a nested array (below). I am trying to use the filter Snap to select specific items only. Unfortunately, the expressions that I have tried have not worked - they select everything or nothing.
$myList.filter(x => x.num==1)
$myList.filter(x => x.num==1)[0].num==1

[{"myList":[{"msg":"Hello, World1","num":1,"type":"Q"},{"msg":"Hello, World2","num":2,"type":"X"},{"msg":"Hello, World3","num":3,"type":"Q"}]}]

Any help or suggestions will be very much appreciated.

 

5 Replies

  • LanreOyewole - I think what you really want to do is put your expression in a Mapper and target back to the same field name:

     

  • So the Filter works at the document level - what you had was a single document with an array, which is why I went for the Mapper.  If this is confusing, you can share your pipeline which might let me explain it more clearly with an example.

    • LanreOyewole's avatar
      LanreOyewole
      New Contributor II

      Thanks koryknick, that is certainly what I have discovered - the docs don't mention this constraint though, which is why I spent so much time trying to get it to work 😀👍🏿

      I have also noticed that expressions don't work well when one or more of the ancestor nodes is an array 🤔

  • LanreOyewole's avatar
    LanreOyewole
    New Contributor II

    Hello koryknick,
    Thanks a lot for the response, much appreciated.
    I have already used the mapper, and that works fine. However, I am going through the training materials and I noticed that the filter Snap is used for de-duplication in one of the examples. The documentation also says "This Snap filters a document stream based on a expression", which made me try to use it to select specific documents.