Forum Discussion

darshthakkar's avatar
darshthakkar
Valued Contributor
4 years ago
Solved

Fetch data whose date is less than 10hrs from the existing date!

Hi Team, I’m trying to achieve a filter condition but haven’t found any luck so far. My data has a field as Last_Updated which has date stored in the format 2022-06-23 03:54:45 I want to cons...
expression_language
pattern
sample_pipeline
transform
  • bojanvelevski's avatar
    bojanvelevski
    4 years ago

    Than I go back to my original statement. This expression will do the job:

    Date.parse($Last_Updated) > Date.now().minusHours(48)

bojanvelevski's avatar
bojanvelevski
Valued Contributor
4 years ago

In order to exclude duplicates, you’ll have to use a Diff mechanism. Practically, when get records that are updated in the last 48 hours, you need to compare them with Salesforce records added in the last 48 hours by using a Diff snap.

darshthakkar's avatar
darshthakkar
Valued Contributor
4 years ago

Thank you @bojanvelevski, cheked Diff snap, wasn’t aware of this up until now. It suggests that the input streams should be sorted, I will have to use a sorter as my entire pipeline was unsorted.

  • bojanvelevski's avatar
    bojanvelevski
    Valued Contributor
    4 years ago

    Yes @darshthakkar, you’ll have to sort the streaming data on the same field you’ll differentiate on.