Forum Discussion

dwhite's avatar
dwhite
Employee
8 years ago

Replace your two scripts with mappers should net you some performance gain. You don’t need to use a script to make a string of json.

Script in “Split events” pipeline which is doing

data = self.input.next()
newData = {}
newData[‘record’] = data
newData[‘sequence_id’] = data[‘sequence’][‘id’]
newData[‘event_id’] = data[‘event_id’]
self.output.write(newData)

replace with a mapper like

Script in write events is like
data = self.input.next()
self.output.write(data[‘record’])

replace with a mapper like

  • aditya_sharma's avatar
    aditya_sharma
    New Contributor III
    8 years ago

    Thanks Dwhite

    I’ll make that change, but I checked the average execution time for both the scripts , it’s very less (0.141 sec).
    Also, how should we decide what to set in pool size property of Pipeline Execute snap.

    Thanks