How to increment the counter (variable) based on the incoming data in snaplogic

[srikotla]

How to increment the counter (variable) based on the incoming data in snaplogic.

Source Data:

CostCenter, TestCompany
Test_Cost_Center_001,Test_company_1
Test_Cost_Center_001,Test_company_2
Test_Cost_Center_002,Test_company_1
Test_Cost_Center_002,Test_company_2
Test_Cost_Center_002,Test_company_3
Test_Cost_Center_002,Test_company_4

Target Data should be:

CostCenter, TestCompany, ID
Test_Cost_Center_001,Test_company_1, 1
Test_Cost_Center_001,Test_company_2, 2
Test_Cost_Center_002,Test_company_1, 1
Test_Cost_Center_002,Test_company_2, 2
Test_Cost_Center_002,Test_company_3, 3
Test_Cost_Center_002,Test_company_4, 4

Experts over here, please help in achieving the above solution using the right snap.

[srikotla]

Thank you So much sarathmattam,

I’m able to achieve this using mapper in the following way.

1. Read the data in sorted order

2. Generate the sequence

3. Copy the data into two outputs

4. For one o/p, Use Aggregate to find the min sequence grouping on the cost_center

5. From other o/p join the data using the cost_center

6. Use Mapper, and in one of the port define the logic as : (Value - min_Seq)+1

   Here Value is the sequence generated from the sequence snap

   Min_Seq is the minimum sequence value of the respective group obtained from aggregate snap.

[sarathmattam]

Thanks for sharing your solution 🙂

[Pretpark]

Even though this is an old question, i will answer it. I encountered this problem myself, and could only solve it using a script snap. The script i used is regardless of input data (depending on the layer of where you want to count).

The script snap was set to use Python, for which this is the code:

# Import the interface required by the Script snap.
from com.snaplogic.scripting.language import ScriptHook

class TransformScript(ScriptHook):
    def __init__(self, input, output, error, log):
        self.input = input
        self.output = output
        self.error = error
        self.log = log

        self.all_indocs = []

    # The "execute()" method is called once when the pipeline is started
    # and allowed to process its inputs or just send data to its outputs.
    def execute(self):
        self.log.info("Executing Transform script")
        counter = 0
        while self.input.hasNext():
            try:
                # Read the next input document, store it in a new dictionary, and write this as an output document.
                inDoc = self.input.next()

                if 'counter' not in inDoc:
                    inDoc['counter'] = counter
                # inDoc['downOneLevel']['downAnother'] = counter <-- use this to set an existing field to a new value, and comment the 2 lines above

                self.all_indocs.append(inDoc)
                
                counter += 1

            except Exception as e:
                errDoc = {
                    'error' : str(e)
                }
                self.log.error("Error in python script")
                self.error.write(errDoc)

        outDoc = {
            'output_group': self.all_indocs
        }

        self.output.write(outDoc)

        self.log.info("Script executed")

    # The "cleanup()" method is called after the snap has exited the execute() method
    def cleanup(self):
        self.log.info("Cleaning up")

# The Script Snap will look for a ScriptHook object in the "hook"
# variable.  The snap will then call the hook's "execute" method.
hook = TransformScript(input, output, error, log)

 I placed a comment in line 25, which you can un-comment in the case you have an existing JSON key, which needs a new value.