Forum Discussion

venkat475's avatar
venkat475
New Contributor II
2 years ago

reading fixed width cobol file in snaplogic using python script

hi Team,

i am currently using this copybook layout in datastage and reading the content in the file.now i am trying to read this content using snaplogic script snap using python script.but i am not getting the expected output as datastage process.so could you please help me to get this done.

i am attaching the existing cobol file copybook layout which i am using in datastage and the python script which i am using for snaplogic process.

attachments added

1.python code_1

2.python code_2

3.cobol layout file

4.output for the datastage and python script in output.txt file

5.sample data input data

3 Replies

  • Max - The simplest way that I can think to accomplish what you're asking for is to use a Gate snap with the default settings and rename the input view to "exchangeRates" as below:

    A word of caution on the Gate snap with the default settings - this snap consumes all input into a single document, which may consume significant memory resources if processing a large volume of data.

    Hope this helps!

     

    • Max's avatar
      Max
      New Contributor II

      koryknick- The gate snap worked in this use case. However, it is concerning that it may have resource issues if processing large data. The data set in this case is around 35K records. But we have a lot of cases this logic will need to be used and our datasets are huge. Is there a better way/more efficient way of doing this with big result sets? 

  • It depends on the use case.  You can use the Structure or Mapper snaps to manipulate individual input JSON documents, or use Group By Fields or Gate snaps to accumulate JSON documents into arrays.

    SnapLogic is well suited to work with large data sets, you just need to think about how it is being processed and whether or not you need to accumulate everything into large, nested JSON documents for your endpoint.