Forum Discussion

hrichtermn's avatar
hrichtermn
New Contributor
6 years ago

Oracle dimension tables to Azure ADLS

We’re starting a project to move dozens of Oracle warehouse dimension tables to ADLS, and are using parent/child pipelines to pass table names. The pipelines correctly pass and resolve the table na...
del's avatar
del
Contributor III
9 years ago
Copy LinkReport

@matt.bostrom, The simplest way that I know of is to use a Script snap. I’m not a Python expert (so someone can improve on this), but code in the execute method would look something like this:

def execute(self):
    tickets = []
    out = java.util.HashMap()
    while self.input.hasNext():
        try:
            in_doc = self.input.next()
            tickets.append(in_doc)
        except Exception as e:
            errWrapper = {
                'errMsg' : str(e.args)
            }
            self.log.error("Error in python script")
            self.error.write(errWrapper)
    out["tickets"] = tickets
    self.output.write(out)

I’ve run into similar needs before on a couple of occasions, so I have written a custom snap to achieve this same result. I can share the java code with you as well if you want to develop your own snap.

Related Content