Data Integration
Discuss data integration use cases and solutions from ETL, ELT, reverse ETL, and AutoSync.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

HDFS Reader - Reading a partition with space

I am using HDFS Reader snap to read the data from Hadoop for the ETL. I am unable to read few partitions which has space character. Same partitions, i am able list using hadoop fs -ls command using escape character before the space(\ ). I have tried ...

Updating Databricks/Delta Lake tables in Standard Pipelines

Hi community, I was wondering if anyone has tried doing Inserts/Updates to Databricks/Delta Lake Table using the generic JDBC Snaps (or any other work around if you have it). Using the latest Databricks JDBC Driver, I was able to setup an account suc...

fkhan060 by New Contributor II
  • 1 replies
  • 0 kudos

Duplicate data using SQL offset

We are attempting to load transactional tables on a pipeline we created Issue faced - While loading data from a transactional table by leveraging the help of limit and offset in SQL SERVER select snap (partition and loading), we are getting duplica...

The Data Lake will get Cloudy in 2017

Written by Chief Enterprise Architect, Ravi Dharnikota. Originally posted in SandHIll Magazine. In 2016, we saw a lot of interesting discussions around the data lake. In the spirit of reflecting on the past and looking at the future of this technolog...

KERBEROS issues - HDFS cluster

Hi , We’ve implemented Kerberos in our hadoop cluster, but we are having issues with our pipelines. We are getting this error "Reason: SIMPLE authentication is not enabled. Available:[TOKEN, KERBER...

Petrica by New Contributor II
  • 3 replies
  • 0 kudos

Parquet without Hadooplex

Has anyone come up with a “simple” process to convert files to Parquet format without the need to use a Hadooplex and without going through HDFS. I would like to read a text file from S3 and convert it to Parquet back onto S3.

gomezn by New Contributor
  • 2 replies
  • 0 kudos