Loading ElastiCube from s3
We will have our data files, one per day residing in S3. We would like to load these files into Sisense Elasticube. From the Sisense website, it appears that there should be an S3 connector, but the ...
Forum Discussion
We’re considering Athena, as the other solution you posted seems much more complex.
Whats odd, is that on the website, there appears to be native S3 integration, but then the links go nowhere.
Hi bpeikes ,
I flagged your comment above with our Web Design team. They have gone through and fixed the link. Thank you for bringing it to our attention!
https://www.sisense.com/data-connectors/amazon-s3/
Best,
Sofia Losada
Couple of questions:
1) Is the CData Amazon S3 JDBC driver free?
2) What file formats does it work with?Hi bpeikes,
Any CData connector on this page can be used with Sisense for no additional charge - which includes the S3 driver mentioned earlier.
After setting this up myself (I focused on Elasticube not Live Model), it looks like the Amazon S3 driver will assist in navigating metadata only (such as getting a list of buckets).
With a connection string that looks something like this:
jdbc:amazons3:AWSAccessKey=XXXXXXXXXXXXX;AWSSecretKey=XXXXXXXXXXXXXXXXXXXXXXX;AWSRegion="OHIO";
From the docs:
In order to stream the underlying data (for instance pull csv data stored in a bucket), I used the corresponding files type driver - so CData JDBC Driver for CSV in this case.
The connection string will look something like this:
jdbc:csv:URI="s3://path/to/files";AWSAccessKey=XXXXXXXXXXXXX;AWSSecretKey=XXXXXXXXXXXXXXXXXXXXXXX;AWSRegion=ohio;
And it gave me access to all csv files that were in the bucket as individual tables.
Based on the documentation it looks like there are some more advanced schema options for manipulating the data as it is streamed into Sisense.
Hope that helps!
