Loading ElastiCube from s3
We will have our data files, one per day residing in S3. We would like to load these files into Sisense Elasticube. From the Sisense website, it appears that there should be an S3 connector, but the ...
Forum Discussion
Hi bpeikes, we recommend you use the Generic JDBC connector plus the CData driver to visualize Amazon S3 data in Sisense.
Here are some set up documents to refer to:
- Connecting to Vendor JDBC Connectors - allows you to use vendor JDBC drivers
- CData Amazon S3 Download - CData driver download
- CData: Visualize Amazon S3 Data in Sisense - instructions on how to set up the CData driver in Sisense
- CData JDBC Driver for Amazon S3 - Full CData instructions on available connection string parameters
Another option is to use the Amazon Athena connector which is included as a connector in Sisense but does require Athena be set up to query your s3 buckets.
Hope that helps!
We’re considering Athena, as the other solution you posted seems much more complex.
Whats odd, is that on the website, there appears to be native S3 integration, but then the links go nowhere.
Hi bpeikes ,
I flagged your comment above with our Web Design team. They have gone through and fixed the link. Thank you for bringing it to our attention!
https://www.sisense.com/data-connectors/amazon-s3/
Best,
Sofia Losada
Couple of questions:
1) Is the CData Amazon S3 JDBC driver free?
2) What file formats does it work with?Hi bpeikes,
Any CData connector on this page can be used with Sisense for no additional charge - which includes the S3 driver mentioned earlier.
After setting this up myself (I focused on Elasticube not Live Model), it looks like the Amazon S3 driver will assist in navigating metadata only (such as getting a list of buckets).
With a connection string that looks something like this:
jdbc:amazons3:AWSAccessKey=XXXXXXXXXXXXX;AWSSecretKey=XXXXXXXXXXXXXXXXXXXXXXX;AWSRegion="OHIO";
From the docs:
In order to stream the underlying data (for instance pull csv data stored in a bucket), I used the corresponding files type driver - so CData JDBC Driver for CSV in this case.
The connection string will look something like this:
jdbc:csv:URI="s3://path/to/files";AWSAccessKey=XXXXXXXXXXXXX;AWSSecretKey=XXXXXXXXXXXXXXXXXXXXXXX;AWSRegion=ohio;
And it gave me access to all csv files that were in the bucket as individual tables.
Based on the documentation it looks like there are some more advanced schema options for manipulating the data as it is streamed into Sisense.
Hope that helps!
