Hi Everyone,
I have my data in S3 buckets and this data refreshes once in a day. So, For me Live connection datamodel under Sisense doesn't make sense because unnecessarily for every dashboard load-- query will run (either in Athena or Redshift) which will cost us more.
So, I went ahead with below approach:
S3 Data ---> Elastic Cube in Sisense on top of S3 data using Athena --->Build of this Elastic Cube is scheduled for once in a day---> Dashboards connected to this Elastic Cube ---> Dashboards run from data stored in FSX storage (which is faster as well as cost effective)
With above approach, I see Elastic Cube build is successful, if I have less data (approx. 1 M records ~ 2GB).
The moment data is more, I start getting build failures with below mentioned reason:
The build failed due to an exception encountered during the task "Build Table sales". The specific error message indicates that the base table "sales" was not completed as expected.
The failure occurred during the reading of row 579744, with the underlying cause being an Amazon Athena ETL session failure. Additionally, there was an out-of-memory error (GC overhead limit exceeded) during the data writing process for the session ID 'tq_lOlCozh6N'.
Mainly the failure is happening during writing process.
I am attaching the logs also. Any help will be highly appreciated.
