cancel
Showing results for 
Search instead for 
Did you mean: 
Ophir_Buchman
Sisense Team Member
Sisense Team Member

Introduction

Customers often run into the question of what data model they should use (an Elasticube or a Live Model). The following article presents a few of the aspects you should consider when choosing between them.

Sisense recommends you discuss your needs and requirements during the Jumpstart process to best meet your business expectations.

Table of Contents

Definitions

The Elasticube Data Model

Importing data into an "ElastiCube data model" allows the customer to pull data from multiple data sources on-demand or at a scheduled timeframe and create a single source of truth inside Sisense. The imported data can then be transformed and aggregated to meet your business needs. Once imported, the data snapshot is used to generate analytical information.

To read more about Elasticubes, refer to the following link

The Live Model

Using a "Live data model" does not require importing data, but instead, just defining its schema. Once configured, analytical information required by the user is directly queried against the backend data source.

To read more about Live Models, refer to the following link

Determining Factors

Refresh Rate

One of the most fundamental aspects of determining your data model is your data's refresh rate.
The data refresh rate refers to the age of the data in your dashboards:

  • On Live Models, the data displayed on your dashboards is near-realtime as every query is passed directly to the backend database. A good example of using a live model (due to refresh rate requirements) is a dashboard showing stock prices.
  • On Elasticubes, the data displayed on your dashboard is current to the last successful build event. Every query is passed to the local database for execution. A good example of using an Elasticube (due to refresh rate requirements) is a dashboard showing historical stock prices. In this case, a daily ETL process will provide good enough results.

To make a choice based on this factor - Answer the following questions:

  • How frequently do I need to pull new data from the database?
  • Do all my widgets require the same data refresh frequency?
  • How long does an entire ETL process take?

Data Transformation Options

The ETL process includes a "Transformation" phase. This transformation phase usually includes:

  • Migrating the data tables into a dim-fact schema
  • Enriching your data
  • Preagregating the data to meet your business needs

The amount of data transformation on Sisense will help determine the suitable data model:

  • On Live Models, Sisense allows minimal to no data transformation. Data isn't imported before a query is issued from the front-end, and therefore data cannot be pre-conditioned or pre-aggregated. Most data sources used by live models are data warehouses which may perform all data preparations themselves.
  • On Elasticubes, data is imported before a query is issued from the front-end and therefore may be pre-conditioned and pre-aggregated. A user may customize his data model to answer his business questions optimally.

To make a choice based on this factor - Answer the following questions:

  • Is my data in a fact-dim schema?
  • Does my data require enriching or pre-conditioning?
  • Does my data be pre-aggregated?

Operational Database(s) Load

Your operational databases do more than serve your analytical system. Any application loading the operational databases should be closely examined:

  • On Live Models, Sisense will constantly query information from your operational databases and feed it into your dashboard widgets - This takes place every time a user loads a dashboard.
  • On Elasticubes, Sisense highly stresses your operational databases during an ETL process while reading all tables.

To make a choice based on this factor - Answer the following questions:

  • Does the analytical system stress my operational database(s)?
  • Can the query load be avoided by using a "database replica"?

Operational Database(s) Availability

Your operational database(s) availability is critical for collecting information for your analytical system.

  • On Live Models, all queries are redirected to your backend database. If that database is not available - Widgets will generate errors and won't present data.
  • On Elasticubes, database availability is critical during the ETL process. If that database is not available - The data on your widgets will always be available, however, data won't necessarily be up to date.

To make a choice based on this factor - Answer the following questions:

  • How frequently are analytical database(s) offline?
  • How critical is my analytical system? Is being offline/showing out-of-date information acceptable?

Additional Vendor Costs

Various database vendors use a chargeback charging model.
Meaning, you'll be charged by the amount of data you pull out of the database or the compute power required for processing your data. 

  • On Live Models, every time a user loads a dashboard, each widget will trigger (at least) one database query. A combination of a chargeback charging model and a large user load may result in high costs.
  • On Elasticubes, every time the user triggers an ETL process, a large amount of data is queried from the database and loaded into Sisense.

To make a choice based on this factor - Answer the following questions:

  • What is the amount of users using my dashboards / What is my "build" frequency?
  • What data model will result in fewer costs? What is the tipping point?
  • Are you willing to pay more for real-time data?

Database Size

Sisense is not a data warehouse - It has a limitation on the amount of user data it may store:

  • On Elasticubes, the rule-of-thumb limitation is 300M rows per Elasticube (based on an average amount of columns and data types).
  • On Live Models, there is no limitation as data is not imported to Sisense but just the data's schema.

To make a choice based on this factor - Answer the following questions:

  • What is the amount of data I need in my data model?
  • What is the amount of history I need to store?
  • Can I reduce the amount of data (e.g., trimming historical data? reducing columns? etc.)

Query Performance

Query Performance depends on the underlying work required for fetching data and processing it. Although every widget generates a query, the underlying data model will determine the work necessary for executing it.

  • On Elasticubesevery query is handled inside Sisense: The client-side widget sends a JAQL query to the Sisense analytical system. The query Is translated to SQL syntax and run against an internal database. The query result is transformed back to a JAQL syntax and returned to the client-side.
  • On Live Models, every query is forwarded to an external database and then processed internally: The client-side widget sends a JAQL query to the Sisense analytical system. The query Is translated to SQL syntax and run against an external database. Sisense waits for the query to execute. Once returned - The query result is transformed back to a JAQL syntax and returned to the client-side.

To make a choice based on this factor - Answer the following questions:

  • How sensitive is the client to a delay in the query's result?
  • When showing real-time data - Is this extra latency acceptable?

Connector Availability

Sisense supports more than 160 data connectors (Link). However, not all connectors are available for live data models. The reasoning behind this has to do with the connector's performance. 

A "slow connector" or one that requires a significant amount of processing may lead to a bad user experience when using Live models (i.e., widgets take a long time to load):

  • On ElasticubesSisense allows the user to utilize all 160 data connectors.
  • On Live Models, Sisense limits the number of data connectors to a few high-performing ones (these include most data warehouses and high performing databases).

To make a choice based on this factor - Answer the following questions:

  • Does my data source's connector support both data model types?
  • Should I consider moving my data to a different data source to allow live connectivity?

Caching Optimization

Sisense optimizes performance by caching query results. In other words, query results are stored in memory for easier retrieval in case they are re-executed. This ability provides a great benefit and increases end-user experience:

  • On ElasticubesSisense recycles (caches) query results.
  • On Live Models, Sisense performs minimal caching to make sure data is near real-time (note caching can be turned off upon request).

To make a choice based on this factor - Answer the following questions:

  • Do I want to leverage Sienese's query caching?
  • How long do I want to cache data?

Dashboard Design Limitations

Specific formulas (such as Mode and Standard Deviation) and widget types (such as Box plots, Whisker plots) may result in "heavy" database queries:

  • On Live Models, Sisense limits the use of these functions and visualizations as the results of these formulas and visualizations may take long - Causing a bad user experience.
  • On ElasticubesSisense allows the user to use them as processing them is internal to Sisense.

To make a choice based on this factor - Answer the following questions:

  • Do I need these functions and visualizations?
  • Can I pre-aggregate the data and move these calculations to the data source instead of Sisense?
Version history
Last update:
‎01-19-2022 05:35 AM
Updated by:
Contributors
Community Toolbox

Recommended quick links to assist you in optimizing your community experience:

Need additional support?:

Community Support Request