Sisense Elasticube vs Live Models with Snowflake

[Adriana_Onti]

As the modern data stack continues to gain momentum, there are certain patterns and technologies emerging as predominant elements of every stack. Snowflake and Sisense are two of those technologies. Because these two technologies show up side by side so often, it warrants further investigation into how these technologies work together. When we look under the hood, the primary distinction that immediately stands out is the data connectivity functionality offered by Sisense. 

While many business intelligence (BI) tools are compatible with Snowflake, what makes the Sisense/Snowflake relationship powerful is the ability to use either Sisense ElastiCubes or Live Models with any Snowflake instance. Allowing customers to use either ElastiCubes or Live Models allows for incredible flexibility for any use cases or SLAs that may exist. But, before we examine the benefits of these two approaches, let’s first investigate in more detail the differences between ElastiCubes and Live Models. 

 

What Is An ElastiCube?

An ElastiCube is a Sisense proprietary analytics database designed to handle the extensive workload required by BI tools. Because an ElastiCube is a standalone database, the source of the data being brought into the ElastiCube has little impact on the performance of the ElastiCube. In addition to this, users are not restricted to using a singular data source, such as just using Snowflake, but can add as many data sources to an ElastiCube as needed. 

Behind the scenes, the ElastiCube acts as the optimized intermediary between your raw data source(s) and the user interacting with Sisense. Because data is stored in an ElastiCube, separate from your raw data, there will likely be latency between the data stored in the ElastiCube and the data in your data warehouse or other sources. To account for this, users must decide a refresh schedule for each ElastiCube. During a refresh, an ElastiCube will reload each table from each data source included, and in doing so, will reflect any changes and updates to the data from the previous refresh. Any dashboards and widgets associated with ElastiCubes are subjected to the same refresh schedule in order to show net new data. 

What Are Sisense Live Models?

Sisense Live Models are exactly what they sound like: data models that are connected directly to your live data source. In this case, the data source would be Snowflake. Any time an interaction occurs with a Live Model, a query is run directly against the data source that it is connected to. Live Models are limited to a singular data source, but they allow for real time data to be displayed in your dashboard. 

Finally, because Live Models are direct connections to the underlying data source, the performance of each live query is dependent upon the optimization of the data source.

Now that we have a basic understanding of what ElastiCube and Live Models are, let’s quickly peek under the hood of Snowflake so we know how Sisense connects to it.

 

How Do Snowflake and Sisense Connect?

When data is loaded into Snowflake, it is stored in an underlying database. To access or interact with that information, a cluster of compute resources is required. In Snowflake, each resource compute cluster is referred to as a virtual warehouse, or just simply, a warehouse. A warehouse is what Sisense will connect to as the engine to perform the necessary actions to support an ElastiCube or Live Model. Additionally, each warehouse will only be online if an active process is running against it, otherwise, the warehouse will turn itself off after a specified period of time until a new process initiates the warehouse to turn back on. Once Sisense knows what database and warehouse to connect to, it is time to decide whether to use an ElastiCube or Live Model. 

We’ve seen the differences between an ElastiCube and Live Model. We’ve also seen how Sisense interacts with Snowflake. Armed with this information, we can now evaluate the implications that each approach has on Snowflake.

The Advantages of ElastiCube

There are many advantages to using an ElastiCube. The forefront of which is the ability to bring in data from multiple sources. Because the ElastiCube is an optimized, analytical database, when multiple data sources are brought into it, each one is treated the same, highly optimizing each source to be analyzed in the context of any other. However, when leveraging Snowflake, it is likely that many, if not all data sources, have been centralized within Snowflake. This does not defeat the purpose of loading different data sources into an ElastiCube - rather, it allows for flexibility in testing and modeling new data. 

For example, let’s say there is a Google Sheet that contains the email addresses of all internal employees at a company. An analyst wants to see if any difference is made to the company’s total website sessions if they filter out internal emails from the web traffic data source. Instead of needing to ETL the data into Snowflake, create a new schema, model the data, and bring it into Sisense, the analyst could simply connect the Google Sheet to an ElastiCube and within a matter of minutes have an answer to their question. 

In addition to supporting multiple data sources, ElastiCubes are a great option if there aren’t stringent data latency SLAs required from your data. If a refresh every hour doesn’t impact your business, then having an ElastiCube on a scheduled refresh may make a lot of sense. The question may then arise, “why would I want any latency in my reporting?” The answer is in another major advantage of ElastiCubes: analysis speed. 

In many cases, reporting on events that happened moments prior is unnecessary. However, waiting tens of seconds or even minutes for dashboards to display data is even more unnecessary. The ElastiCube technology allows for hundreds of millions of data rows to return in seconds. Using the ElastiCube allows for lightning fast reporting, while not requiring you to have a large warehouse in place to support large query returns. This helps in controlling cost. 

ElastiCubes also help control costs from a query volume perspective as well. Because Snowflake charges for the amount of time a warehouse is online, when multiple queries are coming from Sisense due to usage, the warehouse never has time to turn off. This can rapidly inflate your Snowflake bill. By using an ElastiCube, Sisense only queries Snowflake every time it needs to refresh the tables in the ElastiCube, not every time there is a query in a dashboard. 

For all the good ElastiCubes provide users, there are some considerations to make before going all in. If data latency is not an option, or you have data SLAs that require updated data down to the minute, ElastiCubes are probably not for you. Additionally, depending on the size of the tables in Snowflake, ElastiCube refreshes can take time, or if you are using multiple ElastiCubes can require additional allotments of power. If you remember, in Snowflake, time is money so it may be wise to consider the amount of time refreshes take per day versus the amount of queries Sisense users are making per day. If refresh time is considerably more than user query time, refreshes may be costing you more money. If either of these scenarios are true, then Live Models may be the better option for you.  

 

The Advantages of Live Models

Live Models have one primary advantage over ElastiCubes: real time data. If the nature of your business requires real time data, or you have user SLAs that require real time data, Live Models can support this. However, as with ElastiCubes, there are several considerations to make before jumping completely onboard. 

Because Live Models are connected directly to Snowflake, they are reliant upon the computing resources provided by Snowflake, not the ElastiCube, for how well they perform. If a user wants real time data but they are connected to the smallest Snowflake warehouse size, query times are likely going to take much longer than those of an ElastiCube. 

Another important factor to consider when leveraging live models is the volume of queries sent through Sisense to Snowflake. Again, since Snowflake charges on the amount of time a warehouse is online, if there are a high volume of queries consistently coming from Sisense, it is easy to rack up a high Snowflake bill. ElastiCubes can help ease this issue, but only if the tables aren’t so large that it takes more time to refresh the ElastiCube tables than it would be to support all the Live Models queries throughout a day. 

Finally, an important consideration when using Live Models is data modeling. If the data in Snowflake has not been modeled for analytical use cases, using Live Models can be difficult. A common example of this would be preparing your tables as fact and dimension tables. If your data models are not fit for analytical use cases, an ElastiCube can help alleviate some of the rigidity that a Live Model requires. ElastiCubes provide more flexibility when it comes to post warehouse modeling and visualization preparation. 

In Summary

When looking at both ElastiCubes and Live Models, there are both advantages and disadvantages to both solutions. The best approach for choosing which solution to leverage is to understand your use case. If you only need fresh data every six hours, ElastiCubes are probably going to serve you just fine. If you need snappy response but also need to keep your Snowflake warehouse cost low, ElastiCubes are likely a great choice. If you need immediate access to the most up to date information, Live Models are a no brainer. Understanding your business use case and using technology to solve those specific problems is always better than guessing.

Sisense wants its customers to have the best experience possible to optimize their data, and control warehouse costs, so they have partnered with Untitled Firm to create an offering for Sisense customers who are on Snowflake. If you have any questions please contact Ellie Puckett, Strategic Partnerships Director - [email protected] and we can help you determine which is the best path for your business.