Analyze Data Faster with Google Cloud’s BigQuery Storage API

[GuyRaz]

We live in an era of Big Data. The sheer amount of data being generated is greater than ever (we hit 18 zettabytes in 2018) and will continue to grow. In addition, this data lives in so many places that it can be hard to derive meaningful insights from it all. This is where analytics and data platforms come in: these systems, especially cloud-native Sisense, pull in data from wherever it’s stored (Google BigQuery data warehouse, Snowflake, Redshift, etc.). Teams can then clean and transform the data, and perform different types of analyses on it, revealing deeper insights and allowing your company to make smarter decisions.

APIs are vital for connecting data within BI platforms

An increasing number of companies are offering cloud solutions alongside their core business product. Businesses, for their part, will continue to choose the cloud solution that best meets their needs, or go multi-cloud. It’s like the iOS vs. Android decision: buying an iPhone means using Apple hardware and you are essentially locked into the Apple ecosystem. With Android’s open-source operating system, you have a wider array of choices. Choosing a single BI provider does not lock you into a single cloud option (as we recently discussed). Whichever cloud you ultimately select, you’ll want a BI tool that can seamlessly connect to it.

This is where APIs, like Google Cloud’s BigQuery Storage API, come in. APIs are a vital tool in a data team’s toolbox for connecting information from disparate systems to a BI solution. They provide engineers and data scientists access to a rich array of functions to enhance how they interact with their organization’s data. Using APIs can also help teams get more out of their BI platforms by making queries and analytic apps more efficient.

Google BigQuery: Made for Big Data

Pairing a company’s Google Cloud data with a great analytics tool like Sisense enhances query experience improves their digital journey.

Sudhir Hasbe, Director, Product Management, Google Cloud

There are so many options today when choosing a cloud data warehouse. Customers who choose BigQuery often do so because it’s completely serverless and scales seamlessly to tackle a growing organization’s data needs. It’s a fully-managed solution (so you don’t need a dedicated database admin for it) that’s good for fluctuating workloads with ad-hoc querying of vast datasets (especially if you have large chunks of information you don’t need to query often). And the biggest benefit — it’s cost-effective.

If you’ve got data, chances are you want to do some machine learning to pull predictive insights out of it. A lot goes into making ML a reality for companies with complex data, but BigQuery ML simplifies the process, allowing data scientists to quickly create a prototype for their more advanced analytics use cases in record time. That makes BigQuery ML ideal for smaller data science teams with limited budgets, but big data needs.

The BigQuery Storage API provides teams with faster access to their managed storage via an RPC-based protocol, using multiple data streams in the same session to read disjointed rows from a table. This reduces data movement and increases efficiencies for teams.

“The release of the BigQuery Storage API reflects our ongoing commitment to enabling digital transformations as more companies continue to migrate data into the cloud,” said Sudhir Hasbe, Director, Product Management, Google Cloud. “Customers don’t conduct business in a vacuum — by pairing their Google Cloud offerings, like BigQuery, with a great business intelligence and analytics tool like Sisense, the enhanced query experience improves their digital journey. The success of the latest API release means shared customers of both platforms will enjoy a more integrated data ecosystem.” 

“In our labs, we’ve achieved a 10x increase in speed when using the Sisense BigQuery connector with the Google BigQuery Storage API enabled,” says Limor Fledel Vagman, a Sisense product manager. “When importing data from BigQuery to Sisense using the Elasticube, the Storage API’s parallelism helps speed up queries, even on large, complex datasets.”

Sisense’s Elastic Data Engine empowers deeper analysis, easily

Sisense rebuilt our architecture from the ground up to seamlessly connect to cloud data sources of all kinds and to power the next wave of analytic apps, embedded in software products.

Aviad Harell, Sisense Chief Operating Officer and Co-founder

After deciding to use BigQuery, it’s important to provide real-time insights through a native connection. Using Sisense’s live connector for BigQuery, teams can create powerful real-time queries and dashboards to support their teams looking to make smarter, up-to-the-minute data-driven decisions. In parallel, teams can choose to manage high-query, slow-moving data by easily caching it using Sisense’s proprietary performance acceleration technology. Plus, if you have data coming from additional sources outside BigQuery or are dealing with a multi-cloud setup, Sisense’s cloud-native architecture can be leveraged to seamlessly connect to these sources, creating a much more holistic snapshot of your data.

“Sisense rebuilt our architecture from the ground up to seamlessly connect to cloud data sources of all kinds and to power the next wave of analytic apps, embedded in software products. Connecting with BigQuery to power client analyses is another example of our commitment to being a cloud-first, cloud-agnostic platform and helping our users adapt to any situation.” said Aviad Harell, Chief Operating Officer and Co-founder of Sisense.  

Creating a single source of truth in our Elastic Data Engine also yields tremendous benefits. You can create one high-performance data model and use it continuously for analyses across your entire organization. Running all analyses through one platform can also dramatically improve adoption, simplify governance, and reduce management overhead.

Integrating technology to run faster queries 

As an official Google Cloud partner, Sisense is focused on helping builders of all kinds do more with their data in BigQuery. Our native BigQuery connector and cloud-native, cloud-agnostic architecture make dealing with your BigQuery deployment easy and efficient. Using Sisense enhances the performance benefits of a service like BigQuery, while providing advanced capabilities to both prep and manage data. 

Ultimately, being able to derive greater insights from your data allows you to more effectively drill down into the trends and outcomes that will help drive change in your business. Google BigQuery and Sisense, with the power of Google’s new BigQuery Storage API, are a powerful combination to handle even your most challenging data needs. 

Ready to start using BigQuery and Sisense yourself? Head to the Google BigQuery documentation page to learn more!