Sisense Community

Community Feedback
Chapters
Events
Forums
Blogs
KB Docs
Support
Learning
Use Case Gallery

Categories

Infused Analytics

Sisense Intelligence & AI
Exploring RAG: A Sisense-Based Approach with BigQuery and Gemini

Exploring RAG: A Sisense-Based Approach with BigQuery and Gemini Continuing from our previous article, Automated Machine Learning with Sisense Fusion: A Practical Guide , where we discussed how Sisense and AutoML simplify complex machine learning workflows, I am now excited to introduce an advanced feature: a chatbot interface powered by Retrieval-Augmented Generation (RAG) and a large language model (LLM) like Gemini. This enhancement allows users to interact with their data in natural language through Sisense's Native Blox widget . Imagine asking questions like "How many products did I sell per category?" and receiving insightful answers instantly. This not only improves data accessibility but also bridges the gap between technical data repositories and business decision-making. In this article, I’ll cover the first step of enabling this functionality: setting up a seamless integration pipeline between Sisense and Google BigQuery, embedding table and column metadata, and preparing the data for efficient querying. Important Note: Experimental Project : This implementation is an experimental project and not an official Sisense feature. It demonstrates how Sisense’s functionalities can be utilized to build a custom Retrieval-Augmented Generation (RAG) pipeline or how you can use this example by importing the provided notebooks. Google Ecosystem : The example provided is specific to the Google ecosystem, utilizing services like BigQuery (BQ) and Gemini for query generation and processing. Associated Costs : Please note that each prompt sent to the chatbot incurs a cost, which will be billed to your Google Cloud Platform (GCP) account. This includes charges for LLM processing and database queries. Building the Foundation: Data Preparation and Embeddings To enable natural language interactions with your data, we first need to establish a robust data infrastructure. This includes creating a vector store for embeddings in Google BigQuery (BQ) and preparing metadata about tables and columns. The process is fully automated using a pre-built Sisense custom code notebook , which simplifies embedding generation and management. Dataset in BigQuery The journey begins with your dataset in Google BigQuery. The notebook retrieves metadata such as table and column names, descriptions, and schema information from BigQuery’s Information_Schema . If any descriptions are missing, the system automatically generates them, ensuring comprehensive metadata coverage. Setting Up the Vector Store in BigQuery The vector store acts as the backbone for similarity searches within the RAG system. Using the custom notebook, the system: Creates the vector store : A structured repository to store embeddings for tables and columns. Organizes metadata : Ensures all table and column descriptions are structured and accessible. Generating Table and Column Descriptions Missing descriptions can hinder data understanding. The notebook includes a Description Agent that automatically generates meaningful text for: Tables : Contextual descriptions based on schema and usage. Columns : Descriptions highlighting their role within the dataset. These enhancements ensure the metadata is both informative and ready for embedding generation. Creating Embeddings with Vertex AI To enable semantic search, metadata descriptions are converted into numerical embeddings using Vertex AI’s TextEmbeddingModel . This is facilitated by the EmbedderAgent , which: Accepts strings or lists of strings (e.g., column descriptions). Generates embeddings through Vertex AI. Handles both single and batch processing for efficiency. Efficient Embedding with Chunked Processing For large datasets, embeddings are generated in chunks using the get_embedding_chunked function. This ensures: Scalability : Handles datasets of all sizes without performance issues. Parallel Processing : Processes text chunks simultaneously to speed up the workflow. Structured Outputs : Embeddings are returned in a structured DataFrame for storage or analysis. Storing Embeddings in the Vector Store The final step is storing these embeddings in the BigQuery vector store. This ensures that: Similarity searches are fast and efficient. Metadata is always accessible for chatbot interactions. ALT text: A screenshot of a data table displaying various columns such as "table_schema," "column_name," "data_type," "source_type," and several other attributes. The table shows sample values and metadata related to a database structure, organized in rows and columns. How the Chatbot Interface Works Now that the foundation for embeddings and metadata storage is set, let’s explore the chatbot interface in action. Imagine opening the chatbot in the Blox widget and asking a question about your dataset. Within moments, the chatbot responds in natural language, providing actionable insights. But what exactly happens under the hood to generate this seamless interaction? RAG Notebook and the Chatbot Workflow The chatbot operates using a pre-built RAG custom code transformation notebook , which orchestrates the end-to-end process. With this notebook, the entire pipeline—from understanding the query to generating the response—is automated. The notebook uses multiple specialized agents , each responsible for a specific task, ensuring precision and efficiency at every step. SQL Query Builder Agent BuildSQLAgent This agent specializes in constructing SQL queries for BigQuery. It uses the LLM to analyze the user’s natural language query and matches it with table schemas and column details from the vector store. It outputs a fully formed SQL query tailored to the user’s dataset and question. SQL Validation Agent ValidateSQLAgent The ValidateSQLAgent validates the SQL query before execution using a Large Language Model (LLM). Validation ensures the query adheres to essential rules, including: The presence of all referenced columns and tables. Proper table relationships and join conditions based on the schema. Formatting and compliance with BigQuery-specific SQL standards. Validation occurs during the debugging process, specifically within the DebugSQLAgent , to identify potential errors before attempting a dry run or execution . It provides a detailed JSON response: If valid, the process moves to the next step (dry run or execution). If invalid, the DebugSQLAgent uses the error details to refine the query iteratively. SQL Debugging Loop Agent DebugSQLAgent This agent runs the debugging loop to refine queries that fail validation or execution. The process includes: Validation : The query is passed to ValidateSQLAgent to check syntax, schema compliance, and structure. If valid, the query is ready for execution. Dry Run : If validation passes, the query is tested using a dry run via the test_sql_plan_execution function to confirm execution readiness. Execution : Once validation and dry runs succeed, the final query is executed using the retrieve_df function, which returns results as a DataFrame. Iterative Refinement : If the query fails either validation or the dry run, the DebugSQLAgent uses the LLM to troubleshoot and generate an alternative query. The loop repeats until a valid query is generated or the maximum debugging rounds are reached. This agent ensures the final query is: Correctly structured and semantically valid. Optimized for performance and aligns with the original user intent. Response Agent ResponseAgent This agent translates the SQL query results into natural language. It bridges the gap between technical SQL outputs and user-friendly communication. By combining the query results with the user’s original question, it crafts a clear and relevant response. How the Workflow Executes Here’s the step-by-step process for generating a response: User Query Embedding The EmbedderAgent converts the user’s natural language question into a numerical embedding. Using BigQuery native vector search , the system retrieves similar embeddings from the vector store created in the first phase. Schema and Content Retrieval Based on the retrieved embeddings, the system fetches relevant table and column schema details from the vector store. SQL Query Generation The BuildSQLAgent uses the retrieved schema details to construct an SQL query that aligns with the user’s question. SQL Validation and Execution The ValidateSQLAgent checks the generated SQL for accuracy and potential errors. If the SQL passes validation, it is executed against the BigQuery database. Debugging (if needed) If the query fails or generates an error, the DebugSQLAgent refines it iteratively until a valid query is produced. Response Generation The ResponseAgent uses the query results and the user’s original prompt to generate a natural language response. If the system fails to generate a valid response, it communicates the issue to the user. Conclusion By combining the foundational embedding process with this RAG-powered workflow, the chatbot transforms how users interact with their data. From seamless SQL query generation to delivering natural language responses, the system exemplifies the power of Sisense Fusion and advanced AI tools to simplify data-driven decision-making. As always, please reach out to your Customer Success Manager (CSM) if you would like to implement this in your own environment.
Himanshu NegiPosted 1 year ago
0
Widget & Dashboard Scripts
Limiting Date Range Filters in Sisense Dashboards

Limiting Date Range Filters in Sisense Dashboards Use Case Overview Wide date ranges in Sisense dashboards can lead to performance issues, especially when using live models or querying large datasets. For live data models, large queries increase costs as more data is pulled from the data warehouse. For Elasticubes, this can cause performance bottlenecks. To avoid these issues, it’s essential to limit the date range users can select, ensuring both cost-efficiency and smooth performance. Solution To address this, we can use a dashboard-level script that automatically limits the date range. When users apply a date range filter, the script checks if the range exceeds a defined maximum (e.g., 30 days). If it does, the script adjusts the FROM date to be within the limit while keeping the TO date as the user selected. Key Insight: The “filterschanged” event is triggered before the query is sent to the backend. This means the query is only sent once, with the modified date range, avoiding redundant queries that could increase load or costs. Implementation Here’s the concise script for implementing this logic: dashboard.on('filterschanged', function(el,args){ //console.log(args); //**************** User Input **************** var datefilterTable = "dim_date"; var datefilterColumn = "date"; var allowedDateRangeInDays = 30; var warningMessage = `Your From Date is modified to accommodate for the allowed date range of ${allowedDateRangeInDays} days`; var displayWarningMessage = true; // Set to false to turn off alert //******************************************** if(args.items.$$items){ if((args.items.$$items).length > 0){ (args.items.$$items).forEach( (item) => { if(item.jaql.table == datefilterTable && item.jaql.column == datefilterColumn ){ var dateFilter = item; var fromDateStr = item.jaql.filter.from; var toDateStr = item.jaql.filter.to; const fromDateObj = new Date(fromDateStr); const toDateObj = new Date(toDateStr); // Calculate the difference in days const currentDateRangeInTime = toDateObj - fromDateObj; const currentDateRangeInDays = currentDateRangeInTime / (1000 * 3600 * 24); // Checking if the Current Date Range is more than the allowed range if(currentDateRangeInDays > (allowedDateRangeInDays+1)){ const newFromDateObj = new Date(toDateObj); newFromDateObj.setDate(toDateObj.getDate() - (allowedDateRangeInDays)); // Format the date back to "YYYY-MM-DD" const year = newFromDateObj.getFullYear(); const month = String(newFromDateObj.getMonth() + 1).padStart(2, '0'); const day = String(newFromDateObj.getDate()).padStart(2, '0'); const newFromDateString = `${year}-${month}-${day}`; dateFilter.jaql.filter.from = newFromDateString; // Display warning message and update filter if(item.jaql.filter.to != newFromDateString){ console.log(warningMessage); if(displayWarningMessage){window.alert(warningMessage)}; args.dashboard.filters.update(dateFilter, {refresh: true, save: true}); } } } }) } } }); Key User Parameters: datefilterTable : The table containing the date filter. datefilterColumn : The specific date column to watch. allowedDateRangeInDays : Maximum allowed date range (e.g., 30 for 30 days). warningMessage : The message displayed if the date range is modified. displayWarningMessage : Controls whether a pop-up alert is shown. Conclusion This simple script ensures that user-selected date ranges stay within defined limits, improving performance and reducing costs. It adjusts the FROM date automatically, while the TO date remains as chosen, and ensures that only one query is sent to the backend—saving resources and improving efficiency. Additional Resources: https://academy.sisense.com/master-class-advanced-dashboards-with-plug-ins-and-scripts https://docs.sisense.com/main/SisenseLinux/customizing-sisense-using-code.htm
Sisense UserPosted 1 year ago
0
News & Updates
Building Confidence (and BloX Widgets) with the Help of Sisense Trainers

UX Dashboard Design The first class was UX Dashboard Design, where we covered the dashboard creation flow. This high-level process has been beneficial in evaluating our current dashboards and has served as a reliable guide for potential improvements. Some of the most salient points include knowing your audience and laying out the story, given that our users range from C-Suite to technical analysts. Any new dashboard gets checked for contrast using WebAIM’s website ( https://webaim.org/resources/contrastchecker/ ), and when I’m able to go beyond the company color palette, https://color.adobe.com/create helps me choose the right colors for the message. For example, the homework for this class allowed me to create a dashboard for family spending. Since my audience was non-technical, there were lots of indicator widgets and gauges to make the story clear and the data pop. Using step 7, getting feedback , I’ve now incorporated metrics that help evaluate the effectiveness of our dashboards based on a user behavior dashboard that our administrator created. I look forward to using the “New Dashboard Introduction” document in the future when we make enhancements to existing dashboards or create a new dashboard. This template ensures our end users will get a clear understanding of the objective, the call to action, and the updating frequency of any new enhancements or dashboards. BloX for Beginners The second course was BloX for Beginners, where we received hands-on experience creating BloX widgets from the BloX templates. (Pro-tip: always use a template) . Our current dashboards use the “Jump-To” buttons, but that was the extent of my BloX knowledge coming into the class. Little did I know of the endless possibilities with BloX widgets! We started to scratch the surface during training, where we created buttons linked to filters. For this particular example, we made three separate date range buttons for “Last 90 Days”, “Last 30 Days”, and “Last 7 Days”. A code snippet is included for the “Last 90 Days” button. We also learned how to incorporate images and hyperlinks into our BloX widgets. One of the coolest applications we added was Slack integration. We were sending messages from our BloX widget directly to Slack! 🤯 The homework for this class was challenging and drove home the point that practice is essential . The assignment asked for an embedded image and a text box, plus a drop-down filter that linked a name with the picture and the text. I got 80% of the way there, and during the homework review in session 2, the instructor helped me fix the dropdown filter, as I was only showing one name. The feedback was eye-opening. I had over-complicated the code. Reflection The two interactive training sessions were a great experience. As a new user of Sisense, meeting instructors and community members made me feel connected and welcomed. 10/10 Would recommend it!!! I look forward to additional interactive training sessions. About the Author Sarah Chronquist I graduated from UW-Madison with a BA in Sociology and a minor in Concentration in Analysis & Research. Started career as a SAS consultant, auditing a reporting processes for subprime loans in accordance with Sarbanes-Oxley regulations. Moved on to healthcare marketing analytics for Healthgrades for 13 years. While at Healthgrades, I received my MBA from Cardinal Stritch University. Started at Sift Healthcare in June 2022 as a Senior Business Intelligence Analyst and began working with Sisense. Personally, I’m an avid reader, and I enjoy walks with the dogs. In the non-winter months, you can find me in the gardens or biking with the family. Company Spotlight Sift Healthcare is a data and analytics company that enables healthcare providers to leverage the full scope of their data to improve financial and operational outcomes. Sift’s Payments Intelligence Platform equips organizations with meaningful insights and ML-driven workflow optimizations that accelerate cash flow, reduce the cost to collect, and inform revenue strategy. Sift’s Payments Intelligence Solutions include: Rev/Track - Data visualization and insights tools, including denials and payments dashboards, payer trend tracking, and operational benchmarking. Rev/Collect - ML-driven Denials Management and Patient Payment Management workflow integrations Rev/Protect - ML-driven Denials Prevention workflow integrations
eS_CePosted 3 years ago • Last reply 1 year ago
2
News & Updates
Saving Lives with the Power of Data

Saving Lives with the Power of Data At ESO every presentation, every conference, starts with our mission statement. It's only proper we do that here, “Improving Community Health and Safety Through The Power of Data” Our mission isn't merely a statement, it's our guiding principle. At ESO, we collaborate with first responders, including firefighters, paramedics, physicians, and nurses. Our collective passion unites us as we strive to assist others in their time of need. Recently, a customer shared with me the profound impact of our work, providing a heartening reminder. I learned about a customer who utilized a report I created on opioid use, leveraging Sisense as the foundation for our self-service reporting platform. This report played a crucial role in addressing opioid overdoses, emphasizing the urgent need for medical intervention, such as the administration of Narcan. While Narcan rapidly reverses the effects of an overdose, it is not a standalone treatment. With a quick drug half-life of 30-80 minutes, patients are at risk of a recurrence of overdose and respiratory depression. The challenge lies in patients feeling better after receiving Narcan, often underestimating the severity of their condition. Consequently, they may refuse transport to a hospital for further treatment, leading to a potential relapse into overdose, and respiratory failure. Therefore, it is imperative to track and follow up with these individuals. Our self-service reporting platform, powered by Sisense, enabled an EMS agency to generate a report tracking such refusals. By combining insights from both the opioid and refusal reports, they identified individuals at the highest risk. This empowered the agency to administer additional treatments, mitigating the risk of mortality from overdose. This exemplifies how the power of data can enhance community safety and combat the harmful effects of the opioid epidemic. Self-service analytics play a pivotal role, allowing customers to leverage quality data and adapt pre-built reports to address use cases previously unforeseen. Putting the power of data into everyone's hands enables better decision-making, ultimately influencing how we approach pre-hospital care. I feel privileged to contribute, in my own way, to helping those grappling with the opioid epidemic and providing our first responders with the data they need to save lives. Witnessing the tangible impact we can have on others is incredibly rewarding. To fellow data analysts, remember that the significance of your work can extend beyond what you might realize. Be proud of your contributions, and continue using data to improve our world. Ashton Woiwood Data Analyst @ ESO Let's Connect!
awoiwood1Posted 2 years ago
0
Widget & Dashboard Scripts
Color the Row and Value fields of a Pivot Table

This article will show you how to quickly color a pivot table into two sections: by Row and by Value 1. Build your pivot table. 2. In the widget editor, add the widget script below. Information on how to add a widget script here: https://sisense.dev/guides/js/extensions.html widget.transformPivot( { type: ['value','member'] }, function setCellBackground(metadata, cell) { if(metadata.type == 'value'){ cell.style = cell.style || {}; cell.style.backgroundColor = '#9A94BC'; } else { cell.style = cell.style || {}; cell.style.backgroundColor = '#ffe6e6'; } } ); 3. Change the colors by changing the color codes above and you're all set! Note: If you'd like to target different parts of the pivot table, refer to our Pivot 2 API documentation .
edselvPosted 4 years ago • Last reply 2 years ago
1
Sisense Administration
How to Create Client Secret for your Microsoft Teams Infusion App

This article is relevant for versions L2023.4 and later. When configuring your new Microsoft Teams Infusion App, you are required to generate a client secret for the bot used by the Infusion App. You are also required to pass that client secret to your Sisense instance to ensure your Infusion App can make requests to your Sisense instance, and vice versa. Once you have created a Teams App, and subsequently a Teams bot, here are step by step directions for creating a client secret: Click Tools in the left navigation Click Bot Management Click the Bot that you are using for your Infusion App. Click Client secrets , and then click the button Add a client secret for your bot Copy the newly generated client secret Copy the client secret into the Bot Password field on the Admin > Infusion Management page For additional information, please see Developer Portal for Teams . If you need additional assistance, please reach out to Sisense Support.
Sisense UserPosted 2 years ago
0
News & Updates
Upgrading to Sisense Cloud

Upgrading to Sisense Cloud Migrating from a self-hosted deployment to the Sisense Cloud offers numerous benefits. In this blog, you will learn about some of the advantages of this migration and how the Sisense Professional Services team can help to make the transition as smooth as possible. It is highly recommended that most customers migrate to the Cloud; It’s become the preferred method for deploying Sisense by new clients and it represents the future of our software. To date, we’ve brought 600 customers to our Sisense Cloud, and they have reported higher levels of customer satisfaction than customers who have deployed on their own infrastructure! Some of the key areas customers have experienced the biggest improvements are in the level of support they receive with faster time-to-resolution, an improved upgrade experience, and 24/7 monitoring for critical issues. If you’re not sure whether or not you’re ready to make the jump to the cloud, we’ll go through some key reasons below: Sisense Cloud Security & Compliance Our AWS cloud service was designed to provide peace of mind to our customers. Our industry-leading certificates include SOC 2 Type 2, and ISO 27001, and we also are a HIPAA-ready solution. You can have confidence that your data is safe on our cloud. Read more here . Faster Time to Market One of the most significant benefits of upgrading to a cloud-based platform is the faster time to market. Since you don’t have to worry about installation, you are able to develop your embedded solutions quickly and focus on getting analytics into the hands of your customers. The Sisense Cloud provides a robust and scalable infrastructure that can handle large amounts of data, connect to your data warehouse with live connections and allow you to focus on developing and delivering insights and value to your users. The Ability to Scale With the Sisense Cloud, your infrastructure is able to scale up with your needs, allowing you to avoid wasting time and resources. Our team has you covered when you experience spikes and increased demand, ensuring your services are always available to your customers. Receive Better and Faster Support After migrating to The Cloud, you benefit from better and faster support: we offer 24/7 coverage, ensuring that any issues that arise can be quickly resolved. There have been cases where we didn’t even have to bother our customers! This improved support is often cited as our customers’ favorite benefit after their migration. See SLA details here . Upgrades and Latest Features The Sisense Cloud comes with automatic upgrades, which provide access to the latest functionalities, bug fixes, and security enhancements. When migrating to the cloud, you benefit from these upgrades without needing to invest significant time and resources in your on-premise infrastructure. This means you can focus on delivering value to your customers instead of maintaining your infrastructure. Monitoring Our Sisense Cloud provides 24/7 monitoring of your Sisense environment for critical issues, providing you peace of mind that your end users won’t have to worry about long outages. Cost Savings One of the most overlooked aspects of moving to a cloud is the cost savings it will provide your organization– infrastructure, and labor costs tend to be greatly underestimated. Customers migrating to Sisense Cloud are always amazed by how much the total cost of ownership improves by utilizing cloud services. Your Customer Success Manager can help you calculate your estimated saving– reach out to them to learn more. The Migration Experience Our professional services team has extensive experience migrating On-Premise Windows deployment to our Cloud. We understand the associated challenges and are able to help you navigate the process smoothly. Our team will work with you throughout the assets migration, validation, and testing process to ensure that your environment is go-live ready. If you’re already running Linux and want to move into our cloud, the process is even more simple and can be completed within a matter of days. Next Steps Upgrading to the Sisense Cloud provides extensive benefits. If you are ready to upgrade, contact your Customer Success Manager today to find out more details and ask what our other customers say about the Total Cost of Ownership when they move to the Sisense Cloud!
MotiSapirPosted 3 years ago
0
News & Updates
Snowflake Architecture Patterns for Secure Self-Serve BI Experiences

Self-service business intelligence is a buzzword that gets thrown around a lot in analytics circles. While the idea of true self-service BI is glamorous, actually executing a data strategy to achieve self-service BI is a different story. Decisions such as which tools to use in the stack, how data should be modeled, and how data stays secure, make self-service BI more of a dream rather than a feasible reality. Thankfully, using Snowflake and Sisense, self-service BI is something all organizations can take advantage of by implementing specific Snowflake architecture patterns to provide secure, self-service BI experiences. Designated Databases The first step in achieving an architecture pattern that supports secure, self-service BI is by having multiple databases within your Snowflake account. To be more specific, at the very minimum, you should have two separate databases. The first database captures all raw data loaded into Snowflake. We’ll call this database raw . The second database stores data that has been cleaned, modeled, and is analytics-ready. Let’s call this database analytics . While there is a multitude of reasons why this architecture is advantageous for self-service BI, we’re going to focus specifically on two: data integrity and data security. Having the architecture outlined above, raw and analytics databases, enables you to have data immutability. By having a raw database, your ETL tools and processes are orchestrated to load data into this database. This is especially advantageous if you use an ELT, where you are extracting and loading the data, before transforming it. By doing this, you can have confidence that the data in your raw database is immutable. In other words, your data is unchanged and true to the source from which it came. This allows you to have a source of truth that has all of the data integrity intact. Using a tool like dbt , specific and different types of patterns that can be expected to be found in an analytics database can then be built on top of this raw, immutable data, without actually modifying the underlying data, meaning that for example, data mart and denormalized tables can be created with the goal of supporting specific analysis in your BI tool. This means that there is always an unchanged record of raw data that can always be referenced as a source of truth. The transformations and models built on top of the raw data can be materialized in the analytics database, providing a layer of separation from the raw data that shouldn’t be modified, and the data that is used in the analysis. By separating raw data from analytics (production) data, not only do we gain data integrity, but we can also provision each database so that only users with the correct roles and privileges can access raw data. Because analytics-ready data models are built on raw data and materialized in the analytics database, administrators in Snowflake can assign roles to BI tools and users, only allowing them to access necessary data. This provides an additional level of security as analytics data is not only provisioned in an entirely separate database from raw data, but it is controlled through security roles created and assigned by administrators. This architecture allows for raw data to be stored in a secure, provisioned data store while putting analytics-ready data in a database where only analytic use cases can occur. With this separation, we have the opportunity to optimize queries, creating also a secure and reliable database for your analytic use cases, all with the purpose of providing better performance and experience to the end-users. However, there are additional measures that can be added to these databases to create a more robust Snowflake architecture pattern. Multiple Warehouses With our raw and analytics databases in place, there needs to be at least one warehouse in Snowflake that can interact with those databases. At Untitled, we believe that one isn’t enough. In fact, the pattern we believe in the most is having three warehouses. These warehouses are outlined below: Loading Warehouse: This warehouse is designated for all processes that load data into Snowflake. It’s that simple. Do you use Fivetran? Connect it to this warehouse. Do you have a custom ETL pipeline? Connect it to this warehouse. Transformation Warehouse: This warehouse should be used for all matters related to getting data from its raw form into a usable state. No more, but no less. You should never be running ad hoc queries in this warehouse. Reporting Warehouse: This warehouse is solely put in place to serve all the querying needs of the organization. Sisense connects here. You analysts run queries here. You get the idea. There are no rules for how many warehouses you should have, but we believe it’s best practice to have at minimum these three. The reason we break these warehouses into their respective processes is twofold. First, some processes don’t need the same level of computing as others. By breaking warehouses into processes, we can control costs at a much more granular level as opposed to associating all processes with the same warehouse. Secondly, by separating these warehouses, we get a much better understanding of the workloads supported by each, allowing for optimization to take place. Again, if all processes are lumped into the same warehouse, it’s going to be much harder to understand if traffic is coming from an ETL tool, a dbt model running, or an ad hoc query run by an analyst. We inherently know where traffic is coming from if we only assign each process to a specific warehouse that is responsible for that process. Ideally, by implementing this architecture, users not only save costs, but have a positive BI experience. By having a deep understanding of patterns and trends in each warehouse, administrators can configure and optimize Snowflake to better serve each function. This means that BI users get tailored treatment as their work belongs to one of these processes. Roles and Privileges We have already briefly discussed this, but because it is so important, it is worth a second mention. All objects (databases, warehouses, schemas, tables, etc.) in Snowflake can be secured. While we could write an entire blog series on Snowflake access control, to quickly catch up, you can view their documentation here . When used in conjunction with the architecture discussed above, creating a role that only has privileges to only query data in the analytics database, using the reporting warehouse, allows for a control framework that allows for your self-service BI experiences to be completely secure as well as providing your BI tools and teams with data that is analytics-ready without ever compromising the integrity of your data. Summary Using multiple databases in conjunction with designated warehouses allows for dynamic architecture patterns to be developed to serve your self-serve BI needs. While this blog post was a brief peek inside how you can develop a robust self-service BI architecture, often there are additional complexities that arise when implementing self-service BI. If you need a partner to help overcome those hurdles, reach out to Untitled Firm . Ellie Puckett, Strategic Partnership Director, ellie.puckett@untitledfirm.com Sisense wants its customers to have the best experience possible to optimize their data, and control warehouse costs, so they have partnered with Untitled Firm to create an offering for Sisense customers who are on Snowflake. If you have any questions please contact Ellie Puckett, Strategic Partnerships Director - ellie.puckett@untitledfirm.com and we can help you determine which is the best path for your business.
Adriana_OntiPosted 4 years ago • Last reply 3 years ago
2
News & Updates
Webinar: Building Sisense Dashboards Accelerator

Webinar: Building Sisense Dashboards Accelerator Is your company generating and collecting massive volumes of data from various sources such as customer interactions, sales transactions, marketing campaigns, and operational processes? If so, you have probably experienced difficulty in effectively analyzing and deriving actionable insights from this data can be a significant hurdle. Well, you are in luck, DataBuilders is hosting a Sisense Accelerator Webinar; this webinar is aimed to teach users how to structure their data in a way that enables efficient analysis and visualization by taking you from zero to a fully functioning dashboard in 90 mins. Attendees will also acquire the expertise to create visually appealing and interactive dashboards, reports, and data visualizations that facilitate the identification of patterns, trends, and key insights. Our team will also cover topics like data governance, data security, and best practices for data-driven decision-making. This Sisense Accelerator webinar will provide attendees with the skills they need to solve the data analytics and insights generation challenges– leading them to make more informed business decisions . Learners will be able to identify growth opportunities, optimize their operations, understand customer behavior, and gain a competitive advantage in their industry. Ultimately, the customer can drive improved performance, increased efficiency, and better outcomes by leveraging the power of data and analytics through Sisense's platform. Webinar Details: Register HERE Webinar date and time: 30th May at 10 am (GMT) A Sisense accelerator is a live, virtual training module. It is an ideal start to learning Sisense and a great way to refresh on all that the Sisense Platform can do. Unlike a traditional webinar, the content structure of the course engages the participants through its public chat pane. Hosted on the Strigo training platform, our Sisense Accelerator demands nothing more of its audience than a computer with an internet connection and a willingness to learn. Who is this accelerator for? Whether you are a beginner or an experienced data analyst, the Sisense Accelerator is an opportunity to learn from industry experts, network with like-minded professionals, and gain hands-on experience with the best data analytics tool and techniques. If you are looking to take your data analytics skills to the next level, the Sisense Accelerator is the perfect event for you. It's for beginners and experienced analysts to learn from experts and network with peers while gaining hands-on experience with the latest tools. What Will You Need? A computer with an Internet connection, and a desire to learn. This workshop will take place on Strigo, a virtual training platform; you do not need Sisense or Strigo installed. We will provide A Sisense environment and a link to use Strigo in your browser for you during the workshop. Sisense has always been about integration & collaborative working and is essential for any organization that wants to make data-driven decisions and gain a competitive advantage. Let DataBuilders help you to optimize your operations, improve customer satisfaction, and increase revenue. Can’t wait to see you there!
cmcintyrePosted 3 years ago
0
Sisense Administration
Reverse Proxy with Nginx + SSL configuration

Reverse Proxy with Nginx + SSL configuration Nginx Reverse proxy configuration Step 1. Nginx reverse proxy server set up In this example, we are using nginx, we can install it on the same device as Sisense. To install it run 1. Install nginx for Ubuntu/Debian-like systems: sudo apt install nginx 2. For RHEL systems such a CentOS, use below: sudo yum install nginx 3. Start nginx: sudo systemctl start nginx Step 2. Nginx server configuration 1. Open the browser and go to the IP address of the server. If it's up, you will see the Nginx welcome page– this means nginx is now running on the default port 80. 2. Edit /etc/nginx/sites-enabled/default and add the next configuration under the root server config. Define correct Sisense public IP, and port in the " server {} " section: location /analytics { rewrite /analytics/(.*) /$1 break; proxy_pass http://<sisense-ip>:30845; proxy_http_version 1.1; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $host; proxy_connect_timeout 36000; proxy_send_timeout 36000; proxy_read_timeout 36000; send_timeout 36000; } 3. Before you apply the settings, check that there is no syntax issue by running sudo nginx -t 4. Reload nginx with sudo /etc/init.d/nginx reload or sudo systemctl reload nginx With this configuration, Sisense will be accessed with http://<ip-or-domain-of-nginx-server>/analytics. Also if the https is configured for this nginx server, Sisense would be accessible with https://<ip-or-domain-of-nginx-server>/analytics. If on the proxy level, the HTTPS is enabled, please ensure the application_dns_name has the https prefix to ensure all traffic is used, so something like: application_dns_name: https://company.sisense.com Step 3. Sisense configuration Go to the Admin tab Click on System Management Enter Configuration and choose Web Server In the Proxy URL enter "/analytics" or "http://<ip-or-domain-of-nginx-server>/analytics" as we configured in Nginx. With "/analytics" you will be able to use multiple domains for this instance. Save it and test with a browser by entering http://<ip-or-domain-of-nginx-server>/analytics And now we can configure SSL with our Nginx server, please validate that Nginx is working properly first before moving on. SSL configuration for Nginx Step 1. Obtain self signed SSL certificates You can use a command like this sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt. For an explanation of what the above command does please refer to Setup SSL on Sisense (Linux version) - Link placeholder Step 2. Configure Nginx to use SSL 1. Сreate a new file named self-signed.conf. sudo vi /etc/nginx/snippets/self-signed.conf In self-signed.conf we want to add some variables that will hold the location of our certificate and key files that we generated in Step 1. Like this ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt; ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; Save and close the file. 2. Now we will create a snippet file to define SSL settings. Start by creating a file like this sudo vi /etc/nginx/snippets/ssl-params.conf In this file, we need to include some SSL settings as below. ssl_protocols TLSv1.3; ssl_prefer_server_ciphers on; ssl_dhparam /etc/nginx/dhparam.pem; ssl_ciphers EECDH+AESGCM:EDH+AESGCM; ssl_ecdh_curve secp384r1; ssl_session_timeout 10m; ssl_session_cache shared:SSL:10m; ssl_session_tickets off; ssl_stapling on; ssl_stapling_verify on; resolver 8.8.8.8 8.8.4.4 valid=300s; resolver_timeout 5s; # Disable strict transport security for now. You can uncomment the following # line if you understand the implications. #add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"; add_header X-Frame-Options DENY; add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; Save and close the file. 3. In this step, we need to modify the Nginx configuration to use SSL. Open up your Nginx configuration file which is usually in a location like /etc/nginx/sites-available/<yourconfig>. Before making changes to this file it is best to back it up first in case we break anything. sudo cp /etc/nginx/sites-available/yourconfig /etc/nginx/sites-available/yourconfig.bak And now we open up our current Nginx config file; vi /etc/nginx/sites-available/<yourconfig> In the first server{} block, at the beginning, add the lines below. You might already have a location {} block so leave that there server { listen 443 ssl; listen [::]:443 ssl; include snippets/self-signed.conf; include snippets/ssl-params.conf; server_name your_domain.com www.your_domain.com ; //server_name can be anything location / { try_files $uri $uri/ =404; } } Lastly, we need to add another server{} block at the very bottom of the file, with the following parameters. This is a configuration that listens on port 80 and performs the redirect to HTTPS. server { listen 80; listen [::]:80; server_name default.local www.default.local ; //use same name return 302 https://$server_name$request_uri; } Please note that you must add this server_name to your local desktop or laptop hosts file. In this example, I will go to my local laptop or desktop hosts file and add <ip address of nginx server> <space> <default.local> [Optional] Step 3. Adjust the firewall The steps below assume you have a UFW firewall enabled. You need to review available profiles by running sudo ufw app list You can check the current setting by typing sudo ufw status: Output Status: active To Action From -- ------ ---- Nginx HTTP DENY Anywhere Nginx HTTP (v6) DENY Anywhere (v6 ) We need to allow HTTPS traffic, so update permissions for the “Nginx Full” profile. sudo ufw allow 'Nginx Full' Check the update sudo ufw status Output Status: active To Action From -- ------ ---- Nginx Full ALLOW Anywhere Nginx Full (v6) ALLOW Anywhere (v6) This output above confirms the changes made to your firewall were successful. So you are ready to enable the changes in Nginx. Step 4. Enable to changes in Nginx First, check that there are no syntax errors in the files. Run sudo nginx -t The output will most likely look like Output nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/ssl/certs/nginx-selfsigned.crt" nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful You can disregard the ssl_stapling warning, this particular setting generates a warning since your self-signed certificate can’t use SSL stapling. This is expected and your server can still encrypt connections correctly. If your output matches the out example above, that confirms your configuration file has no errors. If this is true, then you can safely restart Nginx to implement the changes: sudo systemctl restart nginx Step 5. Test the encryption Open up a browser and navigate to https://<server_name>, use the name you set up in Step 2C. Additional information 1. It was reported that File Manager and Grafana doesn't work with reverse proxy. To get the URLs for file manager and grafana to work, following steps should be taken: kubectl -n sisense set env deploy/filebrowser FILEBROWSER_BASEURL='/<baseurl>/app/explore' kubectl -n sisense set env deploy/filebrowser FB_BASEURL='/<baseurl>/app/explore/' kubectl -n sisense set env deploy/sisense-grafana GF_SERVER_ROOT_URL=<baseurl>/app/grafana 2. Once the reverse proxy is enabled, Sisense will still utilize IP addresses as links in their email communications. To setup correct addresses in Sisense e-mails after reverse proxy is configured: in the configuration yaml file set: update: true application_dns_name: "" and start the installation script to update parameters. After update is completed, in Sisense GUI go to Admin -> Server & Hardware -> System management -> Configuration Set the http://YOUR_PROXY_ADDRESS/analytics in the "Proxy URL" field of "Web Server" menu (or https://YOUR_PROXY_ADDRESS/analytics in case of SSL) Go to Admin -> User Management -> Users Try creating a new user or use the "Resend invitation" option for the existing one (if available) Check the inbox of that user for "Sisense account activation" The "Activate Account" link should now redirect to the http://YOUR_PROXY_ADDRESS/analytics/app/account/activate/HASH address
Vlad SolodkyiPosted 3 years ago • Last reply 3 years ago
4