Sisense Community logo
     
    • Community Feedback
    • Chapters
    • Events
    • Forums
      • Help and How To
      • Product Feedback Forum
      • Strategy & Use Cases
    • Blogs
    • KB Docs
      • KB Docs
      • Add-Ons & Plug-Ins
      • APIs
      • Best Practices
      • Blox
      • CDT
      • Cloud Managed Service
      • Data Models
      • Data Sources
      • Embedding Analytics
      • How-Tos & FAQs
      • Onboarding
      • PySisense
      • Security
      • Sisense Administration
      • Sisense Intelligence & AI
      • Troubleshooting
      • Widget & Dashboard Scripts
    • Support
    • Learning
      • Sisense Academy: Free Courses and Certifications
      • Official Developer Documentation
      • Official Product Documentation
      • Official Sisense Youtube Channel
      • Sisense Compose SDK Playground
      • Official Sisense Discord
    • Use Case Gallery
    •      
    Discussions
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
    Discussions
    • TagsChevronRightIcon
    data governance
    • Blog banner
      • News & UpdatesChevronRightIcon

      Product Update | Asset Auditor incorporates user access, permissions, and asset sharing

                                                       

      A more user-centric Asset Auditor In this release, we’re excited to introduce a user-focused expansion of the Asset Auditor, including two new dashboards, Users and Users Validation, along with enriched underlying data. You can now easily understand: Who has access and permissions to which data assets How assets are shared across your organization Whether access could be impacting engagement Revealing how their access and permissions connect to your data assets With this release, you get a clearer, more actionable picture of how people and assets interact to empower better oversight. We’ve also made major improvements across the existing dashboards to integrate this new data, elevate insights, and provide more actionable recommendations. Assets can’t deliver value unless users can access and engage with them In Sisense, dashboards and data models are governed by separate access controls. And they don’t operate in silos. They’re shared, cloned, embedded, and repurposed across teams. When someone shares a dashboard, they may not have permission to share the underlying model. The result? Users open dashboards expecting insights, only to find missing charts or blank visuals. They’re unsure whether the data is broken, restricted, or simply unavailable, while the sharer assumes everything is fine. The Asset Auditor gives clear visibility into which users or groups have: Access to dashboards but not to the underlying data models Access to data models but no corresponding dashboard access No access to any dashboards Yet access alone doesn’t guarantee adoption, and adoption issues are often misdiagnosed as access problems. Are dashboards underused because people truly lack access? Or because they simply aren’t engaging with the content? By surfacing these mismatches, you can prevent confusion, improve collaboration, and ensure every shared dashboard delivers the full experience it’s meant to. By detecting both over-permissioning and under-permissioning, you can tighten governance without slowing productivity. Permission drift happens quietly, introducing operational risk long before it becomes visible Do users have the correct permissions? Do some users have too many permissions? Do users have permissions to data models or dashboards that they shouldn’t?  Use the Asset Auditor to see whether users have the right level of access: too little to be effective, or too much for their role. Identify misaligned configurations, such as users who maintain data model access for development or testing, but no corresponding dashboard access, which is a strong indicator that permissions no longer reflect the real workflow. By detecting both over-permissioning and under-permissioning, you can tighten governance without slowing productivity. Understand the reach of your dashboards across users and groups The Asset Auditor helps you understand  the reach of your dashboards across users and groups, revealing how far each asset spreads and where engagement actually concentrates. Detect and reduce redundancy,  find duplicates or overlapping assets shared across teams. Pair these insights with Sisense Usage Analytics to understand not just who can access assets, but who actively engages with them. By bringing these signals together, teams can zero in on whether the problem is permissions, visibility, or user behavior. The Asset Auditor provides much more data and insights beyond users and shares! Check it out and get smarter about how you manage your data assets . If you want to start getting better visibility into what your assets are doing inside your environment, reach out to us for a live demo or a free trial.

      Mia Isaacson
      Mia IsaacsonPosted 7 months ago
      0
               
    • Blog banner
      • News & UpdatesChevronRightIcon

      QBeeQ Asset Auditor: A smarter way to manage your Sisense data assets

                                                               

        Optimize to cut storage and processing costs, refine data models, and boost performance   Query and dashboard performance are closely linked, often hindered by bloated data models. Excessive columns, unused tables, and inefficient relationships force queries to process unnecessary data, slowing down dashboards. This leads to frustration, delayed insights, and lower productivity. Use the Asset Auditor dashboards to: See all your data sources and follow the dependencies across data sources, data models, tables, columns, and dashboards and widgets Identify table and column utilization across dashboards and widgets for better model design. Target and remove empty and unused data sources, data models, columns, and dashboards By reducing or removing unused tables and columns and optimizing queries, organizations can drive down storage and processing costs while increasing performance and user engagement.   Expose (and prevent) hidden dashboard issues affecting your users  A key risk in delivering analytics is unintended downstream effects from data model changes, causing broken widgets, missing calculations, and misleading insights. Without full visibility, teams may disrupt critical business data. Errors often surface only when users load dashboards, despite backend checks, leading to frustration, missed insights, and wasted troubleshooting time. The Asset Auditor will help you to identify the source of these errors, from deleted data sources or missing data down to widget-level errors - reducing the time to troubleshoot and identify root causes and push fixes. Use the Asset Auditor at each step to verify that dashboards are error-free when delivered to end-users. Plan and execute changes with more confidence When shared elements are scattered across dashboards, making changes can feel overwhelming without knowing the full scope. The Asset Auditor can help you confidently assess scope by identifying widget distribution across dashboards to answer the questions: Where are these widgets used? Can changes be done manually? Or do I need a script? Making changes to the underlying data models, while preventing errors, has never been easier because the Asset Auditor will show you exactly which dashboards are using which data models, and which widgets are using which tables and columns. When teams make modifications without full visibility, they risk disrupting critical business insights. By proactively assessing the impact of changes, organizations can prevent costly errors, reduce time spent troubleshooting, and maintain high-quality analytics. You can't optimize what you can't see Organizations pour resources into analytics, but without visibility into how data assets are used, inefficiencies pile up, wasting storage, slowing performance, and inflating costs.  For those responsible for maintaining Sisense environments, from data architects and model builders to dashboard designers, the challenge isn’t just creating reports—it’s ensuring the entire infrastructure runs efficiently.  Asset Auditor changes the game by providing full transparency into how data is structured, utilized, and performing across your Sisense environment.  With clear insights into dependencies, usage patterns, and optimization opportunities, teams can refine models, improve query speed, reduce storage costs, and ensure users get accurate, fast insights—all while preventing costly disruptions before they happen.

      Mia Isaacson
      Mia IsaacsonPosted 7 months ago • Last reply 7 months ago
      1
               
    • Blog banner
      • News & UpdatesChevronRightIcon

      Outer joins (preview) - Release notes

                                                       

        Introduction An outer join (left, right, full) combines data from two tables, including all matching rows and any unmatched rows from one or both tables, filling in NULL for missing data. Analytical platforms use outer joins to achieve: Broader analytical capabilities : Ensure all relevant data is visible, even if there is no exact match in another table (e.g., view all products, including products with no sales). Gaps identification : Easily spot data integrity issues and missing information or relationships, which is crucial for analysis and reporting. Outer joins in Sisense Outer joins are available beginning with 2025.4 as a preview feature (turned off by default). It is planned to be released as beta in 2026.1.1. Sisense has chosen to expose it through its data modeling attributes, placing stronger emphasis on data governance controls compared to the approaches taken by other market alternatives. Important - Outer joins are not yet ready for production, and thus are not officially supported yet. We recommend using it for testing purposes, on a dev environment only. If you’d like to test it on your own models, you’ll need to first enable the following flag: Admin → Server & Hardware → System Management → Configuration → 5 clicks on the logo → Base Configuration → Query → query.outerJoins.enabled = true Once done, you can access it through the data tab, inside any data model, when editing any table relationship. The Join type drop-down (see the screenshot below) is where you can control it. A build/publish action must be performed after changes to see them reflected in the dashboard. The default value selected is “Default”, which, for now, stands for “Inner join” as was always used in Sisense before. In the future, it might be able to inherit other flexible join behaviors from an upper-level setting/product, so by keeping that option selected, you are allowing it to stay flexible. To enforce an inner join at any time in the future, select “Inner join” explicitly. Example data - To test the outer join, use a data model with data integrity issues, such as Sample Ecommerce, which contains countries in the Dim table (Country) that do not exist in the Fact table (Ecommerce). For example, if you perform a full join between the 2 tables, build the model, and expose it in a dashboard widget, country.country ID 199 Tahiti will appear, side by side with N/A or NULL values in the ecommerce columns. Without an outer join, Tahiti would not appear at all, because there is no matching data in the ecommerce table. Known issues and limitations Planned to be addressed in 2026.1.1: Analytical engine as a prerequisite - The outer join feature is designed to work exclusively with the Analytical Engine (AE). During the preview phase of that feature, if AE is not used in a query, and the query performs a fallback (due to a “compatibility mode” setting), only inner joins will be performed, even if the table relationship indicates otherwise. Starting from 2026.1 and onwards, using and editing a join type for a relationship will be disabled if the analytical engine setting of the model is set for “compatibility mode”, rather than for “Analytical Engine”. Filter propagation issue - Filters are usually translated into WHERE statements, and are applied immediately on the source base tables, before any table join is performed. This is safe and even optimal when using inner joins. When an outer join is used, this behavior may be unsafe, as the result may still include data from tables that are not part of the filtered table. For example: Starting from 2026.1 and onwards, those WHERE statements will be propagated above the joined tables if the filters belong to the non-preserved table(s). Data security risks - Some data security features behave like filters, and although they are not exposed in the “Analyze SQL Query” output, they are implemented on top of it and may suffer from the same filter propagation symptom mentioned above. In addition, not all data security use cases were covered thoroughly before the preview version was released, and while it will be a focus of the next release, please verify it based on your own data security rules, and share with us any concerns or use cases that should be double-verified. Perspectives inheritance issue - Perspectives usually inherit the relationship attributes set in the root level of the data model. Until the next version is out, it is not yet implemented for the join type attribute, and thus needs to be defined individually per perspective. Relationship’s pane fixed visualization order - When defining a relationship between Table X and Table Y, the current interface chooses which table will be presented on the left side of the pane, and which on the right. This fixed order means that you may need to adjust the join type to achieve the desired semantic join. For example, if you want to achieve the semantic result of Table Y LEFT JOIN Table X, but the relationship visualization order is (Table X, Table Y), you should flip it and select the “RIGHT JOIN” type instead. We recognize that having to manually flip the join type can be counterintuitive, but please note that there is no limitation on the desired result, which can still be achieved in any visualized order. Planned to be addressed in future versions: Filtering NULL values in widgets - There is no current option to filter out NULL values that are created as a result of an outer-joined data set, as Sisense does not yet offer result set filters. Circular reference ambiguity - When there are multiple ways to reach from table X to table Y, the system will choose the shortest path that takes into account any active filter and required data points. That means that sometimes, mainly based on filter usage, the path of joined tables performed from table X to table Y may change. And while one path may define an outer join to be used, the other path may not define it. That is not a new behavior, and it may not be an issue if the data modeler considered it, so just make sure to take it into account. Join type flipping in query time - There could be a situation where a data model relationship is defined as Table X LEFT join Table Y, but the widget query performs Tables Y RIGHT join Table X. The result will still be the same, but for query planning purposes, Sisense might switch the join type used in the SQL to be consistent with previous query plans and ensure semantic equivalence. Feedback that we are looking to get In order to improve and deliver a much more mature version of the feature in 2026.1.1 and after, we will be highly appreciative if feedback from you, our dear users, is shared with us. Even partial feedback would be appreciated! We’ve made a list of questions to brainstorm around it, but any open feedback is welcome, and we’ll be happy to receive it as well. To share it, feel free to pass your feedback to your CSM, and/or directly to our product manager, who’s leading this initiative: Morli Ben David at morli.bendavid@sisense.com . Clarity & Naming: Is the join type interface clear and easy to understand at a glance? If you were training a new user, what aspect of the UI would you anticipate causing the most confusion? Default Behavior: Does the default join type (currently assumed to be Inner) meet your expectations, or should the platform suggest a different default based on the data relationship, or based on any other approach? Data Integrity Checks: Did any of the resulting dashboards or widgets built on the new outer-joined relationship display unexpected values, duplicates, or missing data that you did not see with the previous (inner join only) model? Query Performance: After building the model with Outer Joins, were the resulting dashboard queries faster, slower, or comparable to what you would normally expect for a similar level of data complexity? Stability: Were there any unexpected crashes, freezes, or data rendering issues when modeling with or querying data sets built using Full, Left, or Right Joins? Maturity: Given the known issues mentioned above are going to be resolved, which other missing capabilities are must-haves? Is it mature enough to go to production already? User Training/Documentation: What is the one piece of information or training material that would best help you explain this new feature's value to your data team or end-users? Missing Capabilities (Gaps) : Now that you have this control, what is the next most critical data modeling control or feature you feel Sisense is missing? It can be either in the data page or in other areas impacted, such as the dashboard/etc.  

      Morli Ben David
      Morli Ben DavidPosted 8 months ago
      0
               
    • Blog banner
      • News & UpdatesChevronRightIcon

      Important update: connections management GA release & API behavior changes

                                                                       

      Important update: connections management GA release & API behavior changes Hello Sisense Community! I would like to inform you about an upcoming change to the Sisense Fusion platform that will affect how you work with Data Source Connections . As part of the L2024.3 Service Update 2, we are releasing the Connections Management feature in GA, enabled by default. This update aims to unify the management of data source connections, making Connections Management the single solution.  With the GA release, legacy connections are automatically disabled, and we have introduced temporary backward compatibility for certain endpoints that previously returned connection parameters data.  Starting in Q2 2025 (L2025.2), these temporary measures will be removed , fully deprecating legacy connections.  While we support full backward compatibility within the application, some custom script solutions that directly interact with our API endpoints may be impacted. To help you navigate this change, we have prepared the following resources:  API and System Behavior Changes with Connection Management , which will help you determine whether any of the listed endpoints are used in your custom solutions and guide you in making any required adjustments. If you would like to connect with us for support in making the required changes, please do not hesitate to reach out. Thank you for your understanding and continued partnership. Oleksandr Krokha

      Oleksandr_K
      Oleksandr_KPosted 1 year ago
      0
               
      • Best PracticesChevronRightIcon

      Sisense Data Pipeline Best Practices

                               

      Sisense Data Pipeline Best Practices  Architecturally, It is important to understand the role of Sisense in your use case and where it will fit best in your data architecture from a high-level perspective. Sisense is an excellent visualization tool that supports use cases for cached, live, and hybrid data pipelines. The focus of this article is to explain how to manage cached data to optimize Sisense’s performance, quality, and availability. Sisense allows you to design custom solutions specific to your business requirements with a large library of ETL tools and data source connectors. These are important features that make Sisense an industry leader. With great power comes great responsibility; care must be taken with the power that Sisense is giving you. Before delivering data to Sisense, it is recommended you carefully examine your data pipeline. Having clean, consistent data transformed at the source will improve your build performance, data quality, and data consistency across endpoints. Complex transformations should be performed at the source or in an intermediary integration layer. An integration layer (between source and endpoints) allows you the capability to persist historical data, implement business rules (transformations), and perform data governance (cleansing). Large data sets, complex transformations, and large numbers of sources directly impact cache build performance. Sisense’s ability to connect to such a large variety of sources makes it flexible enough to be leveraged for complex data architectures with highly diverse assets. By consolidating your data in an integration layer, it decreases cost and improves performance. You gain the ability to maintain a minimal dataset at your endpoints, minimize the number of data sources directly connected to Sisense, and shape the schema of the data to fit the schema of the consumption endpoint. By persisting your data in an integration layer you gain the ability to maintain a minimal dataset at your endpoints, audit incoming updates, retain historical data, implement quality controls, and standardize business rule enforcement. An integration layer can consist of multiple data stores based on specific use cases and business requirements. Typical data stores that reside in an integration layer include staging, historical, transforming, and aggregation. Staging The ETL process to load data into an integration layer typically uses a temporary staging area where incoming data is checked to assure it matches the expected data types/schema. The staging area may be stored in memory or on a disk depending on process design, source, target, and implementation tools. In data integration processes, the most common failures are related to unexpected changes in the source data types/schema. The staging portion of the ETL process should have robust error handling/notification features to assure that these issues are handled quickly to allow the data flow to continue. While having the staging area on a disk does make production support easier (you can examine the result easily), you may initially implement an in-memory process and add persistence to the disk as needed (probably for your most challenging sources) in a future sprint. You may also find that for your data source/type the IO incurred by persisting to disk is too high regardless of the easier support opportunities.   Historical If you have historical data retention or change audit requirements, the data would move from staging into the historical store. The persistent historical store should contain all data received from all data sources based on the data retention requirements of the business for the relative data domain/source. Data types, schemas, and values should be preserved as received in order to assure compliance and simplify troubleshooting for production support issues. Having a persistent historical store also allows consumption endpoints to retain only the minimal data set (preserving resources) for normal operations while still having the ability to reload history or perform advanced queries against a complete historical set.   Transformed The transformed store is used to implement conformed data models, data governance, and business rules. Implementing transformations to conform and cleanse data for all consumers avoids data inconsistency across consumption endpoints. Development and tracing of data lineages are simplified by supplying a conformed model with standardized naming conventions.   Aggregated When needed, the aggregated store can be used to pre-aggregate common data sets depending on endpoint granularity requirements.   Interface Depending on the source database platform, materialized and/or non-materialized views are used to interface to consumption endpoints. Using these views as an interface allows management of the physical tables without system breakage when change happens. Using these interfaces, complex operations (joins, groups, predicates) can be performed in a native database environment. Please note that support for interface mechanisms on sources varies. Please check before you design.   Summary If you want to decrease costs while increasing performance, quality, and availability, think about implementing an integration layer. By moving processing to its most effective point in your data pipeline you gain the ability to: Reduce cost Improve performance Increase availability Audit incoming updates Retain historical data Implement quality controls Standardize business rule enforcement Minimize endpoint datasets size Minimize Sisense source connections Shape the source schema to reflect the endpoint schema Pre-aggregate common data sets   FAQ Why does Sisense allow me to connect to everything if the best practice is to use an integration layer? The ElastiCube is a data cache that uses MonetDB as an internal store to enhance dashboard performance. There are many use cases that require direct access to raw data. While Sisense meets this requirement, discretion should be used to be sure your implementation is aligned with your organization’s data management practices.   Are there any platform-specific best practices for supported source database platforms? Good idea! Comment below to tell us which platforms pique your interest.   What is a “domain-focused” elasticube? This is an elasticube that is designed to answer a specific set of questions focused on a single business domain. Domain-focused cubes are best implemented against clean consistent data to assure that data shared by multiple elasticubes reflect the same values. Let me know in the comments if you would like to see more information on this topic.   When would you use Sisense to directly query raw data sources for cached data? If you do not need to save historical data, audit data changes, enforce standard business rules, cleanse the incoming data, or supply the data to multiple consumption endpoints.   What is a data pipeline? A data pipeline is a series of data processing steps. If the data is not currently loaded into the data platform, then it is ingested at the beginning of the pipeline. Then there are a series of steps in which each step delivers an output that is the input to the next step. This continues until the pipeline is complete. In some cases, independent steps may be run in parallel.   Conclusion If you need additional help, please contact Sisense Support .

      Darwin
      DarwinPosted 3 years ago
      0
               
      • Best PracticesChevronRightIcon

      Assessing the Quality of Your Dashboard

                                       

      Introduction The following article discusses how to assess the quality and adoption of your dashboard. Table of Contents Table of Contents Introduction Table of Contents How to Measure a Dashboard's Quality? Planning Makes Perfect Assessing a Dashboard's Quality The "Practical" Test How to Test This? How to Resolve This Type of Issue? The "Data Relevance" Test Does It Address the Right Crowd? Does it Contain the Right Amount of Data? How to Test This? How to Resolve This Type of Issue? Incorrect Dashboard Type Missing/Redundant Information The "Data Correctness" Test How to Test This? How to Resolve This Type of Issue? Showing Outdated Data Showing Incorrect Information The "Visual Correctness" Test How to Test This? How to Resolve This Type of Issue? Wrong Widget Type Missing Visual Aids The "Intuitiveness" Test Easy Access to Data Ease of Comprehension Customizability and Interactivity How to Test This? How to Resolve This Type of Issue? Data is Hard to Access Data is Hard to Comprehend The Dashboard isn't Customizable / Interactive The "Adoption" Test How to Test This? How to Resolve This Type of Issue? How to Measure a Dashboard's Quality? Let's begin by defining what a "Good Dashboard" is. To assess the quality of your dashboard, you should measure the following aspects: Is the dashboard practical ? Does it  serve its purpose ? Does the dashboard display  relevant and  correct information ? Is the information in the dashboard displayed correctly ? Was the dashboard adopted  by the end-users? Is the dashboard intuitive for use? Planning Makes Perfect Correct dashboard planning is the key to its success! I recommend you read this article   discussing a dashboard's (high-level) development cycle. It breaks the process into easy measurable steps that start from the initial KPI planning to maintaining and adjusting your end product. Reading through it might help identify flaws during the dashboard's initial rollout process.   Assessing a Dashboard's Quality The "Practical" Test The practicality of a dashboard focuses on whether the dashboard serves its purpose. Going back to a dashboard's planning phase, try to recall the reason for building this dashboard. What were the end-users looking to achieve? What insight were they seeking? Possible answers are: Monitoring a person's or a group's achievements Monitoring a measurable process Help decide what to do next based on historical information Assess future steps in various scenarios (What-If Analysis) A good dashboard : Answers its purpose! How to Test This? To check whether your dashboard is practical, you should: Set up a call with one (or more) end-users and/or stakeholders using the dashboard Ask them whether they can make the intended decision based on it If they are - What is the process of making the decision? If they aren't - What are the obstacles that stand in the way of making it? (e.g., missing/incorrect data, bad user experience, etc.) Next, revisit the dashboard's goals: What question should it answer? What action can the users make based on it? Compare the information you got from your stakeholders and the plan made when designing the dashboard - Define the gaps. How to Resolve This Type of Issue? Work towards re-planning your dashboard to meet its goals. Focus on: Aligning the "Call for Action" Redefining the KPIs and the user story Redesigning and restructuring the KPIs Optimizing/adjusting the data model to be able to answer the different KPIs Resolving this type of issue will most likely require a complete dashboard redesign, followed by an entire UAT cycle. The "Data Relevance" Test The term "Data Relevance" breaks into two aspects: Am I displaying relevant data - Considering the audience of the dashboard? Am I displaying relevant data - Considering the decision  to be made? Does It Address the Right Crowd? Like any deliverable, one of the fundamental things to know is its audience. The answer to this question will affect the following: The type of dashboard you create The granularity of data you present The way information is presented A good dashboard : Is built with the right audience in mind. Does it Contain the Right Amount of Data? Making a decision requires having the right amount of data: Too little data - Could prevent the person from making the right call. Too much data - Could confuse the person and throw them off-track. A good dashboard : Has the right amount of data required to serve its purpose. How to Test This? To check whether your dashboard has relevant data, you'll have to: Assess who are the end-users and/or stakeholders using this dashboard Set up a call with one (or more) end-users and/or stakeholders using it Ask them what the process of making their decision is? Check whether the information they require is presented in the dashboard. Check whether the dashboard has redundant or irrelevant data. How to Resolve This Type of Issue? Incorrect Dashboard Type Check your dashboard's type - Based on the person using this dashboard, should it be operational, analytical, tactical, or strategic? Compare the dashboard type to the intended crowd to determine if it was designed correctly (for example, a "Tactical Dashboard" is aimed towards  upper management and will usually present long-term KPIs and high-level metrics) . Resolving this type of issue will most likely require a complete dashboard redesign, followed by an entire UAT cycle. Missing/Redundant Information If you've identified that certain information is redundant or missing, consider adding, modifying, or removing widgets from the dashboard. Doing so might be a simple task (especially when removing data). However, it might also end up as a more extensive project requiring a partial redesign of the dashboard and a partial/complete UAT cycle.   The "Data Correctness" Test Presenting inaccurate or outdated data is worse than displaying partial or excessive data. Two possible outcomes of stakeholders, basing their decisions on incorrect data, are: Experiencing negative events (such as revenue loss) Compromising the trust relationship between Sisense and the end-users Frequent causes for showing outdated data in your dashboard include: An ETL process that is not run frequently enough An ETL process that often/occasionally fails An ETL process using an incorrect table update behavior (e.g., "Accumulative" where it should be "Full") An ETL process that isn't synchronized with your Data Warehouse A source database being offline for an extended period Frequent causes for showing incorrect data in your dashboard: Wrong data modeling leading to unexpected Many-to-Many relationships Business questions that don't align with the data model (e.g., causing "Random Paths") Unexpected / Missing inheritance of filters Wrong data security configuration The use of multiple data models (in the same dashboard) built at different schedules A good dashboard : Displays precise and up-to-date data How to Test This? To check whether your dashboard has correct data, you'll have to: Assess the different widgets on the dashboard to see the granularity of data displayed For each widget, check what data model it relies on - Use the Usage Analytics "Usage - Builds" dashboard to monitor the behavior of historical builds. Set up a call with one (or more) end-users and/or stakeholders using this dashboard Ask them if they trust the data on the dashboard and if the data refresh frequency is sufficient. How to Resolve This Type of Issue? Showing Outdated Data Ask your stakeholder how frequently they expect the data to be refreshed. Perform the following actions: Check the data model's build frequency - Should the build frequency be modified? Check the build time - Should the data model be optimized? Is the data model too heavy? Check the build success rate - Why are builds failing? Is the system running low on resources? Should "Data Groups" be applied? If using a Data Warehouse (DWH) and an Elasticube - Check the DWH build frequency; Check the synchronization between the DWH ETL finish and the Sisense ETL beginning. If you can't achieve the required build frequency, check whether all widgets require the same data refresh rate - Can a "Live Model" or a "Hybrid Dashboard" be considered? Showing Incorrect Information Ask a stakeholder to generate a report with correct data or ask them to point you to "defective widgets." Perform the following actions: Check the formula behind the figure(s) showing the incorrect data and correct them. Consider the different filters, inheritance behavior, etc. Examine the query path used to calculate each figure (look out for unexpected "Many-to-Many relationships" or "Random Paths") - Use the "Visualize Queries" add-on ( Link ) to compare Sisense behavior against the expected query path. Run the calculation in the data model using a custom table and a SQL statement that emulates the dashboard's calculation.   The "Visual Correctness" Test Data alone isn't enough; it must be visualized correctly. Visualizing data requires using the correct widget type and visual aids to convey the message. For example, say a  KPI is showing the company's revenue. See the different visualization options below: Option #1 Option #2 Option #3 Transitioning from option #1 to #2 provides an added value of "Good" / "Bad" Transitioning from option #2 to #3 provides an added value of revenue ranges A good dashboard : Has widgets that are visualized correctly and convey a clear message. How to Test This? To check whether your widgets are visualized correctly, you'll have to a nalyze each widget individually : Categorize each widget to find out its aim (e.g., show a single figure, compare values, show behavior over time, visualize data to show distribution, etc.) Verify the visualization matches the widget type (e.g., An indicator widget is perfect for displaying a single figure, a line chart is ideal for showing a behavior over time, etc.) Make sure each widget has visual aids (such as conditional formatting) to convey its message clearly - Each widget should tell a small puzzle of the story (which the dashboard should put together to a larger picture) Set up a call with people who are not familiar with the dashboard Ask them to describe each widget (separately) and their conclusion from looking at it. How to Resolve This Type of Issue? Wrong Widget Type Resolve the issue by fixing the visualization type - Use the following chart to help out : Type Use Case Indicator Show a single figure (numerical) Show a single figure and a gauge representing its range Column Chart Show a comparison among different sets of data Track data sets over time Track individual values + Their sum (stacking) Bar Chart Compare many items Track individual values + Their sum (stacking) Line Chart Reveal trends, progress, or changes that occur over time The data set is continuous rather than full of starts and stops Area Chart Displaying absolute or relative (stacked) values over a time period Analyzing a Part-to-whole Relationship Pie Chart One static number, divided into categories that constitute Represent numerical amounts in percentages Table Display RAW granular data Pivot Display RAW granular data Display aggregative data in a table format Scatter Plot Comparing large numbers of data points without regard to time Identify a potential relationship between two variables Calendar Heatmap Show relative number of events for each day in a calendar view Missing Visual Aids Visual aids help the end-users identify "Good" or "Bad" values. Possible visual aids include: Adding colors to numerical labels (e.g., Green figure vs. a red one) Adding background colors to table cells (e.g., Color negative cells red) Adding a line chart representing a threshold (e.g., A red line to indicate a lower threshold) Converting numerical values with text (e.g., Showing a "Revenue increased" label rather than a positive figure) Adding trend lines Adding a forecast range   The "Intuitiveness" Test Dashboard intuitiveness refers to the ability of a non-technical person to: Access the dashboard Understand and conclude the dashboard Understand how to customize the dashboard (using filters, drilling, etc.) Easy Access to Data There are two methods of accessing a dashboard: Sisense offers a complete HTML5 platform (a.k.a. Sisense Web Application ) that can be white-labeled and customized to meet the company's look & feel (i.e., color pallet, logo, links to internal support and documentation pages, etc.). Sisense offers three different embedding deployment options (conventional iFrames, an embedding SDK, and a complete JS-based embedding solution). Embedding the dashboard (or individual widgets) allows infusing analytics into web pages and applications. Choosing the proper access method will affect how end-users access data and benefit the BI solution. To improve intuitiveness - Make sure to streamline the process of consuming data as much as possible. The need to switch between multiple applications will result in a lack of efficiency, a complex adoption, or even a lack of adoption. Ease of Comprehension The comprehension of individual widgets was discussed earlier. However, is "the whole" greater than the sum of the parts? Does the dashboard tell a story? An excellent example for a dashboard: Widget #1 shows the revenue is low. Widget #2 shows a revenue breakdown per department and points out one department losing money. Widget #3 provides an income/expanse category breakdown and points out non-proportional marketing expenses. Widget #4 provides a detailed transactional income/expanse breakdown and points out the individual expenses Customizability and Interactivity Stale reports tell one story. However, a person looking at the dashboard may want to filter, sort, and pivot the data to tell the same story about a specific segment or individual. The tools for customizing/interacting with a dashboard include: Filtering data to a specific segment of interest ( Link ) Drilling into a measure to be able to extract more information about it ( Link ) Adding explanations ( Link ) and narratives ( Link ) A good dashboard : Is easy to access, self-explanatory, easy to customize, and play around with. How to Test This? To check whether your dashboard is interactive, you'll have to: Set up a call with one (or more) end-users and/or stakeholders using it Ask them when they use this dashboard and their workflow of consuming its information (e.g., A customer requires the data when building a financial report, and his workflow includes opening another browser tab and logging into the Sisense Web Application). Open the dashboard and ask them to explain it. Try to identify widgets that were over-explained and widgets they skipped. Track the questions you have to ask to understand what you see. Track the amount of "mouse scrolling" they perform when explaining the dashboard flow. Ask them what interaction they have with their dashboard and lay out the tools to increase interaction. Note the tools they are interested in (e.g., extra filters, adding narratives to widgets, etc.) How to Resolve This Type of Issue? Data is Hard to Access Ease of access is easy to measure - Count the number of clicks the user has to make when they want to consume the data in this dashboard - Fewer clicks = Easier to access. If data is hard to access or requires too much "Clicking around": Integrate SSO or WAT to prevent the user from logging in to the Sisense Web Application Embed the dashboard (or a single widget) into the application/webpage Have additional data imported to Sienese - Allowing the user to have a "One Stop Shop" for their entire workflow Enable sending periodic reports to the user's email Enable NLQ to allow the user to interact with Sisense easily ( Link ) Infuse data into G-Suite applications ( Link ) Enable Pulse alerts to send push notifications to the user ( Link ) Data is Hard to Comprehend If your widgets tell the right story, but the complete picture doesn't make sense: Move widgets around to make the story clearer Add visual separators between certain widgets (requires scripting) Reduce the number of widgets by splitting your dashboard into multiple stand-alone dashboards Use add-ons to simplify dashboard navigation (e.g., Accordion) Bring in a UI/UX designer to help visualize correctly Redesign the dashboard to avoid scrolling the mouse The Dashboard isn't Customizable / Interactive Make the dashboard more customizable by allowing the user to: Filter data based on predefined filters Define hierarchies to make filtering more intuitive Integrate filtering abilities into the dashboard (e.g., BloX buttons, drilling options, clickable widget values) Add additional widgets (e.g., Accordion, Switchable Dimensions, Tabber, etc.) Add premium widgets (e.g., Advanced Input Parameters - Link )   The "Adoption" Test So you've created a great dashboard; it has all the correct data and visualizations, a person can access it quickly and draw the proper conclusion by just looking at it, but it wasn't adopted. Has the company done enough to drive its adoption (e.g., sufficient training, decommissioning the old dashboard, integrating it into the users' application, etc.)? A good dashboard : Is measured by its adoption and usage. How to Test This? To check whether your dashboard has correct data, you'll have to: Find out who this dashboard's audience is Use the Usage Analytics "Usage - Dashboards" dashboard to monitor who accesses this dashboard and how often. Set up a call with one (or more) end-users and/or stakeholders who are not using this dashboard Find out why this dashboard isn't being used. Find out what alternative ways they use to collect the data How to Resolve This Type of Issue? Once you're sure the dashboard is perfect and the only thing missing is adoption: Involve users in the design meetings and UAT loop to make them feel they are part of the process Create an adoption plan! Get executives to buy-in - Causing them to promote their usage Monitor what people are using the dashboard and which aren't - Target the right people Reeducate your end-users on the benefits of the dashboard and the value of using it Decommission old dashboards and reports Refresh your dashboards from time to time (design, contents, etc.)

      Ophir_Buchman
      Ophir_BuchmanPosted 4 years ago • Last reply 4 years ago
      2
               
      • Best PracticesChevronRightIcon

      Many-to-Many Relationships - Knowing and Avoiding Them

                               

      Introduction The following article discussed "Many-to-Many" relationships - Both expected and unexpected. It focuses on how they occur and the best practices which one can implement to avoid them.   Table of Contents Table of Contents Introduction Table of Contents Table Relationships The One-to-One Relationship The One-to-Many Relationship The Many-to-Many Relationship Should One Avoid "Many-to-Many" In All Costs? The "Expected Many-to-Many" The "Unexpected Many-to-Many" Related Risks Dealing with Many-to-Many Relationships Avoiding Many-to-Many Relationships Bullet-Proofing your Data Model Other Mitigation Steps Sisense Best Practices Use a "Fact-Dimension" Table Schema Why? How? Never Link a "Fact" Table to Another "Fact" Table Why? Never Link a "Dimension" Table to Another "Dimensions" Table Why? Link all "Dimension"Tables to All "Fact"Tables Why? How? Consolidate "One-to-One" Dimension Tables Why? How? Hide Irrelevant Columns Why? Scenario #1 Scenario #2 How? Avoid Dimension Table Cycles Why? How? Validate Dimensions' Primary Keys are Not NULL and Unique Why? How? Table Relationships When modeling data, BI recommends you either use a "Flat" or a "Fact-Dimension" table schema: To implement a "Flat" table schema - The user would denormalize all their data into a single table. Denormalizing data will require extensive data duplication, row duplication, and result in exponential growth of the database size. For example: Placing all "product names" in a "sale transactions" table will cause a duplication of strings in your data model. To implement a "Fact/Dimension" table schema - The user would denormalize some of their data and place it in a two-level hierarchy model: The first level is the "Fact" level (a.k.a. "Measures") and is used to store transactional data that could be aggregated.  The second level is the "Dimension" level (a.k.a. "Filters") and is used to store descriptive information about the transactions. This data would be used towards filtering or segmenting.  When using multiple data tables in the model, a user has to teach the system how to join them in case of a cross-table query. For example, if the "Fact" table describes sale transactions and the "Dimension" describes the products sold - The user may request to summarize the sales amount of all red products. The logical connection between the tables is known as a "Table Relationship" and is performed by linking two related fields from the different tables (a.k.a. Keys). For example, a "Fact_Sales" table listing sale transactions holding a "Product ID" column (which represents the product purchased) and a "Dim_Products" table describing products having a "Product ID" column (which represents the product described) A "Table Relationship" describes one of the following relations: A One-to-One relationship A One-to-Many relationship A Many-to-Many relationship   The One-to-One Relationship The "One-to-One" relationship represents a relationship where each value in the "Key Column" of "Table A" exists either once or zero times in the "Key Column" of "Table B" and vice versa. For example: A customer may have a store membership; a store membership belongs to a single customer. A city may be the capital of one country; a country may only have one capital. Each employee (identified by "Employee ID") may have one SSN; an SSN may belong to one employee. A "One-to-One" relationship is usually visualized in the following way: Each entry in "Table A" shows once in "Table B." Each entry in "Table B" shows once in "Table A."   The One-to-Many Relationship The "One-to-Many" relationship represents a relationship where each value in the "Key Column" of "Table A" exists either once or zero times in the "Key Column" of "Table B" and each "Key Column" of "Table B" may appear multiple times in the "Key Column" of "Table A." For example: A person belongs to one family. However, the family includes multiple people. A person was born in a single hospital. However, many people were born in that hospital. An airplane is parked in a single airport. However, the airport accommodates multiple planes. A "One-to-Many" relationship is usually visualized in the following way: Each entry in "Table A" shows once in "Table B." Each entry in "Table B" shows once or more in "Table A."   The Many-to-Many Relationship A "Many-to-Many" relationship represents a relationship where each value in the "Key Column" of "Table A" may appear multiple times in the "Key Column" of "Table B" and vice versa. For example: A person may belong to multiple groups; a group may contain multiple people. A product may belong to multiple categories; each category includes various products. A student participates in multiple classes; a class includes numerous students. A "Many-to-Many" relationship is usually visualized in the following way: Each entry in "Table A" shows once or more in "Table B." Each entry in "Table B" shows once or more in "Table A."     Should One Avoid "Many-to-Many" In All Costs? So, why are Many-to-Many relationships so big of a deal? Let's examine both the "Expected" and "Unexpected" many-to-many relationships. The "Expected Many-to-Many" In some cases, Many-to-Many relationships are expected and planned. Let's examine this company that sets up fan groups and stores their data in the following user/group table schema:   Let's attempt to calculate the following: What is Dan's monthly admission? We'll have to join both tables on the "Group ID" field and filter on Dan's name to calculate this. The result would be a SUM aggregation of the "Admission Price" column (Result is 20$+30$+5$ = 55$) How many people participate in the "Boardgame Fans" group? We'll have to join both tables on the "Group ID" field and filter on the group's name to calculate this. The result would be a COUNT aggregation of the "User ID" column (Result is 2 People) What is the entire cash flow? To calculate this - We'll have to group the user/group table based on the "Group ID" and count the users per group. Then, join both tables on the "Group ID" field. The result would be a SUM of the multiple of the "User Count" and the "Admission Price" (Result is 40$+90$+10$+12$ = 152$)   The "Unexpected Many-to-Many" Many-to-Many relationships are created unexpectedly - This could happen due to an unexpected "wrong" business question or inadequate data modeling. Let's examine this retail use-case where a company stores purchases (made against vendors) and sales (made against customers): Let's attempt to calculate the following: How many "Chocolate Bars" were sold? To calculate this - We'll have to join the "Products" and "Sales" tables on the "Product ID" field and filter on "Chocolate Bars." The result would be a SUM aggregation of the "Quantity" column (Result is 6+1 = 7) Which Customer Made the Most Purchases (money-wise)? To calculate this - We'll have to group the "Sales" table purchases by the "Customer name" and perform a SUM aggregation of the "Sale Price." Then, find the maximal value (Result is "Dan" - 75$) How many sales were performed by "Aid Inc."? This question is an interesting one as the company doesn't sell to vendors - Most likely a "wrong" user question. Even though, Sisesne will attempt to find a way to calculate this. It will do that by Joining the "Vendors" table with the "Purchases" table on "Vendor ID" filtering "Aid Inc.". It will then take the products found and join them against the "Sales" table. The result will include the SUM of Sales Price" (Result is 20$+12$+25$ = 57$) - Which is incorrect.   Related Risks Many-to-Many related mistakes (such as the one presented in the previous section) may lead to the following: Displaying no data in your widget Displaying incorrect data in your widget Generating a huge JOIN operation - Leading to excessive use of memory and/or computing power In severe cases (where data tables are huge) - A many-to-many relationship may lead to an Elasticube crash or a complete Sisense instance failure   Dealing with Many-to-Many Relationships There are two ways of dealing with many-to-many relationships: Avoiding them Bullet-proofing your Data-Model   Avoiding Many-to-Many Relationships The easiest way to resolve many-to-many relationship-related issues is by avoiding them. Avoiding these relationships isn't necessarily easy. Doing so might be very expensive in terms of computing resources (data transformation) and data storage (data duplication). For Example: Before After Notice the data duplication   Bullet-Proofing your Data Model Another way to deal with many-to-many relationships and avoid related risks (discussed in the next section) is bullet-proofing the data model. There are a few simple practices you should follow to avoid errors originating from incorrect business questions - These are discussed in the "Sisense Best Practices" section.   Other Mitigation Steps Talking from experience, you should expect the unexpected and plan ahead - Meaning that someone may unintentionally generate a many-to-many relationship and risk the Sisense instance stability. As a best practice, configure "Data Groups" to limit the resources allocated towards a single Elasticube. Refer to the following link to read more about Data Groups.     Sisense Best Practices Each best practice answers a different data modeling mistake or a potential "wrong" business question.   Use a "Fact-Dimension" Table Schema BI best practices recommend using a two-level table hierarchy model including: Facts (a.k.a. Measures) - Containing data that is aggregated (e.g., A list of sale transactions) Dimensions (a.k.a. Filters) - Describing the data inside the fact (e.g., A list of products and their characteristics) Sisense recommends implementing the "Fact/Dim" schema structure. Why? The reasons behind this guideline are: Having a two-level table hierarchy model reduces the amount of JOIN events during a query's execution. This results in a shorter query runtime, less compute operations, less memory consumption, and provides a better user experience. Having a two-level table hierarchy model simplifies the data model making it easier to debug data inconsistencies and follow the query path. There's only one single way to get from every fact to every dimension (and vice versa) How? Data modeling is a task that requires much planning and may be very challenging. The rule of thumb when migrating your database structure into a dim/fact structure is to start by finding out what the business' questions are. Then, use the business questions to define what data is used for filtering and segmenting (dimension) and what data is used for aggregations (facts). Modeling your data correctly is a key to your dashboard's success (in terms of runtime performance and data accuracy).   Never Link a "Fact" Table to Another "Fact" Table Two "Fact" tables should never connect to each other - A "Fact" table should only connect to "Dimension" tables Why? Connecting two "Fact" tables directly may lead to a JOIN between them - Potentially causing a huge query to take place and crashing the ElastiCube.   Never Link a "Dimension" Table to Another "Dimensions" Table Two "Dimension" tables should never connect to each other - A "Dimension" table should only connect to "Fact" tables Why? "Dimension" tables are used to describe the data in the "Fact" table. Different "Dimension" tables usually don't relate to each other and if they do, linking them may lead to a "Table Cycle" (see the relevant section)   Link all "Dimension" Tables to All "Fact" Tables Every "Dimension" table should be connected to all "Fact" tables. Every "Fact" table should be connected to all "Dimension" tables. Why? As a reminder - Every "Dimension" field may be used towards filtering and/or segmenting data. Every "Fact" field may be used towards aggregating data. You user can choose to filter and aggregate any sets of fields, leading to a "Random Path". How? Some Fact/Dimension relationships are straightforward. However, some are not. Let's check out the previous example: Connecting the "Product Dimension" to the "Fact" tables is straightforward as every sale and purchase event refers to a specific product. However, connecting the "Vendor Dimension" table isn't - A vendor doesn't participate in a sale event. To resolve this, you'd have to create a  Fake Key - Allowing a relationship and making sure the tables either ALWAYS or NEVER join. Read more about fake keys at the following link .   Consolidate "One-to-One" Dimension Tables In some cases, you may have dimensions connecting on the same primary key. If two dimensions holding the same primary key have a One-to-One relationship, Sisense recommends consolidating these to decrease the amount of JOIN operations. Why? Take a look at the following example: Assuming each person has one driver's license and each driver's license belongs to one person, the two dimensions should be consolidated. By doing so, a dashboard attempting to filter tickets based on age range and license type would require one less JOIN operation. How? Refer to the following link  for details on how to create custom tables. The resolution of our example is as follows:     Hide Irrelevant Columns In many cases, columns used for creating relationships (primary/foreign keys) are also public keys. But in case the end-user actually uses a "User ID" in his filtering - A dashboard designer may use the incorrect field name when creating a widget, leading to incorrect query results (Many-to-Many) and query failure in extreme cases. Sisense recommends hiding "Key" columns from the user as much as possible. Why? Take a look at the following example:     Scenario #1 The Customer builds a widget to count the number of sale purchases containing product #3: The aggregative field is "Purchase ID" (in the "Fact_Purchases" table) The filtering field is "Product ID" (mistakenly chosen from the "Fact_Sales" table) The result may be incorrect (i.e., If product #3 was never sold) and will require joining the two "Fact" tables (a VERY expensive operation). Scenario #2 The Customer builds a widget to count the agents who sold apples: The filtering field is "Product Name" (in the "Dim_Products" table) The aggregation field is "Agent ID" (mistakenly chosen from the "Dim_Agents" table) The result will be unpredictable and most likely incorrect as there's no way to determine which Fact table will be used for joining these two "Dimensions" - This is known as a " Random Path " How? Refer to the following link for details on how to hide a column. The resolution of our example is as follows (grayed-out fields are "Hidden"):   In scenario #1 - There's only one "Product ID" field In scenario #2 - There's only one "Agent ID" field Notice that the hidden fields align with the relevant business questions   Avoid Dimension Table Cycles The term "Chaining Dimension Tables" refers to a scenario where you connect a fact table to a dimension table and then chain an additional dimension table to that table. Sisense recommends avoiding table chaining as much as possible. Why? Using this type of structure introduces unnecessary JOIN operations - For example, trying to count all "Red" products sold: Using this type of structure may also lead to cases where to data structure has a Table Cycle  - Leading to unexpected query results - For example, trying to count all sales with a "Red" filter:   How? Resolving table cycles can be done in various methods (depending on the business questions) - Here are three possible solutions for this problem: Denormalize the "Color Name" and place it in both "Product" and "Package" dimension tables. This will reduce on JOIN event per query and allow the user to choose between the two filters Disconnect the color dimension from the packaging dimension. This is a valid solution IF there are no business questions based on packaging color Duplicate the "Color" dimension - Once for the product, and the other for packaging.   Validate Dimensions' Primary Keys are Not NULL and Unique The primary keys of your "Dimension" table should be unique and not NULL Why? Making sure the primary key is unique will guarantee a "One-to-Many" relationship (rather than a "Many-to-Many" one). Consider this case: Note the error made in the "Products" dimension table - Two items have the same "Product ID". A simple query may result in a JOIN operation that requires twice the amount of work, but more importantly, results with an incorrect result. How? To monitor your dimension tables create a dup-check table - Counting the following: Value Definition Expected Value Key Count The total number of keys --- Unique Key Count The number of unique keys Should be equal to "Key Count" Null Key Count The number of NULL keys Should be equal to zero To create such a table - Create a "Custom Table" with a code similar to: SELECT<br/> 'Customers' AS Table_Name,<br/> COUNT(c.[CustomerID]) AS Key_Count,<br/> COUNT(DISTINCT(c.[CustomerID])) AS Distinct_Keys,<br/> SUM(CASE WHEN c.[CustomerID] IS NULL THEN 1 ELSE 0 END) AS Null_Keys<br/>FROM<br/> [dim_Customers] c<br/><br/>UNION<br/><br/>SELECT<br/> 'Employees' AS Table_Name,<br/> COUNT(c.[EmployeeID]) AS Key_Count,<br/> COUNT(DISTINCT(c.[EmployeeID])) AS Distinct_Keys,<br/> SUM(CASE WHEN c.[EmployeeID] IS NULL THEN 1 ELSE 0 END) AS Null_Keys<br/>FROM<br/> [dim_Employees] e<br/><br/>UNION<br/><br/>...

      Ophir_Buchman
      Ophir_BuchmanPosted 4 years ago
      0
               
      • Best PracticesChevronRightIcon

      How to configure Data Groups

                               

      Introduction This article will guide you on how to configure the Data Groups’ parameters based on the current performance and business requirements. What are "Data Groups”? The “Data Groups” administrative feature allows configuring Quality-Of-Service (QoS) to Sisense instances and limiting/reserving the resources of individual data models. By utilizing data groups you’ll make sure available resources are controlled and managed to support your business needs. Why Should One Configure “Data Groups”? The main benefits of configuring “Data Groups” are: Controlling which cluster nodes are used for building and querying Limiting the amount of RAM and CPU cores a data model uses Configuring indexing, parallel processing, and caching behavior of a data model Mitigating “Out-Of-Memory" issues caused by lack of resources Preventing “Safe Mode” exceptions caused by lack of resources Read this article to learn about the different “Data Groups” parameters “Data Group” Use Cases The following two scenarios are good examples of using "Data Groups”: Scenario A customer embeds Sisense (OEM use case) and has many tenants that can design dashboards (a.k.a. Self-Service BI). Each tenant has a dedicated Elasticube data model. Challenge A single tenant is capable of creating a large number of dashboards - Causing the underlying data model to utilize a significant portion of the available CPU/RAM resources. The excessive use of resources by a single tenant may lead to resource depletion and degrade the user experience for them and other tenants. Solution Resource Governance – Set up resource utilization limitations using "Data Groups”. Doing so will limit each tenant’s usable resources and prevent tenants from affecting each other.   Scenario A customer has many Elasticube data models created by various departments of the organization. Some of the data models are used to generate high-priority strategical dashboards (used by C-Level managers). Other dashboards are prioritized as well (e.g., operational dashboards vs. dashboards used for testing) Challenge C-Level dashboards must have the highest priority and should always be available. Other dashboards should still be operational and behave based on their priority. In case of conflict or a temporary lack of resources, a critical Elasticube may run out of memory or trigger a ‘safe mode’ event. Solution Resource Governance – Setting up priority-based Data Groups would result in allocating business-critical data models with more resources and limiting the less critical ones.   Data Groups Resource Governance Strategies Limiting a Data Model’s Resources - Governance rules can be used to limit the resources used by a single data model or a group of multiple data models. This can be done by configuring a "Maximal” amount of allocated CPU/RAM. Note that the data model would be limited to the configured resource restrictions even though additional resources are available for use. Pre-Allocating a Data Model’s Resources - Governance rules can be used to pre-allocate the resources used by a single data model or a group of multiple data models. This can be done by configuring a “Reserved” amount of allocated CPU/RAM. Note that other data models would be limited to partial server resources even though a pre-allocated resource might be idle. Prioritizing Data Models - Governance rules can be used to prioritize certain data models by allocating a different amount of resources to different data models. High-priority data models would be allocated with more resources than lower-level data models. You may also choose not to limit the data model’s resources (no Data Group configuration). However, this will re-introduce the initial risk of resource depletion.   How To Configure Data Groups? Configuring “Data Groups” requires high expertise and attention to detail. A dedicated tool is introduced to assist you in this task. Follow the directions below to acquire the tool and implement Data Groups: Prerequisite To base your calculations and make a decision on how to configure data groups you’ll require data monitoring enabled. To learn more about Sisense monitoring and the data reported read the following article: https://support.sisense.com/kb/en/article/enable-sending-sisense-monitoring-logs Step #1 – Download the “Data Groups Configuration Tool” The data groups tool (Excel Document) is attached to this article. Download a local copy to your computer. Step #2 – Access Logz.IO To access the Logz.IO platform: Log in to the Logz.io website Navigate to the “SA - Linux - Data Groups“ dashboard Set the timeframe filter to a 21-day timeframe Step #3 – Fill out the “Data Groups Configuration Tool” Document The “ Elasticubes ” sheet holds the data for the decision making regarding the different groups: Field Description Comment CubeName The ElastiCube name   Size GB The ElastiCube’s size on disk The measure is taken from the “Data” tab ElastiCube list Estimated size in memory The estimated maximal ElastiCube size in memory Auto-calculated ([Size GB] X 2.5) Peak query memory consumption (GB) The actual maximal ElastiCube size in memory (should be smaller than the estimated maximal size) The measure is taken from the logz.io dashboard: (“Max Query Memory” field) Build frequency The frequency of the ETL Sisense performs The measure is taken from Sisense’s build scheduler Average of concurrent Query Average concurrent queries sent from dashboards to the ElastiCube The measure is taken from the logz.io dashboard: Search for the ‘Max concurrent queries per Cube’ widget and download the CSV file with the data. Calculate the average value of the “Max concurrent queries per Cube: column Business Criticality This is a business measure that determines the QoS of the ElastiCube True (High) / False (Low) Data security Is “Data Security” applied to this ElastiCube This column will help determine if the “ElastiCube Query Recycler” parameter will improve the performance or not. Explanation here under “ElastiCube Query Recycler” Data Group The Data Group this ElastiCube should be a part of Try to fill in the column by classifying your ElastiCubes according to common characteristics both in terms of business and in terms of resource consumption.   Step #4 – Define your Data Groups Using the information you’ve collected and the explanations in this article – Define the data groups you wish to use. Fill in the information in the “Required data groups” sheet. The “ Required data groups ” sheet provides the name and configuration for each data group. Use this to tab describe the Data Groups that meet your business needs, use the “Intent” column to describe each group's purpose. The configuration in this sheet will later be used to configure the Data Groups in the Sisense Admin console: Field Description Comment Group name The Data Group’s name   Intent The Data Group’s description (in the business point of view)   Instances The number of query instances (pods) created in the Sisense cluster This parameter is very useful, however, increasing this value will result in increasing the Elasticube’s memory footprint.   You should only consider changing this value if your CPU usage reaches 100%.   The high CPU consumption is mostly caused by high users concurrency Build on Local Storage   Is enabled, for Multi-node. Using local storage can decrease the time required to query the cube, after the first build on local storage Simultaneous Query Executions The maximum number of queries processed simultaneously In case your widget contains heavy formulas it is worth reducing the number to make the pressure lower. Used when experiencing out-of-memory (OOM) issues. Parallelization is a trade-off between memory usage & query performance. 8 is the optimal amount of queries % Concurrent Cores per Query   Related to Mitosis and Parallelism technology. To minimize the risk of OOM, set this value to the equivalent of 8 cores. e.g. if 32-core, set to 25; for 64 cores set to 14; the value should be an even number.  Can be increased up to a maximum of half the total cores - i.e. treat the default as the recommended max that can be adjusted down only. Change it when experiencing out-of-memory (OOM) issues Max Cores Maximum cores allowed to be used by the qry pod Limit for less critical groups Query: Max RAM (MB) The Max RAM will be consumed per each of the ElastiCubes in each group. Take from the column “MAX of Peak query memory consumption (GB)” in the Summary of data groups sheet   Maximum RAM that is allowed to be used by the qry pod (dashboards). By each of the query instances if were increased in the Instances. Please note that 85% of the pod usage OR overall server RAM will cause a  Safe-Mode  exception. In the case of Dashboards qry pod) will delete and start the pod again. At the Safe Mode, the users of a dashboard would see the error of a Safe Mode and the qry pod will be deleted and started again. So the next dashboard refresh will bring the data Step #5 – Verify The “ Summary of Data Groups ” sheet includes a pivot chart that will calculate the max memory consumption from each data group. This value correlates to the “ Query: Max RAM in MB” configuration. We need to take the maximal value of Peak query memory consumption (GB) from the Summary of data groups tab and multiply it by 1.2 0 to avoid Safe mode. Step #6 – Process the results and configure Sisense Data Groups Prerequisite - Read this article on how to create data-groups (5 min read): https://documentation.sisense.com/docs/creating-data-groups On the Sisense Web Application, for each new data group: Navigate to “Admin” --> Data Groups” and click the “+ Data Group” button. Fill out the information from the “Required data groups” tab. Step #7 – Monitor the Sisense Instance The final step is to follow the information in the “Sisense Monitor” and to make sure the performance is improving. Review the documentation below for more details regarding monitoring https://docs.sisense.com/main/SisenseLinux/monitoring-sisense-on-linux.htm Good luck!

      Liran_Elnekave
      Liran_ElnekavePosted 4 years ago
      0
               
    • Blog banner
      • News & UpdatesChevronRightIcon

      Introducing Sisense Fusion 2021.10: Enhanced personalization, streamlined support, and more!

                               

      Infusing analytics into workflows is the way of the future. As Sisense, we’re always working on enhancements to help you get more out of your data without endlessly switching between programs. Our latest release takes Linux deployments to the next level, allowing users to upload new font libraries, streamline the customer support experience, and even parameterized web tokens, allowing users to interact with dashboards and other analytics without logging in.  Read on to dig into the details of these new features and see how they can improve your analytical experience. Q4 2021 October release: L2021.10 Release notes Sisense Fusion 2021.10 was launched a few days back and provides several new features and significant improvements to the Sisense Fusion platform on Linux. Key highlights:  Uploading custom fonts Personalize and match the aesthetic of your brand or website typography to your Sisense UI by uploading custom fonts and assigning them to specific themes, providing a fully customized and seamless rebranded experience for your users. Easily upload and apply your custom fonts when your preferred fonts are missing from the Sisense out-of-the-box fonts selection. Now molding Sisense into your organization's image becomes that much easier. Grant Access to your Account to Support Agents  At Sisense, we are always looking for ways to provide our customers with the best user experience possible. If you are a managed services customer, Sisense Support agents can now connect to your Sisense application in order to help troubleshoot issues as long as this option is turned on. All of this to reduce downtime and improve your productivity. Web Access Token Security is top of mind for organizations today, and Sisense will continue to provide the best security possible to our customers. The new web access token feature grants a secure, scalable, and customizable viewer role experience. This provides the ability to interact with dashboards and widgets without user login or password. The Web Access Token feature unlocks the ability to generate stateless and volatile sessions, with anonymous and public access to the analytical content. For further details on the features, look at the Release Notes with a full feature list as well as the list of enhancements, or contact your CSM.   Also, check out this video from our training team demonstrating how to use these new features: Sisense October 2021 Release Training Video on 4 selected Features .

      vidisha_v
      vidisha_vPosted 4 years ago
      0
               
    • Blog banner
      • News & UpdatesChevronRightIcon

      Sisense October 2021 release training video on 4 selected features

                               

      In this release, we focus on improving scalable usage of Sisense dashboards and managed services support and enhancing UI customization user management.  Here are the 4 selected features from the October release that you should know about: Support Access LDAPS support with Public key Web Access Token Custom Fonts Here is a short explanation of each feature:  Support and CloudOps teams now have secured and monitored access for customers’ environments enabling non-interruptive monitoring and troubleshooting. Active Directory for user management now supports public keys to avoid security issues.   Improved dashboard sharing for non users via any platform (including social platforms) using Web Access Tokens for a secure, scalable, and highly customizable view. Improved UI customization capability with customized fonts that can be used across all environments including embedded methods. And here is a 3-minute video showing you how each feature is implemented:     We hope you enjoy these new features and use them daily to make your work more efficient.  To learn more, go to Sisense LMS* under these courses: Admin capabilities ,  and Embedded Analytics . * The links above are to our LMS - if you don’t have a user - please contact your CSM. 

      AdvaDadush
      AdvaDadushPosted 4 years ago
      0