Sisense Community logo
     
    • Community Feedback
    • Chapters
    • Events
    • Forums
      • Help and How To
      • Product Feedback Forum
      • Strategy & Use Cases
    • Blogs
    • KB Docs
      • KB Docs
      • Add-Ons & Plug-Ins
      • APIs
      • Best Practices
      • Blox
      • CDT
      • Cloud Managed Service
      • Data Models
      • Data Sources
      • Embedding Analytics
      • How-Tos & FAQs
      • Onboarding
      • PySisense
      • Security
      • Sisense Administration
      • Sisense Intelligence & AI
      • Troubleshooting
      • Widget & Dashboard Scripts
    • Support
    • Learning
      • Sisense Academy: Free Courses and Certifications
      • Official Developer Documentation
      • Official Product Documentation
      • Official Sisense Youtube Channel
      • Sisense Compose SDK Playground
      • Official Sisense Discord
    • Use Case Gallery
    •      
    Discussions
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
    •                    
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
    Discussions
    • TagsChevronRightIcon
    Data Connection
    • Ihor Yokhym

      Knowledge Base Docs

               
      Ihor Yokhym
      Posted 2 weeks ago
      Connecting Sisense to Amazon DocumentDB via generic JDBC [Linux]
                                               

      Introduction: Learn how to successfully connect Sisense to Amazon DocumentDB using a generic JDBC connection. This guide applies to Sisense Linux editions (both Cloud and On-Premises deployment models) and resolves common TLS-related connection reset errors. Step-by-Step Guide:   Step 1: Whitelist the Sisense Instance Before modifying any configurations in Sisense, verify that your Sisense instance IP address or network range is properly whitelisted in your AWS DocumentDB cluster's security groups. The default connection port is 27017 . Step 2: Download the driver Get the JDBC Driver for Amazon DocumentDB from the Amazon website and Github: AWS DocumentDB: Documentation AWS DocumentDB: Driver Github Step 3: Register the connector in Sisense 2.1 Go to Admin > Server & Hardware > System Management > Configuration . 2.2 Scroll to the bottom and click Manage Connectors . 2.3 In the left pane, click +JDBC Connector and fill in: Connector ID: [YOUR_NAME] Display Name: [YOUR_NAME] Class Name: software.amazon .documentdb.jdbc.DocumentDbDriver Upload JAR File: upload jar file Click Create . Step 4: Use the Generic JDBC Connection String Because of architectural differences, connecting via the native Sisense MongoDB connector can result in errors. Instead, connect using the generic AWS DocumentDB JDBC driver syntax. JDBC Connection String Template jdbc:documentdb://<hostname>:<port>/<database-name>?<options> Step 5: Configure TLS Settings and Resolve Socket Exceptions According to official AWS DocumentDB JDBC Driver documentation, the driver’s default behavior is tls=true. If your DocumentDB configuration expects plain text, a protocol mismatch occurs: the JDBC driver attempts an encrypted SSL handshake with a database node expecting unencrypted packets. DocumentDB will abruptly drop the network socket, resulting in the following error: java.net.SocketException: Connection reset To fix this, modify your connection string based on your database environment: For Non-TLS Environments: Explicitly turn off TLS encryption by adding tls=false and completely removing the tlsAllowInvalidHostnames flag. For TLS Environments: Ensure both tls=true and tlsAllowInvalidHostnames=true are active. Code Type: JDBC Connection String Example (TLS Disabled) jdbc:documentdb://<a target="_blank" rel="noopener noreferrer nofollow" href="http://my-cluster.cluster-xyz.us-east-1.docdb.amazonaws.com:27017/my_database?tls=false"><u>my-cluster.cluster-xyz.us-east-1.docdb.amazonaws.com:27017/my_database?tls=false</u></a> [Screenshot: Generic JDBC connection setup window in the Sisense Data tab] Description: The image above illustrates where to input the customized DocumentDB JDBC connection string and credential details within the Sisense Data Connector UI. Conclusion:  Successfully connecting Sisense to Amazon DocumentDB requires leveraging the generic JDBC driver framework rather than the native MongoDB connector. The most common pitfall is a TLS mismatch; explicitly declaring tls=false in your connection parameters will bypass the java.net .SocketException when working with unencrypted database nodes. References/Related Content  Sisense Documentation: Deploy Custom JDBC connector AWS DocumentDB: Documentation AWS DocumentDB: Driver Github

                                             
      0
               
    • Luke Flett

      Help and How-To

               
      Luke Flett
      Posted 7 months ago • Last reply 5 months ago
      Filter to only show Relevant Dimension Values
                                       

      Morning Sisense community!  I have my Fact Table and a related Dimension Table . My dimension table contains more unique values than are actually present in the fact table. When I use a field from the Dimension Table as a filter, the filter displays all values from the dimension, even those that have no corresponding fact records . I only want the filter to show dimension values that are relevant to the data in the fact table . Example Tables: fact_table   dimension_table   id dim_id dim_id dim_name 1 1 1 one 2 2 2 two 3 1 3 three 4 2 4 four fact_table and dimension_table have a relationship in the EC based on dim_id Desired Outcome: In the filter based on dim_name, I only want to see "one" and "two" , but currently, I see all four: "one," "two," "three," and "four." How do I achieve my desired outcome?

                                             
      5
               
    • EdwardWhitemore

      Help and How-To

               
      EdwardWhitemore
      Posted 7 months ago • Last reply 5 months ago
      Dashboard not auto-refreshing for mobile app performance data
                               

      Hi everyone, I’m using Sisense to track performance metrics from my mobile entertainment app — like daily active users, watch duration, and session time. Lately, I’ve noticed that some dashboards don’t refresh automatically, and I have to manually reload them to see updated results. I’ve already checked the data connector and refresh schedule, but everything looks fine there. Could this delay be due to caching, or maybe a dashboard setting I’m missing? Would appreciate any suggestions or settings to look into — thanks in advance!  

                                             
      4
               
    • Blog banner
      • News & UpdatesChevronRightIcon

      Important update: connections management GA release & API behavior changes

                                                                       

      Important update: connections management GA release & API behavior changes Hello Sisense Community! I would like to inform you about an upcoming change to the Sisense Fusion platform that will affect how you work with Data Source Connections . As part of the L2024.3 Service Update 2, we are releasing the Connections Management feature in GA, enabled by default. This update aims to unify the management of data source connections, making Connections Management the single solution.  With the GA release, legacy connections are automatically disabled, and we have introduced temporary backward compatibility for certain endpoints that previously returned connection parameters data.  Starting in Q2 2025 (L2025.2), these temporary measures will be removed , fully deprecating legacy connections.  While we support full backward compatibility within the application, some custom script solutions that directly interact with our API endpoints may be impacted. To help you navigate this change, we have prepared the following resources:  API and System Behavior Changes with Connection Management , which will help you determine whether any of the listed endpoints are used in your custom solutions and guide you in making any required adjustments. If you would like to connect with us for support in making the required changes, please do not hesitate to reach out. Thank you for your understanding and continued partnership. Oleksandr Krokha

      Oleksandr_K
      Oleksandr_KPosted 1 year ago
      0
               
      • BloxChevronRightIcon

      BloX Write Back Form to a Source Database

                               

      BloX Write Back Form to a Source Database Sisense BloX Database Writeback This guide demonstrates setting up a simple BloX widget to enable front-end users to write data back to the database. While this guide specifically uses AWS lambda to execute a SQL update statement, the general flow can be adapted depending on your setup and needs. In this guide, we are looking at an example use case of a salesperson adding notes to an opportunity from Sisense. The complete source code can be found in this repository . See it in action Note: This is a live connection with a refresh rate of 5 seconds to demonstrate the writeback   How it works:  Quick Implement Follow these steps to create this BloX widget in your own environment: 1. Download the following files: BloxTemplate-DbWriteback.json BloxAction-DbWriteback.js 2. Create a widget type of BloX and upload a template Tutorials - See 'Exporting and Importing BloX Templates 3. Create a custom action in Blox, copy the contents of BloxAction-DbWriteback.js into this action, and make changes to your API. Overview The general flow that enables database writeback is as follows: BloX widget contains an input field and submit button. Submit button triggers a custom action, sending data from the input to a REST API. REST API executes the backend script to write data from the input field to the database. BloX The BloX widget is composed of two parts, the JSON editor script, and the JS action script. The JSON editor script defines the visual appearance of the widget, the input field and submit button, the data accessible to the action script, and when the action script is called. The JS action script controls the interaction between Sisense and the REST API. Here we define the URL of the API along with the data we are sending. Editor - JSON The key components in the BloX editor to focus on are the input container and the submit button. The code snippets below contain the relevant components. First, our input field is created with "type": "Input.Text" and we use "id": "data.note" to make the user input accessible from the back end. Note: Cloud Data Warehouses typically have APIs set up to accept SQL statements. These can be used in place of setting up your own REST API and backend service. Database security and schema are set up appropriately to accept update statements from the backend service. { "type": "Container", "items": [ { "spacing": "medium", "type": "TextBlock", "text": " ", "color": "black" }, { "type": "Input.Text", "id": "data.note", "placeholder": "Update project notes...", "isMultiline": true, "rows": "4", "borderRadius": "8px", "borderStyle": "none", "backgroundColor": "#F4F4F8" } ] }   Next, we use an ActionSet to create a submit button. We define which custom action is used with "type": "updateDb" - this indicates the custom action with the name updateDb will trigger when the button is clicked. Additionally, we have "data": {"id": "{panel:id}"} which makes the opportunity id accessible in our action.   { "type": "ActionSet", "actions": [ { "type": "updateDb", "title": "Send to DB", "data": { "id": "{panel:id}" } } ] }    Action - Javascript The Action defines the behavior that is performed after the button is clicked. The script below first calls a POST request and then redraws the widget to remove the note from the input box. For our post request to work, we need to add the URL of the API that will accept our POST request along with adding the data we are sending in the body. Here, we are sending the opportunity id along with the note the user entered in the input box.    url = 'rest-api-url' const insertOpportunityNote = () => { fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ 'id': `${payload.data.id}`, 'note': `${payload.data.note}` }) }) .then(response => response.json()) .then(response => console.log(JSON.stringify(response))) } insertOpportunityNote() payload.widget.redraw() // console.log(payload.data)   Most CDWs have APIs that could be leveraged here, such as Snowflake's SQL API . Custom APIs can also be used, which we will explore in the next section. Regardless, CORS must be set up to accept requests from your Sisense IP. API & Backend For this example, AWS Lambda and Amazon API Gateway were used in conjunction to create the REST API functionality that writes to our database. API Gateway After your Lambda function and API Gateway are connected ( Using Lambda with API Gateway ), double-check that CORS is handled correctly in the API. The following headers with correct values must be configured to allow traffic from your Sisense IP. Access-Control-Allow-Headers Access-Control-Allow-Methods Access-Control-Allow-Origin Lambda Our Lambda function uses the psycopg2 library to write to a postgres database. This requires three steps: establish the database connection, execute our insert statement, return a response. The lambda function accepts the event from the API gateway, extracts the data that was sent from Sisense, then creates our connection.    # Retrieve the necessary details to connect to the PostgreSQL database db_host = "your-db-url" db_port = "5432" db_name = "mydb" db_user = "postgres" db_password = "postgres" # Retrieve the details of the record to be inserted db_table = "opportunity_notes" db_col1_name = "opportunity_id" db_col1_value = event['id'] db_col2_name = "note_text" db_col2_value = event['note'] # Connect to the PostgreSQL database conn = psycopg2.connect( host=db_host, port=db_port, dbname=db_name, user=db_user, password=db_password ) cur = conn.cursor()   Next, we build our query, insert the values, and commit the changes. # insert query insert_query = f''' INSERT INTO {db_table} ({db_col1_name}, {db_col2_name})\ VALUES(%s, %s)''' # Values for the insert query insert_values = (db_col1_value, db_col2_value) # Execute the insert query cur.execute(insert_query, insert_values) # Commit the changes conn.commit()   Finally, we return a response. Note that the headers for CORS have been included here as well.   return { 'statusCode': 200, 'headers': { "Content-Type": "application/json", 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 'Access-Control-Allow-Origin': ' https://your-sisense-url.com ', 'Access-Control-Allow-Methods': 'OPTIONS,POST,GET' }, 'body': 'Record inserted successfully' }   Final Thoughts This example of database writeback is very basic. Ideas for enhancing this include: Enable BloX to post updates for multiple items Add the name or email of the Sisense user to their writeback request for visibility into who created the note Enhance security by adding authorization requirements to API Gateway Implementing standards in the database to indicate writeback tables

      alex-frisbie
      alex-frisbiePosted 2 years ago
      0
               
      • Data ModelsChevronRightIcon

      Choosing the Right Data Model

                               

      This post has become outdated. You can find guidance on choosing a data model on our documentation site  here . https://docs.sisense.com/main/SisenseLinux/choosing-the-right-data-model.htm   Introduction Customers often run into the question of which data model they should use (an ElastiCube, a Live model, or a Build-to-Destination). The following article presents some of the aspects you should consider when choosing between them. Sisense recommends that you discuss your needs and requirements with Sisense's technical team during the Jumpstart process, so the result will best meet your business expectations. Table of Contents Table of Contents This post has become outdated. You can find guidance on choosing a data model on our documentation sitehere. https://docs.sisense.com/main/SisenseLinux/choosing-the-right-data-model.htm Introduction Table of Contents Definitions The ElastiCube Data Model The Live Data Model Determining Factors Refresh Rate Data Transformation Options Operational Database Load Operational Database Availability Additional Vendor Costs Database Size Query Performance Connector Availability Caching Optimization Dashboard Design Limitations Definitions The ElastiCube Data Model Importing data into an ElastiCube data model allows the customer to pull data from multiple data sources on-demand or at a scheduled time, and create a single source of truth inside Sisense. The imported data can then be transformed and aggregated to meet your business needs. Once imported, the data snapshot is used to generate analytical information. The process of importing the data, known as a "Build", includes the following steps: Extract the data : Query the different data source(s) for data. Load the data : Write the data extracted to Sisense (the local MonetDB). Transform the data : Transform the local MonetDB (using SQL queries). To read more about ElastiCubes, see  Introducing ElastiCubes . The Live Data Model Using a Live data model does not require importing data. Only the data's schema needs to be defined. Once configured, analytical information required by the user is queried directly against the backend data source. To read more about Live models, see  Introducing Live Models . Determining Factors Refresh Rate One of the most fundamental aspects of determining your data model is your data's refresh rate. The data refresh rate refers to the age of the data in your dashboards: For Live models,  the data displayed on your dashboards is near-real-time, as every query is passed directly to the backend database. A good example of using a live model (due to refresh rate requirements) is a dashboard that shows stock prices. For ElastiCubes , the data displayed on your dashboard is current to the last successful build event. Every query is passed to the local database for execution. A good example of using an ElastiCube (due to refresh rate requirements) is a dashboard that shows historical stock prices. In this case, a daily ETL process will provide results that are good enough. To make a choice based on this factor, answer the following questions: How frequently do I need to pull new data from the database? Do all my widgets require the same data refresh frequency? How long does an entire ETL process take? Data Transformation Options The ETL process includes a "Transformation" phase. This transformation phase usually includes: Migrating the data tables into a dim-fact schema Enriching your data Pre-aggregating the data to meet your business needs The amount of data transformation on Sisense helps determine the suitable data model: For Live models,  Sisense allows minimal to no data transformation. Data is not imported before a query is issued from the front end. Therefore, data cannot be pre-conditioned or pre-aggregated. Most data sources used by Live models are data warehouses that may perform all data preparations themselves. For ElastiCubes , data is imported before a query is issued from the front end. Therefore, it may be pre-conditioned and pre-aggregated. A user may customize the data model to optimally answer their business questions. To make a choice based on this factor, answer the following questions: Is my data in a fact-dim schema? Does my data require enriching or pre-conditioning? Can my data be pre-aggregated? Operational Database Load Your operational databases do more than serve your analytical system. Any application loading the operational databases should be closely examined: For Live models,  Sisense will constantly query information from your operational databases, and feed it into your dashboard widgets. This occurs every time a user loads a dashboard. For ElastiCubes , Sisense highly stresses your operational databases during an ETL process while reading all tables. To make a choice based on this factor, answer the following questions: Does the analytical system stress my operational database(s)? Can the query load be avoided by using a "database replica"? Operational Database Availability Your operational database(s) availability is critical for collecting information for your analytical system. For Live models, all queries are redirected to your data sources. If the data source is not available, widgets will generate errors and not present any data. For ElastiCubes , data source availability is critical during the ETL process. If the data source is not available, the data in your widgets will always be available, but not necessarily be up to date. To make a choice based on this factor, answer the following questions: How frequently are analytical data sources offline? How critical is my analytical system? Is being offline (showing out-of-date information) acceptable? Additional Vendor Costs Various database vendors use a chargeback charging model, meaning that you will be charged by the amount of data you pull from the database or the computational power required to process your data.  For Live models,  every time a user loads a dashboard, each widget will trigger (at least) one database query. A combination of a chargeback charging model and a large user load may result in high costs. For ElastiCubes , every time the user triggers an ETL process, a large amount of data is queried from the database and loaded into Sisense. To make a choice based on this factor, answer the following questions: What is the number of users using my dashboards / What is my "build" frequency? Which data model will result in lower costs? What is the tipping point? Are you willing to pay more for real-time data? Database Size For ElastiCubes , please refer to these documents:  Introducing ElastiCubes   Minimum Requirements for Sisense in Linux Environments For Live models, there is no limitation as data is not imported to Sisense, only the data's schema. To make a choice based on this factor, answer the following questions: What is the amount of data I need in my data model? What is the amount of history I need to store? Can I reduce the amount of data (e.g., trimming historical data? reducing the number of columns? etc.) Query Performance Query performance depends on the underlying work required to fetch data and process it. Although every widget generates a query, the underlying data model will determine the work necessary to execute it. For ElastiCubes , every query is handled inside Sisense: The client-side widget sends a JAQL query to the Sisense analytical system. The query Is translated into SQL syntax, and run against an internal database. The query result is transformed back to JAQL syntax and returned to the client-side. For Live models, every query is forwarded to an external database and then processed internally: The client-side widget sends a JAQL query to the Sisense analytical system. The query Is translated into SQL syntax, and run against an external database. Sisense waits for the query to execute. Once returned, the query result is transformed back into JAQL syntax and returned to the client-side. To make a choice based on this factor, answer the following questions: How sensitive is the client to a delay in the query's result? When showing real-time data, is this extra latency acceptable? Connector Availability Sisense supports hundreds of data connectors (see  Data Connectors ). However, not all connectors are available for live data models. The reasoning behind this has to do with the connector's performance.  A "slow connector" or one that requires a significant amount of processing may lead to a bad user experience when using Live models (that is, widgets take a long time to load): For ElastiCubes , Sisense allows the user to utilize all the data connectors. For Live models, Sisense limits the number of data connectors to a few high-performing ones (including most data warehouses and high-performing databases). To make a choice based on this factor, answer the following questions: Does my data source's connector support both data model types? Should I consider moving my data to a different data source to allow live connectivity? Caching Optimization Sisense optimizes performance by caching query results. In other words, query results are stored in memory for easier retrieval, in case they are re-executed. This ability provides a great benefit and improves the end-user experience: For ElastiCubes , Sisense recycles (caches) query results. For Live models, Sisense performs minimal caching to make sure data is near real-time. (Note  that caching can be turned off upon request.) To make a choice based on this factor, answer the following questions: Do I want to leverage Sienese's query caching? How long do I want to cache data? Dashboard Design Limitations Specific formulas (such as Mode and Standard Deviation) and widget types (such as Box plots or Whisker plots) may result in "heavy" database queries: For Live models, Sisense limits the use of these functions and visualizations as the results of these formulas and visualizations may take a long time, causing a bad user experience. For ElastiCubes , Sisense allows the user to use them, as processing them is internal to Sisense. To make a choice based on this factor, answer the following questions: Do I need these functions and visualizations? Can I pre-aggregate the data and move these calculations to the data source instead of Sisense? See also  Choosing a Data Strategy for Embedded Self-Service .

      Ophir_Buchman
      Ophir_BuchmanPosted 4 years ago • Last reply 3 years ago
      1
               
      • TroubleshootingChevronRightIcon

      How To Troubleshoot Build Failures (Linux OS)

                       

      HOW TO TROUBLESHOOT BUILD FAILURES (Linux OS) Building an ElastiCube imports the data from the data source(s) that have been added. The data is stored on the Sisense instance, where future dashboard queries will be run against it. You must build an ElastiCube at least once before the ElastiCube data can be used in a dashboard. This article will help you understand and troubleshoot common build failures and covers the following: Steps of the Build Process Troubleshooting Tools Troubleshooting Techniques Common Errors & Resolutions Please note the following was written for Sisense on Linux as version  L2022.5.   Steps of the Build Process 1.  Initialization: The initialization stage of the build process prepares the platform for the data import, which includes checking and deploying the resources geared towards performing the build. 2.  Table Import: This step imports the data from the external data platforms into the ElastiCube. The ec-bld is created as a temporary, build-only pod – it lives only for the duration of a build, does all the heavy lifting of data ingestion and transformation, then gets shut down. The build pod, which uses the given connector framework, connects to the given source(s). By  default, 500,000 lines of data are read and imported per cycle during this phase.  The MServer is responsible for getting the data from the connectors and writing it to storage into Sisense’s database (MonetDB). While importing the data, the process uses the given query assigned to the data source (either the default Select All or a custom query). 3.  Custom Table Columns: This step of the build process runs the data enrichment logic defined in the ElastiCube modeling.  There are three types of custom elements:  Custom Columns (Expressions) Custom Tables (SQL) Custom Code Tables (Python-based Jupyter Notebooks) Custom Elements uses the data previously imported during the Base Tables phase as its input. The calculations/data transformation happens sequentially one after the other based on the Build Plan/Dependencies generated earlier in the process between finalizing the Initialization phase and at the starting of the Base Tables phase. Calculations occur locally based on the data in the ElastiCube, and can consume lots of CPU and RAM based on the complexity of the Expressions/SQL/Python Jupyter Notebooks. 4.  Finalization: These steps in the process finalize the ElastiCube’s build and readies it for use. The steps include: I. The current (up-to-date) data of the ElastiCube is written to the disk (local disk for single-node and shared disk for multi-node environment). II. The management pod stops the current ElastiCube running in Build Mode (ec-bld Pod, and its ReplicaSet + Deployment controllers). III. The management pod creates a new ElastiCube running in Query Mode (ec-qry Pod, and its ReplicaSet + Deployment controllers). IV. Once the new ElastiCube is ready, it becomes active and available to answer data queries (e.g., dashboard requests). V. The management pod stops the previous ElastiCube running in Query Mode (ec-qry Pod, and its ReplicaSet + Deployment controllers).   Builds may be impacted by several factors. It is recommended to test your build process and tune accordingly when changes are made to the following: Hardware Sisense architecture Middleware Data source Connectivity, networking, and security policies Sisense upgrade/migration from Windows to Linux Sisense configuration Increase in data volume Data model schema (i.e., number and complexity of custom tables and import queries) Troubleshooting Tools Leverage the following when troubleshooting build issues: Inspect log files Each log contains information related to a different part of the build process and can help identify the root cause of your build issue. Depending on your Sisense deployment, logs may be located in different directories. The default path for Single Node is /var/log/sisense/sisense . For Multi Node, exec into the the fluentd (<=2025.4) and fluent-bit-aggregator (>=2026.1) pod. If you need to collect logs, make sure to do so soon after the build failure, as logs will be trimmed after they reach a certain size. Log name Description Build.log = General build logs will contain information for all the Elasticubes. Query.log = General query logs will contain information for all the queries. Management.log = Elaborate log file, which contains service communication requests. (Build will reach out to Management to fetch info from MongoDB etc.) Connector.log = General information for all builds and connectors. build-con-<cube name>-<...>.log = More verbose logs provide connector-related details for specific builds. Translation.log = All the logs related to the translation service.  ec-<cube name>-bld-<...>.log = This contains the latest build log for each cube. It can also be viewed through the UI, as shown here. ec-<cube name>-qry-<...>.log = Contain logs related to specific Elasticubes’ queries. Combined.log = Aggregation of all logs in one file. It can be downloaded via the Sisense UI, as shown here . Please note if you are a Managed Services customer, only the combined log and latest build log for each cube are available. Use Grafana to check System Resources Grafana is a tool that comes packaged with Sisense that can be used to monitor system resources across pods. Every build has its own pod. This allows you to see the CPU and RAM that each build uses, as well as what is used by your whole Sisense instance. Excessive CPU and RAM usage is a common cause of build failures. 1. Go to Sisense Admin > Server & Hardware > System Management > Click on Monitoring. Click on the Search icon and then select All pods per namespace and then select namespace where Sisense is deployed (by default is “sisense”).   2. In the Pod dropdown, search for “bld” and select the cube you want to observe. *You may need to reduce the timeframe to get results:     3. Observe CPU and RAM over the duration of the build. *In the CPU graph, 1 core is represented by 100% See this article for additional information on using Grafana. Use Usage Analytics to observe build metrics Usage Analytics contains build data and pre-built dashboards to assist you in identifying build issues and build performance across cubes over time. See here for documentation on this feature. Ensure you have usage analytics turned on and configured to keep the desired history! Troubleshooting Techniques  Below are some common issues and suggestions for build errors. The first step is to read and understand the error message on the Sisense portal. This will help resolve the exact build issue. 1. Whenever you face build issues,  check the Memory consumption.  Options include either ssh to your Linux machine and run “kubectl -n sisense top pods --sort-by=memory” command to check the runnuing pods and its memory consumption, or you can also open grafana and check memory consumption in a graphical user-frendly UI. If you see high memory usage, then please try to schedule builds in the off hours to see if that helps.  2. If the cube is too big, try to break the cube into multiple cubes by sharing the data or separating use cases. 3. Check the data groups first to see if one specific cube is very large or if you only have a default data group. If all the cubes are part of that data group, then create a different group for the large cube.  4. If the error message is related to Safe Mode (“Your build was aborted due to abnormal memory consumption (safe mode)”), then check the Max RAM value set in the data groups . You can increase the Max RAM value and verify the build. ( https://support.sisense.com/kb/en/article/how-to-fix-safe-mode-on-build-dashboard )   See the following two articles for details on managing data groups: https://documentation.sisense.com/docs/creating-data-groups https://community.sisense.com/t5/knowledge/how-to-configure-data-groups/ta-p/2142 5. Running concurrent build processes can also be an issue. Try to not run multiple builds at the same time . If that is the issue, then open the Configuration Manager via Sisense Admin > Server & Hardware > System Management > Configuration > CLick 5x on the Sisense logo, expand the Build section, Change the value of Base Tables, Max Threads to 1 and save. (Relevant pod should restart automatically, but you can also restart the build Pod manually using “kubectl -n sisense delete pod -l app=build” 6.  Lack of sufficient storage space to create a new ElastiCube (either in Full or Accumulative build) can also result in build failure. It is recommended to free up some space and then check the build. 7. Check the log files and the query running in the backend to try to break down complex queries to avoid memory consumption. 8. The below items outline the configurations that affect troubleshooting: -Base Tables Max Threads : Limits the number of Base Tables that are built simultaneously in the SAME ElastiCube -Timeout for Base Table : Will probably “forcefully” fail the build if any Base Table takes more than this amount of time to build, available via the “Build” configuration        Remember that making any changes to these settings might require pod restart: To restart the pod, run the following command: kubectl -n sisense delete pod -l app=build Check the pod restarted based on the pod age: Kubectl -n sisense get pods -l app=build 9. If you have many custom tables, try to use the import query ( move the custom table query into the custom import query ). Documentation:  Importing Data with Custom Queries - Introduction to Data Sources  10. Please check your data model design and confirm that it conforms to Sisense best practices. For example, M2M takes more memory and can result in build failures.  https://support.sisense.com/kb/en/article/data-relationships-introduction-to-a-many-to-many 11. Builds can also fail because of the network connection between data sources and the Sisense server.  Perform a Telnet test to verify connectivity from the Sisense server to the data server. Common Build Errors and Resolutions Error: BE#468714 Management response : ElastiCube failed to start - failed to detect the EC; Failed to create ElastiCube for build process. Description: This means the process does not have enough resources to bring up the build pod. Resolution: If the Kubernetes process is still running for creating the Pod, the following command will allow you to monitor the given Build pods being brought up and check once they are healthy, up, and running.  Command: kubectl -n sisense get pods -l mode=build -w If the value for that pod in the restarts column is greater than 0, it means that the Pod is not able to be initialized properly and will retry 5 times until it fails and terminates the process. If the build process had already terminated in the past, view the container Reason Error, events and logs by running the following commands: kubectl -n sisense describe pod ec-<data_model_name>-bld-<pod_id> kubectl -n sisense logs ec-<data_model_name>-bld-<pod_id> Error: BE#196278 failed to get sessionId for dataSourceId Description: This error indicates that the user running the build does not have permission to run the build for the given ElastiCube.  Resolution: The ElastiCube needs to be shared with the user with “Model Edit” permission. Error: BE#470688 — Cumulative build failure (missing/corrupted farm directory) Description: The reason for this issue is a cumulative build is being performed, which relies on having the ElastiCube stored in the farm fails because either access to the directory in the farm storage location or directory/files are corrupted or are not there. Resolution: The only way to resolve it is to either restore the farm directory from a backup for the ElastiCube or re-build the ElastiCube with a full build. Error: BE#313753 Circular dependency detected Description: This happens when you have a lot of custom tables and custom columns which depend on each other Resolution: Please check the below articles on how to avoid loops:  https://documentation.sisense.com/docs/handling-relationship-cycles#gsc.tab=0 Error: Failed to read row:xxxxxxx, connector  Description: Sisense is importing data from the database using a Generic JDBC connector. Why did this fail suddenly? The data added recently is not in the correct format or as expected in the table. Resolution: If you are using a Generic JDBC connector, then it’s worth checking the connector errors online where you may find useful information to resolve the issue related to the connector. Error: BE#640720 Build failed: base table <table name> was not completed as expected. Failed to read row: 0, Connector (SQL compilation error: invalid number of result columns for set operator input branches, expected 46, got 45 in branch 2). Description: Most likely issue in the custom import query or on the target table. Resolution: Please check if there are right amount of columns used in the query and refresh table schema. Error: Build failed: BE#636134 Task not completed as expected: Table TABLE_NAME : copy_into_base_table build Error -6: Exception for table TABLE_COLUMN_NAME in column COLUMN_NAME at row ROW_NUMBER: count X is larger than capacity Y Resolution: This could be resolved by changing BaseTableMax (parallel table imports) from 4 to 1 in the Configuration Manager. Conclusion Understanding the exact error message is the first step towards resolution. Based on the symptom you can try some of the suggestions listed above and can quickly resolve build failure issues. If you need any additional help, please contact Sisense Support or create a  Support Case with all the log files listed above, and a Support Engineer will be able to assist you. Remember to include all relevant log files for an efficient troubleshooting process!   Krutika Lingarkar, Technical Account Manager in Customer Success, wrote this article in collaboration with Chad Solomon, Technical Account Manager, Senior in Customer Success, and Eran Ganot, Tech Enablement Lead in Field Engineering. 

      Krutika
      KrutikaPosted 3 years ago • Last reply 3 years ago
      15
               
      • Sisense AdministrationChevronRightIcon

      Reducing Windows Memory Pressure by Removing unused JVM Connectors

                               

      Reducing Windows Memory Pressure by Removing unused JVM Connectors A simple technique can reduce memory pressure on Windows Sisense versions by inactivating unused JM connectors.  To do this you should search for the Sisense JVM Connectors Configuration application from the Windows Start menu.     After opening the Sisense JVM Connectors Configuration application simply uncheck any connectors which are not in use and press the Save button.   The amount of memory saved varies depending on which connector.  On average each connector uses 2GB. So if you inactivate  5 connectors, you just saved 10 GB of memory. If you need additional help, please contact Sisense Support. PLEASE NOTE:   If you cannot locate the JVM as seen in the screenshot, the executable name is "usedConnectorsEditor.bat" and it should be located in the following directory C:\Program Files\Sisense\DataConnectors\JVMContainer\bin As seen in below picture.   

      Darwin
      DarwinPosted 3 years ago
      0
               
      • Data SourcesChevronRightIcon

      Google Analytics CDATA Connector

                       

      Google Analytics CDATA Connector   What You’ll Learn: In this course users will learn how to: Download a CDATA Google Analytics (GA) driver Install a CDATA GA driver Configure a CDATA GA driver How to connect using a CDATA GA driver   Prerequisites: This article will be helpful for users with basic knowledge of the following:  Familiarity with Sisense UI Familiarity with the concept of data connectors Basic knowledge of Linux/ Windows   Introduction: In May 2022, after Google’s April announcement deprecating an old API, Sisense announced that they will also be deprecating the native Google Analytics connector.  For existing customers, the native connector will continue to work until early August 2023. Despite these hurdles, users can still use CDATA drivers as a workaround in order to connect.    Instructions On How To Connect To Google Analytics Using A CDATA Driver: There are 2 ways to use a CDATA connector: Option 1, i nvolves using the CDATA connector's internal generic clientId and clientSecret. It takes less effort but is also less customizable. Option 2, requires creating an app in Google to generate credentials (clientId and clientSecret). Although this option is more customizable, it also takes more effort. Option 1: Less Customized, Less Effort: Download the driver from Sisense Connectors | Sisense Data Connectivity | CData Software . Install it locally in the customer’s environment. Navigate to (if installed in the default location): /Applications/CData/CData JDBC Driver for Google Analytics 2022/lib if using MacOS. C:\Program Files\CData\CData JDBC Driver for Google Analytics 2022\lib if using Windows. Execute cdata.jdbc.googleanalytics.jar Add the following parameters to make it work: AuthScheme=OAuth; OAuth=GETANDREFRESH; Resulting connection string looks as follows: jdbc:googleanalytics:AuthScheme=OAuth;InitiateOAuth=GETANDREFRESH Hit “Test Connection”. You will be redirected to the web browser to log in to your Google account. Once authenticated, the connection successful message will appear in the jar and the connection string will be updated. Click “Copy to Clipboard” to copy the connection string.   OAuthSettings.txt will be generated containing the connection parameters in the following location: %APPDATA%\\CData\\GoogleAnalytics Data Provider\\OAuthSettings.txt for Windows ~/Library/Application Support for macOS. If you can’t find the file, I recommend setting the OAuthSettingsLocation in the jar itself. Copy OAuthSettings.txt to the Sisense server: C:\OAuthSettings.txt for Windows deployments. /opt/sisense/storage/data/OAuthSettings.txt for Linux deployments. Copy cdata.jdbc.googleanalytics.jar to the Sisense server. You can check the following documentation for an exact step-by-step guide. For example: /opt/sisense/storage/data/GoogleAnalytics - for Linux C:\Program Files\Sisense\DataConnectors\JVMContainer\Connectors\CData JDBC Driver for Google Analytics 2022\lib - for Windows Connect to Google Analytics using Sisense: Use the connection string from step 7 but change the OAuthSettingsLocation: jdbc:googleanalytics:AuthScheme=OAuth;InitiateOAuth=GETANDREFRESH; OAuthSettingsLocation="/opt/sisense/storage/data/OAuthsettings.txt" ;_persist_oauthrefreshtoken=oauthRefreshToken;_persist_oauthexpiresin=3599;_persist_oauthaccesstoken=oauthAccessToken;_persist_oauthtokentimestamp=XXXXXXXXXXXXX; Jar’s folder (the one you copied the jar file into in step 10): /opt/sisense/storage/data/GoogleAnalytics C:\Program Files\Sisense\DataConnectors\JVMContainer\Connectors\CData JDBC Driver for Google Analytics 2022\lib Driver class name: cdata.jdbc.googleanalytics.GoogleAnalyticsDriver     Option 2: More Customized, More Effort: Download the driver from Sisense Connectors | Sisense Data Connectivity | CData Software page. Install it locally in the customer’s environment. Navigate to (if installed in the default location): /Applications/CData/CData JDBC Driver for Google Analytics 2022/lib if using MacOS. C:\Program Files\CData\CData JDBC Driver for Google Analytics 2022\lib if using Windows. Execute cdata.jdbc.googleanalytics.jar You will need to add the following parameters to make it work: AuthScheme=OAuth; OAuth=GETANDREFRESH; OAuthClientId=clientId; NOTE: If the customer is new, use this documentation to create a google app. For customers who already use our native Google Analytics driver, you will need to specify the clientId which can be retrieved from manifest.json from the old connector. Possible locations: /opt/sisense/storage/connectors/googleanalytics/manifest.json C:\Program Files\Sisense\DataConnectors\DotNetContainer\Connectors\GoogleAnalytics\manifest.json C:\Program Files\Sisense\DataConnectors\JVMContainer\Connectors\GoogleAnalytics\manifest.json OAuthClientSecret=clientSecret; NOTE: The same as OAuthClientId retrieved from manifest.json. The resulting connection string should look like the following: jdbc:googleanalytics:AuthScheme=OAuth;InitiateOAuth=GETANDREFRESH;OAuthClientId=clientId;OAuthClientSecret=clientSecret;   6. Before hitting “Test connection” in the jar file you will need to navigate to your Google application using Google API Console and add “ http://localhost:33333 ” to the ‘Authorized redirect URIs’ field. 7. Hit “Test Connection”. You will be redirected to the web browser to log in to your Google account. 8. Once authenticated the connection successful message will appear in the jar. 9. The connection string will be updated to incorporate tokens obtained from Google. Click “Copy to Clipboard” to copy the connection string. You will need it to set up the connection in Sisense: 10. OAuthSettings.txt will be generated containing the connection parameters in the following location: %APPDATA%\\CData\\GoogleAnalytics Data Provider\\OAuthSettings.txt for Windows ~/Library/Application Support for MacOS If you can’t find the file, I recommend setting the OAuthSettingsLocation in the jar itself. 11. Copy OAuthSettings.txt to the Sisense server: C:\OAuthSettings.txt for Windows deployments. /opt/sisense/storage/data/OAuthSettings.txt for Linux deployments. 12. Copy cdata.jdbc.googleanalytics.jar to the Sisense server. You can check the following documentation for an exact step-by-step guide. For example: /opt/sisense/storage/data/GoogleAnalytics - for Linux C:\Program Files\Sisense\DataConnectors\JVMContainer\Connectors\CData JDBC Driver for Google Analytics 2022\lib - for Windows 13. Connect to Google Analytics using Sisense: Use the connection string from step 9, but change the OAuthSettingsLocation: jdbc:googleanalytics:AuthScheme=OAuth;InitiateOAuth=GETANDREFRESH;OAuthClientId=clientId;OAuthClientSecret=clientSecret; OAuthSettingsLocation="/opt/sisense/storage/data/OAuthsettings.txt" ;_persist_oauthrefreshtoken=oauthRefreshToken;_persist_oauthexpiresin=3599;_persist_oauthaccesstoken=oauthAccessToken;_persist_oauthtokentimestamp=XXXXXXXXXXXXX; Jar’s folder (the one you copied the jar file into in the step 12): /opt/sisense/storage/data/GoogleAnalytics C:\Program Files\Sisense\DataConnectors\JVMContainer\Connectors\CData JDBC Driver for Google Analytics 2022\lib Driver class name: cdata.jdbc.googleanalytics.GoogleAnalyticsDriver   Please keep in mind that the CDATA table/column structure is different from Sisense’s native drivers which means that you might need to redesign your data models. To learn about the CDATA table/column structure and the nuances of using the connector, I highly recommend checking out the following CDATA documentation: CData JDBC Driver for Google Analytics - CData JDBC Driver for Google Analytics   Conclusion: If you were worried about Sisense’s announcement deprecating the native Google Analytics connector, don’t worry! This practical article shows users how to use the CDATA driver to connect to and keep using Google Analytics as a data source. 

      antonvolov
      antonvolovPosted 3 years ago
      0
               
      • Sisense AdministrationChevronRightIcon

      Determine Driver's Class Name for JDBC Connector

                       

      Determine Driver's Class Name for JDBC Connector Question How do you determine the Driver’s Class Name from the JDBC driver file that is installed with the Sisense JDBC Connector?   Prerequisites: Sisense Data Administrator may require a separate JDBC Connector to connect to a specific data source. https://www.sisense.com/data-connectors/ https://www.cdata.com/solutions/bi/sisense/ A download of the specific JDBC driver is required. https://docs.sisense.com/main/SisenseLinux/connecting-to-custom-connectors-with-jdbc-drivers.htm   Relevant Background Information: As per Sisense documentation: https://documentation.sisense.com/docs/connecting-to-dynamodb .  Steps are described in this document for installing a DynamoDB JDBC driver:  Reference in the document section: Adding DynamoDB Tables to your ElastiCube (Step 7) The  document provides the Driver’s Class Name to use.     cdata.jdbc.amazondynamodb.AmazonDynamoDBDriver The document provides the Driver’s Class Name but does not contain details on how to obtain it. Answer 1. Let's use for example the SalesForce Marketing Cloud JDBC Driver https://www.cdata.com/drivers/salesforcemarketing/jdbc/ Open the JDBC driver file as an archive file.   Filename: cdata.jdbc.sfmarketingcloud.jar Note :  You can use any other application like Winzip, 7zip, or Winrar, to extract the jar file contents.   2. Navigate to the META-INF/services/ subdirectory within the archive.   3. Extract the file named java.sql.Driver to view.    4. Open java.sql.Driver in a text editor. Driver’s Class Name:  cdata.jdbc.sfmarketingcloud.SFMarketingCloudDriver   Additional Notes: These steps can be applied to installing any JDBC driver that can be supported with Sisense. If you need additional help, please contact  Sisense Support  or create a   Support Case .  Document References: https://www.sisense.com/data-connectors/ https://www.cdata.com/solutions/bi/sisense/ https://docs.sisense.com/main/SisenseLinux/connecting-to-custom-connectors-with-jdbc-drivers.htm https://documentation.sisense.com/docs/copying-a-cdata-jar-file-installed-locally-to-a-remote-server https://www.cdata.com/drivers/salesforcemarketing/jdbc/  

      KevinY
      KevinYPosted 3 years ago
      0