Build Stability Improvements on Heavy Loaded Systems
Symptoms Relevant for Linux Below are a couple of select symptoms your system could be experiencing under a big load with simultaneous builds. Please note the list is not exhaustive: Build failures over random cubes or a couple of cubes that are being rebuilt manually successfully CubeIsUnreachable error Failure due to Build Service restart Diagnosis Build flow is connected to three main services: Build: Takes care of triggering builds and moving logs between the Pod and UI Management: Creates ec-bld pod and takes care of Kubernetes level communication over the flow Ec: <name_of_the_cube>-bld - actual cube import process which creates the folder, makes import If memory consumption of the ec-bld pod is being controlled by the DataGroup Max RAM for Build (more in-depth article here) and affects just this cube, then Build/Management works for all builds running on the system. Sometimes when there are 4+ cubes that run in parallel, Build/Management services could be under heavy load and require additional RAM. Since both of the services are Java based they have a default memory limit mechanism that allows 500 MB of RAM to be used. If the service needs more, it could be throttling which can affect defined timeouts or cause service restart. If you want to ensure that this is the case, please check graphana for build/management services when builds fail. Linked is additional information on how to use Grafana to troubleshoot performance issues. Keep in mind that by default services are limited to 500 MB but could consume more since there are additional parts of the service that are not Java-based and Java limits can peak from time to time. However, if you see that the service is under load, it is a good idea to allocate more RAM for stability improvements. Solution Increase the Memory Limits for the Build and Management services. Since this is a Java service there are two places to update: Kubernetes deployment limits Java service on the Sisense Configuration side To update Memory Limits please follow the steps below: 1. SSH to the server 2. Execute line 1 for build or line 2 for management: kubectl edit deployment -n <namespace> build kubectl edit deployment -n <namespace> management 3. Find "resources: limits" and update memory size to 4000 (press “i” on the keyboard to enter edit mode → edit value → ESC to exit edit mode → use “:wq!” to exit and save changes). 4. Build/management pod will automatically restart after deployment modification. 5. Navigate to Admin → System management → Configuration → Build → Memory limit for build pod and Navigate to Admin → System management → Configuration → Advanced Management params → Memory limit for management pod. Set the value to be 1000 lower than the value set in step #3 for the deployment. 6. Save settings. Tips: Please ensure that the Memory Limit value is correctly entered. If there is a problem with the value, the service will not start. Please keep in mind that the Deployment value should be 1000m bigger than the Configuration value. If you need any additional help, please contact Sisense Support.Connection Tool - Programmatically Remove Unused Datasource Connections, and List All Connections
Managing connections within your Sisense environment can become complex over time, if there are a large number of connections, and connections are often added, and replace earlier datasource connections. In some scenarios unused connections can accumulate, potentially cluttering the connection manager UI with no longer relevant connections. Although unused connections typically represent minimal direct security risk, it's considered best practice to maintain a clean, organized list of connections, and in some scenarios it can be desired to remove all unused connections. Sisense prevents the deletion of connections actively used in datasources, safeguarding your dashboards and datasources from disruptions. However, inactive or "orphaned" connections remain after datasources are deleted or a connection is replaced, potentially contributing to unnecessary UI complexity in the connection manager UI. Connections can be of any type Sisense supports, common types include various SQL connections, Excel files, and CSV files, as well as many data providers, such as Big Panda. This tool can also be used to list all connections, with no automatic deletion of unused connections.How to Implement Linkable Text in a Widget (Linux)
How to Implement Linkable Text in a Widget (Linux) Introduction This article provides a guide on implementing linkable text to navigate to another dashboard in Sisense. The goal is to enable users to access a linked dashboard by clicking on a text element rather than interacting with a widget. This serves as a workaround for the default functionality, which typically requires a widget click (using Jump to Dashboard with native widgets). Step-by-Step Guide Using Text Widget: Use the direct link to the desired dashboard. This method allows users to be redirected directly to the dashboard upon clicking the text link. For example, you can add a hyperlink to the Text Widget. Prefer refer to this guide: Text Widget Documentation. Using BloX: Utilize the BloX widget to create a button that simulates the appearance and position of text within your design using the Action Button feature. This approach enables users to open the dashboard by clicking the button. Please follow this guide for reference: Creating Action Buttons. If you need to apply filters, configure the BloX widget to perform the Jump to Dashboard (JTD) action. For further instructions on setting up BloX with JTD, refer to this community article on using Jump to Dashboard in a BloX widget. Conclusion Implementing linkable text in a widget provides a seamless navigation experience in Sisense, bypassing the need for widget interactions. Whether using a Text Widget for direct links or leveraging BloX for more customization, these methods allow users to efficiently jump between dashboards. References/ Related Content How to Use Jump to Dashboard in a BloX Widget Creating Action Buttons Text Widget Documentation Jump to DashboardSisense application logs
Sisense application logs The Sisense Support team typically uses the application log data to investigate outages, troubleshoot bugs, or analyze reported issues. Analyzing application logs can help the Support team determine the root cause of incidents. The location of log directories on a single node: /var/log/sisense/namespace/ /var/log/sisense/sisense/query.log /var/log/sisense/sisense/combined.log -- logs of all services If your environment does not have this setup, here is how you can gather the appropriate logs to share For Example, if interested to Access galaxy logs: kubectl -n sisense logs $(kubectl -n sisense get pods -l app="galaxy" -o custom-columns=":.metadata.name" ) Tail the log and print the last 100 lines: kubectl -n sisense -f --tail=100 logs $(kubectl -n sisense get pods -l app="galaxy" -o custom-columns=":.metadata.name" ) Get logs from specific pod: sisense@node1: kubectl logs $(PODNAME) -n $(NAMESPACE) Helpful Tips: In all Sisense installs, it would be very helpful to generate the Support File in the Admin tab under 'System Management'. When the issue occurs, you can download the file; this application log is a file that contains information about events that have occurred within a software application. This file contains logs of all services. Additional Resources: Guideline for submitting a Support inquiry Overview to Interpreting Sisense 7.2+ LogsUserReplaceTool - Automating Dashboard Ownership Transfers - Useful for Deleting User Accounts
Managing and deleting user accounts in Sisense can create manual processes when users leave an organization or change roles. A frequent issue is the reassignment of dashboard ownership to prevent losing Sisense dashboards when a given user account is deleted, as deleting a Sisense user will delete all dashboards owned by that user. The UserReplaceTool addresses this task by automating the transfer of dashboard ownership of all dashboards owned by a given user, ensuring continuity and data integrity. UserReplaceTool is a Python-based, API-based Tool solution designed to seamlessly transfer the ownership of dashboards and data models from one user to another in Sisense. This tool simplifies and automates this process, allowing organizations to reassign dashboard ownership without manual processes or the risk of losing dashboards and widgets. All components are accomplished by using Sisense API endpoint requests.How to properly reboot a k8s node(s)
As part of maintaining a healthy and robust Kubernetes (K8s) cluster, occasional reboots of nodes might be necessary. Whether for system updates, hardware maintenance, or other reasons, it's essential to follow a structured process to ensure minimal disruption to running workloads. Below is a step-by-step guide on safely rebooting nodes within a Kubernetes cluster, covering both Red Hat Enterprise Linux (RHEL) and Ubuntu systems.Update and add new Highcharts modules for use in Sisense plugins
Update and add new Highcharts modules for use in Sisense plugins The JavaScript library framework Highcharts is natively included in Sisense and is utilized in many native Sisense widgets as well as in numerous Sisense plugins. Although Sisense typically does not alter the Sisense Highcharts library version with every release, the versions of Highcharts included in Sisense may change when upgrading to a new major version release. Highcharts can load additional chart types and other types of functionality via JS module files that contain code-adding features such as additional chart types, which can be used within plugins along with additional code to create additional widget types. If a plugin utilizes a Highcharts module, you can source the module directly in the "plugin.json" file's source parameter, as shown in this example: "source": [ "HighchartModule.js", ], To determine the current Highcharts version being used in your Sisense version, you can use the "Highcharts" command in the web console while viewing any page on your Sisense server. After identifying the current Highcharts version, you can find the corresponding module hosted on the Highcharts code hosting website using the following URL format: https://code.highcharts.com/${Highcharts_Version}/modules/${module_name}.js For example: https://code.highcharts.com/6.0.4/modules/heatmap.js You can save this module and upload it to the plugin folder or replace the older module JS file simply by copying and pasting the code directly. Be sure to update the "plugin.json" file to point to the new module file if the file name has changed or if this is the first time the module is included. Simply sourcing the module file in the "plugin.json" file is sufficient to load the module into Highcharts; no further code is required to load the module.
EKS Kubernetes Version Upgrade
EKS Kubernetes Version Upgrade Below are steps on how to upgrade the Kubernetes version in AWS Kubernetes Service (EKS). First, ensure that the current version is compatible with the K8S version you will upgrade to. If not, upgrade the Sisense version compatible with the K8S version first! It is also strongly recommended to have the cloud_autoscaler feature enabled in your configuration yaml. Upgrade the control plane of the cluster, open the EKS page, and find your cluster. You should see the 'Upgrade now' link if a newer major K8S version is available. Click on that link and select the version (Please note, usually, only one version is available because you cannot jump over versions). Click the Update button, and you should see a similar information bar as the one below: The cluster is being updated: As the control plane upgrade is being completed, the cluster version should also update as well: If needed, the kubectl version (client binary file) should be updated manually. It should be one value more or less than the server version, e.g., for the cluster with EKS 1.23, you can use kubectl 1.22, 1.23, and 1.24. Please notice that the nodes are still running the previous Kubernetes version. Open the cluster page and go to the Compute tab. You should see the upgrade options for NodeGroups. Click on update for every node group and select "Force Update." The new nodes should appear while the old nodes are cordoned. Eventually, you should see only new nodes running: If you had the cloud_autoscaled feature disabled, run the installer with the new node names.
