Sisense Configuration Tips
Sisense Configuration Tips What You’ll Learn In this course you will learn how to: Export Sisense configuration Search for settings in the configuration Change the configuration settings Compare different configurations Import configuration Use Cases For Applying What You Will Learn The following are some scenarios where this content can be applied when working with Sisense configurations: You see the operation in the logs hitting the timeout equaling 300000, but you are not sure if there is a setting for such a timeout in the configuration. You are performing an upgrade and want to check if the upgrade affected some of your configurations. You want to back up the configuration. You want to compare the configurations of two different environments. Prerequisites Familiarity with Sisense UI Basic knowledge of Linux Basic knowledge of Kubernetes Configuration Types In Sisense Sisense configuration settings may be logically divided into a couple of main categories: Fixed: Those that are ‘fixed’ for the current deployment like the ones you set during Sisense's installation. Configurable: Those that can be changed using the Configuration UI or the SI Client commands, without having to reinstall or update the Sisense application. Add-ons: Those that are not directly related to the core application, but apply to add-ons like plugins or connectors In this article, we will focus on Configurable settings that can be accessed from the Admin tab > System Management > Configuration To get access to the full configuration menu, click on the Sisense logo 5 times: Services Responsible For Configuration Management Two services in Sisense are responsible for configuration management: Zookeeper stores the configuration settings. Configuration is responsible for manipulating configuration parameters and providing the configuration data to other services using ConfigurationSDK. Configurations are set when the configuration service starts. The service sets all of the base (shared) configurations under the base configurations path in Zookeeper. All other services copy their required configurations during boot time using the configuration manifest file each service has stored using the Sisense-configuration SDK. Base Vs Specific Configuration The base configuration is shared between services. For example, if you change the log level in the base configuration it will change for all the services. Specific configuration settings for services affect those services only. You should be mindful of the Base and specific configuration when setting configuration parameters using tools outside of the UI! Hidden UI Configuration “File Explorer” As we already learned, we cannot change the base configuration parameters for specific services using the standard UI configuration menu. But what if we want to change the log level just for the Galaxy service? The hidden UI configuration, “file explorer” will come in handy. It will display all available Zookeeper configuration parameters. To access the hidden configuration “file explorer” click on the General section header 5 times. The menu will expand and then, you will be able to see the File Explorer sub-menu Find the galaxy service folder in the file tree. From there you will be able to see the base configuration settings for galaxy services: Please note, if you change the base configuration setting, it will reset the individual value. Changing Configuration Settings There are 4 ways to change configuration settings: From the UI regular menu Admin > System Management > Configuration From the UI hidden file explorer (see above) From inside the Zookeeper pod using zkCli.sh Using SI CLI commands Using the UI and hidden file explorer is quite straightforward, so below is a review on how to use zkCli.sh and SI CLI commands. From Inside The Zookeeper Pod Using Zkcli.Sh Exec into Zookeeper: kubectl -n sisense exec -it $(kubectl -n sisense get po -l=app=zookeeper -o name) -- bash Run zkCli.sh You will see the prompt: help — to see the help with all commands ls — to list configurations, Sisense configurations are located at ls /Sisense/S1/configuration/production/ get — to get the particular value for example get /Sisense/S1/configuration/production/translation-service/SwitchOffSameGuids set — to set the particular value in the format, set the parameter value. For example, set /Sisense/S1/configuration/production/translation-service/SwitchOffSameGuids true Using SI CLI commands si configuration is responsible for manipulating the configuration settings: get — to get the particular value in the format of si configuration get -key category.entry -type key set — to set the particular value in the format of si configuration set -key category.entry -new-value 1412 Read more about SI commands in Sisense’s documentation site: Using Sisense CLI Commands. Exporting Configuration Settings: Say you see in the logs that the request is hitting the 300000 msec timeout threshold, but you don’t know where the timeout is set and if there is such a timeout setting at all. It would be great to search the entire configuration set for the value 300000 and see if anything comes out, but the UI doesn’t provide that functionality yet, since you can only search per service. Please note, that at this time you cannot search through all configuration settings via the comprehensive configuration menu nor in the file explorer. To perform a global search in all configuration settings, we can export the configurations and perform the search in the output file. Remember that configurations are stored in Zookeeper while the configuration service is responsible for interacting with the Zookeeper database. Therefore, we will use configuration commands to perform the export. 1. Run this command to get the Zookeeper service ClusterIP: kubectl get svc -A | grep zookeeper. 2. Put the IP in this command (but don’t run it yet!). node /usr/src/app/node_modules/@sisense/sisense-configuration/bin/sisense-conf export -p -c 10.43.198.82:2181 3. Then exec into the configuration pod. kubectl -n sisense get po | grep configuration kubectl exec -it -n sisense configuration-84777bdfd9-n2wn5 -- /bin/bash 4. Then run the above command (with the correct cluster IP). It will dump the output into sisenseConfiguration.json, then run exit. 5. Copy sisenseConfiguration.json out of the pod, to the host, so you could work with the file kubectl -n sisense cp configuration-84777bdfd9-n2wn5:/usr/src/app/sisenseConfiguration.json sisenseConfiguration.json Or simply run this command from the host, which will do everything for you. kubectl -n sisense exec -it $(kubectl -n sisense get po -l=app=configuration -o name) -- node /usr/src/app/node_modules/@sisense/sisense-configuration/bin/sisense-conf export -p -c sisense-zookeeper:2181 > sisenseConfiguration.json It will put sisenseConfiguration.json into the folder you are running it from. Now you can search for the 300000 value in the entire configuration. To perform an extensive search, you can use your favorite editor like vim or nano to open the file or use the Linux command “less”. Importing Configuration Settings Copy sisenseConfiguration.json to the new environment and to the configuration pod. Then exec into the configuration pod on the new environment using the following command: kubectl exec -it -n sisense configuration-84777bdfd9-n2wn5 -- /bin/bash and run: node /usr/src/app/node_modules/@sisense/sisense-configuration/bin/sisense-conf import -p -c sisense-zookeeper:2181 -i sisenseConfiguration.json Comparing Configurations Of Different Environments Say you want to compare configurations of two different environments or compare configurations before and after the upgrade to make sure nothing has changed. Now it is easy to do! You can export configurations into files and compare files using diff or your favorite online text-comparing tool like https://text-compare.com/ Summary I hope this article helps you perform searches in Sisense’s configuration, compare configurations and check if a configuration setting exists at all in Sisense. Using this article, we hope to empower Sisense users and show that users are not limited by UI limitations when working with configurations.
Leveraging Usage Analytics for maintaining a healthy environment
Analytical Need In this article, we can find a suggested approach on how to better maintain our Sisense environment. The current tools and methodologies take us to an encumbering and challenging work. By creating a dedicated dashboard to show our environment status and add actionable widgets we can make such work much easier. The final output of this article is a dashboard which will present our environment status (e.g. unused dashboards/ElastiCubes/Users and low/high usage components). + actionable drill-down dashboards (powered by BloX) which make these dashboards much more actionable. Solution Effort estimation: ~1 hour Technical Skills required: Low Usage Analytics Usage Analytics is part of this solution. Extending the default number of days stored (default is to keep only 30 days of usage) is optional but will provide a clearer status of your environment. Enable Sisense Usage Analytics - Follow this article to enable the Usage Analytics. https://docs.sisense.com/main/SisenseLinux/enabling-usage-analytics.htm Optional: Extending the number of days stored In the Sisense configuration manager, under the "Services" tab, locate "Usage service" and update the usage.csvRotateDays to the desired period. https://docs.sisense.com/main/SisenseLinux/configuration-manager-sisense-documentation.htm *Warning! Increasing the number of days to a higher number than 30 days might affect server performance. Mainly during the usage Analytics cube build. It is strongly recommended to test the cube stability, capacity and performance before scheduling it to a frequent build. Get unused Dashboards/ElastiCubes/Users data The Usage Analytics presents information on Sisense objects that had at list one usage. In case we had no usage by a user it won't appear in the Usage Analytics data. This is the reason why we need to call some metadata API's to fetch the missing information. The process per each API call mentioned below is as follows: Make sure you are logged in as an Admin user Copy the API URL, change if required to the server location, and Paste the URL in the HTML address/Postman/etc. (It includes the required fields for this solution) Convert the JSON to a CSV format. I Recommend this online converter: https://toolslick.com/conversion/data/json-to-csv Download the CSV output and rename Users API: http://localhost:8081/api/v1/users?fields=_id%2CuserName%2Cemail%2Ccreated%2ClastLogin Dashboards API Call (for a user which has access to all dashboards) http://localhost:8081/api/v1/dashboards?fields=oid%2Ctitle%2Ccreated%2ClastOpened%2ClastUpdated%2Cdatasource.title%2ClastPublish%2CparentFolder%2Cowner Or Dashboards API Call (Admin) https://localhost:8081/api/v1/dashboards/admin?dashboardType=owner&asObject=false&fields=title%2Ccreated%2ClastOpened%2ClastUpdated%2ClastPublish%2Cdatasource.title%2CparentFolder%2Cowner%2CwidgetCount%2ClastRepublished ElastiCubes API: http://localhost:8081/api/v1/elasticubes/getElasticubes Usage Analytics Re-Design to support Unused objects data The Usage Analytics ElastiCube is not a standard cube and the internal queries are being rewritten on each build. The only way to add additional data is by adding new entities/dimensions. This is the reason why the cube structure is not a classic star schema. Steps to adjust the Usage Analytics ElastiCube for the suggested solution: Download the Usage Analytics cube file, import and replace the existing Usage Analytics model Ecube file Smodel file Adjust the file data sources to the CSV sources created in the previous step In case CSV table names were changed, adjust the custom sql based tables (DimDashboards Extended, DimElasticubes, and Dim Users Extended) Build! Healthy Environment Dashboards Check that the OOTB Usage Analytics dashboards work properly. They shouldn't be affected by the adjustments we've made. Import attached dashboards Dashboard 1 Drill1 Drill2 Enjoy!
Reducing Windows Memory Pressure by Removing unused JVM Connectors
Reducing Windows Memory Pressure by Removing unused JVM Connectors A simple technique can reduce memory pressure on Windows Sisense versions by inactivating unused JM connectors. To do this you should search for the Sisense JVM Connectors Configuration application from the Windows Start menu. After opening the Sisense JVM Connectors Configuration application simply uncheck any connectors which are not in use and press the Save button. The amount of memory saved varies depending on which connector. On average each connector uses 2GB. So if you inactivate 5 connectors, you just saved 10 GB of memory. If you need additional help, please contact Sisense Support. PLEASE NOTE: If you cannot locate the JVM as seen in the screenshot, the executable name is "usedConnectorsEditor.bat" and it should be located in the following directory C:\Program Files\Sisense\DataConnectors\JVMContainer\bin As seen in below picture.Reverse Proxy with Nginx + SSL configuration
Reverse Proxy with Nginx + SSL configuration Nginx Reverse proxy configuration Step 1. Nginx reverse proxy server set up In this example, we are using nginx, we can install it on the same device as Sisense. To install it run 1. Install nginx for Ubuntu/Debian-like systems: sudo apt install nginx 2. For RHEL systems such a CentOS, use below: sudo yum install nginx 3. Start nginx: sudo systemctl start nginx Step 2. Nginx server configuration 1. Open the browser and go to the IP address of the server. If it's up, you will see the Nginx welcome page– this means nginx is now running on the default port 80. 2. Edit /etc/nginx/sites-enabled/default and add the next configuration under the root server config. Define correct Sisense public IP, and port in the "server {}" section: location /analytics { rewrite /analytics/(.*) /$1 break; proxy_pass http://<sisense-ip>:30845; proxy_http_version 1.1; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $host; proxy_connect_timeout 36000; proxy_send_timeout 36000; proxy_read_timeout 36000; send_timeout 36000; } 3. Before you apply the settings, check that there is no syntax issue by running sudo nginx -t 4. Reload nginx with sudo /etc/init.d/nginx reload or sudo systemctl reload nginx With this configuration, Sisense will be accessed with http://<ip-or-domain-of-nginx-server>/analytics. Also if the https is configured for this nginx server, Sisense would be accessible with https://<ip-or-domain-of-nginx-server>/analytics. If on the proxy level, the HTTPS is enabled, please ensure the application_dns_name has the https prefix to ensure all traffic is used, so something like: application_dns_name: https://company.sisense.com Step 3. Sisense configuration Go to the Admin tab Click on System Management Enter Configuration and choose Web Server In the Proxy URL enter "/analytics" or "http://<ip-or-domain-of-nginx-server>/analytics" as we configured in Nginx. With "/analytics" you will be able to use multiple domains for this instance. Save it and test with a browser by entering http://<ip-or-domain-of-nginx-server>/analytics And now we can configure SSL with our Nginx server, please validate that Nginx is working properly first before moving on. SSL configuration for Nginx Step 1. Obtain self signed SSL certificates You can use a command like this sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt. For an explanation of what the above command does please refer to Setup SSL on Sisense (Linux version) - Link placeholder Step 2. Configure Nginx to use SSL 1. Сreate a new file named self-signed.conf. sudo vi /etc/nginx/snippets/self-signed.conf In self-signed.conf we want to add some variables that will hold the location of our certificate and key files that we generated in Step 1. Like this ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt; ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; Save and close the file. 2. Now we will create a snippet file to define SSL settings. Start by creating a file like this sudo vi /etc/nginx/snippets/ssl-params.conf In this file, we need to include some SSL settings as below. ssl_protocols TLSv1.3; ssl_prefer_server_ciphers on; ssl_dhparam /etc/nginx/dhparam.pem; ssl_ciphers EECDH+AESGCM:EDH+AESGCM; ssl_ecdh_curve secp384r1; ssl_session_timeout 10m; ssl_session_cache shared:SSL:10m; ssl_session_tickets off; ssl_stapling on; ssl_stapling_verify on; resolver 8.8.8.8 8.8.4.4 valid=300s; resolver_timeout 5s; # Disable strict transport security for now. You can uncomment the following # line if you understand the implications. #add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"; add_header X-Frame-Options DENY; add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; Save and close the file. 3. In this step, we need to modify the Nginx configuration to use SSL. Open up your Nginx configuration file which is usually in a location like /etc/nginx/sites-available/<yourconfig>. Before making changes to this file it is best to back it up first in case we break anything. sudo cp /etc/nginx/sites-available/yourconfig /etc/nginx/sites-available/yourconfig.bak And now we open up our current Nginx config file; vi /etc/nginx/sites-available/<yourconfig> In the first server{} block, at the beginning, add the lines below. You might already have a location {} block so leave that there server { listen 443 ssl; listen [::]:443 ssl; include snippets/self-signed.conf; include snippets/ssl-params.conf; server_name your_domain.com www.your_domain.com; //server_name can be anything location / { try_files $uri $uri/ =404; } } Lastly, we need to add another server{} block at the very bottom of the file, with the following parameters. This is a configuration that listens on port 80 and performs the redirect to HTTPS. server { listen 80; listen [::]:80; server_name default.local www.default.local; //use same name return 302 https://$server_name$request_uri; } Please note that you must add this server_name to your local desktop or laptop hosts file. In this example, I will go to my local laptop or desktop hosts file and add <ip address of nginx server> <space> <default.local> [Optional] Step 3. Adjust the firewall The steps below assume you have a UFW firewall enabled. You need to review available profiles by running sudo ufw app list You can check the current setting by typing sudo ufw status: Output Status: active To Action From -- ------ ---- Nginx HTTP DENY Anywhere Nginx HTTP (v6) DENY Anywhere (v6) We need to allow HTTPS traffic, so update permissions for the “Nginx Full” profile. sudo ufw allow 'Nginx Full' Check the update sudo ufw status Output Status: active To Action From -- ------ ---- Nginx Full ALLOW Anywhere Nginx Full (v6) ALLOW Anywhere (v6) This output above confirms the changes made to your firewall were successful. So you are ready to enable the changes in Nginx. Step 4. Enable to changes in Nginx First, check that there are no syntax errors in the files. Run sudo nginx -t The output will most likely look like Output nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/ssl/certs/nginx-selfsigned.crt" nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful You can disregard the ssl_stapling warning, this particular setting generates a warning since your self-signed certificate can’t use SSL stapling. This is expected and your server can still encrypt connections correctly. If your output matches the out example above, that confirms your configuration file has no errors. If this is true, then you can safely restart Nginx to implement the changes: sudo systemctl restart nginx Step 5. Test the encryption Open up a browser and navigate to https://<server_name>, use the name you set up in Step 2C. Additional information 1. It was reported that File Manager and Grafana doesn't work with reverse proxy. To get the URLs for file manager and grafana to work, following steps should be taken: kubectl -n sisense set env deploy/filebrowser FILEBROWSER_BASEURL='/<baseurl>/app/explore' kubectl -n sisense set env deploy/filebrowser FB_BASEURL='/<baseurl>/app/explore/' kubectl -n sisense set env deploy/sisense-grafana GF_SERVER_ROOT_URL=<baseurl>/app/grafana 2. Once the reverse proxy is enabled, Sisense will still utilize IP addresses as links in their email communications. To setup correct addresses in Sisense e-mails after reverse proxy is configured: in the configuration yaml file set: update: true application_dns_name: "" and start the installation script to update parameters. After update is completed, in Sisense GUI go to Admin -> Server & Hardware -> System management -> Configuration Set the http://YOUR_PROXY_ADDRESS/analytics in the "Proxy URL" field of "Web Server" menu (or https://YOUR_PROXY_ADDRESS/analytics in case of SSL) Go to Admin -> User Management -> Users Try creating a new user or use the "Resend invitation" option for the existing one (if available) Check the inbox of that user for "Sisense account activation" The "Activate Account" link should now redirect to the http://YOUR_PROXY_ADDRESS/analytics/app/account/activate/HASH address
