Managing Linux Build Service Settings for Optimal Performance and Stability

Managing Linux Build Service Settings for Optimal Performance and Stability

An Elasticube build is a complex process that invokes multiple services.  This article explains some of the settings that are used for controlling the build service which affects performance and availability. The primary services used by the build process are connectivity, management, and build. This article is specific to the build service within the ElastiCube build process. Additional documentation is also available for Troubleshooting Memory Issues and Data Groups.

The correct values for these settings vary from deployment to deployment and depends on many factors. The primary factors are available memory, available CPU, and whether it is a single or multi-node configuration. Changes to these settings must always be within the context of the overall resources available across the system.

Base Table Max Threads

The Base Table Max Threads setting controls parallel processing (threads), and is set to 4 by default. This setting can be changed to 1 temporarily for troubleshooting table builds without parallel processing. Correct settings will vary depending on CPU availability on the build node (if multi-node) while balancing against other processes that require system resources. This is an especially important setting if you are seeing any warnings or errors related to parallel processing.

Admin > System Management > Configuration (top right) > scroll down > Build. 

Maximum Memory Limit

The maximum memory limit setting controls the amount of JVM memory available for all currently running build processes combined. Changing this setting will cause a very short outage as services restart. 

Admin > System Management > Configuration (top right) > 5 clicks on Sisense Logo (top left) > General > System Configuration > scroll down > Build. 

Build Pod Memory and CPU Limits (Linux)

This controls resources available for the build service. If you increase the value too much you may not be able to schedule. Any value above 3000 is not recommended without Sisense consultation.

Linux > SSH > Kubectl -n <namespace> describe deployment build > Containers: build: Limits

To view this setting:

1. SSH to the Sisense server
2. View the build configuration:
   kubectl -n <namespace> describe deployment build
3. Find CPU and memory under Containers:build:Limits

To edit this setting:

1. SSH to the Sisense server
2. Make a backup of the current build deployment yaml
   kubectl -n <namespace> get deployment build -o yaml > build_deployment.yaml 
3. Edit the build deployment:
   kubectl -n <namespace> edit deployment build
4. Edit the memory limit to 1000 less than the Build MaxMemoryLimit setting:
   Find "resources: limits" and update memory size (press “i” on the keyboard to enter edit mode → edit value → ESC to exit edit mode → use “:wq!” to exit and save changes).
5. Build pod will be automatically restarted after deployment modification.

While these settings are the foundation for the build service, they are only a part of the overall build process.  Additional documentation is also available for Troubleshooting Memory Issues and Data Groups.

If you need additional help, please contact Sisense Support.