Troubleshooting Sisense validator [Linux]
Validator is typically launched after the Sisense installation/upgrade is completed and verification of the application’s readiness is required. In general, Validator is a Kubernetes pod handled by a respective job that checks the status of mandatory Sisense services and exits with code 0 in case no errors are present. However, in some cases Validator might fail due to various reasons, such as a higher Kubernetes API response than expected or other environmental causes. This guide is dedicated to the troubleshooting process of the Sisense Validator service.
Step-by-Step Guide:
In case the Validator is failing, we can perform some adjustments and review its process from the pod itself to verify what exactly went wrong.
As the Validator pod is launched by its job, it is designed to be stopped after completion, whether it was successful or not.
So, in order to proceed, we will need to copy the existing job and launch its test duplicate, but with adding ‘sleep’ option added so it would remain running even after completion. We will also need to get rid of auto-generated values filled during the original job creation.
You can run the following command that will:
- Search for the latest Validator job
- Remove unneeded values
- Re-launch its copy with “test” name and “sleep” option
===============
kubectl -n sisense get job $(kubectl -n sisense get jobs -l app.kubernetes.io/name=validator -o json | jq -r '.items | sort_by(.metadata.creationTimestamp) | last | .metadata.name') -o json | \
jq 'del(
.metadata.creationTimestamp,
.metadata.generation,
.metadata.resourceVersion,
.metadata.uid,
.spec.selector,
.status,
.spec.template.metadata.labels["controller-uid"],
.spec.template.metadata.labels["batch.kubernetes.io/controller-uid"]
)
| .metadata.name = "validator-test"
| .spec.template.metadata.labels["batch.kubernetes.io/job-name"] = "validator-test"
| .spec.template.metadata.labels["job-name"] = "validator-test"
| .spec.template.spec.containers[0].command[2] |= gsub("# sleep 6h"; "sleep 6h")' | kubectl apply -f -
===============
After running this command, the new job and its pod with the name validator-test-xxx will be created and will remain in the running state for 6 hours.
For further troubleshooting, follow the steps below:
- Exec into the pod:
kubectl -n sisense exec -it $(kubectl -n sisense get pod | grep validator-test | awk '{print $1}') -- bash - Execute run.sh script with ‘bash -x’ (will show detailed process of script execution):
bash -x ./run.sh - Observe the results that will help to better understand the nature of the issue
After troubleshooting is done, feel free to delete the test Validator job:
=========
kubectl -n sisense delete job validator-test
=========
In case you need further assistance from Sisense, please feel free to open a Support Ticket.
Conclusion:
This article provides the steps that are useful for troubleshooting Validator issues in Sisense.
Disclaimer: This post outlines a potential custom workaround for a specific use case or provides instructions regarding a specific task. The solution may not work in all scenarios or Sisense versions, so we strongly recommend testing it in your environment before deployment. If you need further assistance with this, please let us know.
