Controlling JAQL Timeout | Linux
Please note this article is for Linux users.
What You’ll Learn
- Main Properties of a JAQL request
- When the JaqlPool parameters for managing JAQL request timeouts are triggered
- When to change the default values for these JAQL request timeout parameters
- Reconciling JaqlPool parameters with the galaxy log
Use Cases For Applying What You Will Learn
These JAQL request timeout parameters can be adjusted when Data Security is enabled and:
- Large cubes are slow to start after a build
- Cubes are responding slowly, possibly under a heavy workload
- JAQLs are expected to be a long-running
- There are a significant number of data security rules
Content
When a user runs a dashboard, Sisense generates one or more JAQL requests for widgets, filters, data security rules, and other entities. The JAQL has many properties with the main ones being:
Datasource Name of the ElastiCube against which to execute the query
Metadata Contains an array of JAQL elements. A JAQL element is essentially dimension
or measures.
type Defines the metadata element type for dimension or measure.
dim Name of the dimension.
Sample JAQL using main properties
{ "datasource": "sample-data", "metadata": [ { "jaql": { "dim": "table1.column1" } }, { "jaql": { "dim": "table1.column2" } } ] }
The JAQL syntax is covered in this documentation for anyone who wants to learn more about it.
Sisense consists of several microservices to handle different functions. The JAQL request is processed by the query service and then sent to the translation service for conversion to SQL which is then returned to the query service. The query service forwards the SQL request to the Elasticube monetDB for data retrieval.
Sometimes, for reasons mentioned earlier in this article, the JAQLs can take some time to run.
In releases prior to L2021.3.1, the galaxy service was based on 'jaqlRunner' functionality which did not check if the cube was running and responding to requests so JAQL requests would fail right away rather than waiting for the cube to respond. The galaxy service has since been improved to work with the more robust functionality 'jaqlPool' which was introduced to address this gap. JaqlPool pings the cube before running the JAQL to make sure it is running.
There are a few parameters in the Sisense galaxy service configuration that can be adjusted to help prevent the JAQL requests from timing out.
These configuration parameters, shown in the following screenshot can be accessed from:
Admin tab -> System Configuration -> Configuration Manager -> Service Configuration -> Galaxy
In the JaqlPool section of the Galaxy service configuration, three JAQL parameters are used to manage the JAQL timeouts.
jaqlPool.checkin.Timeout (default value: 5 seconds)
This timeout will be invoked if the submission of a single JAQL to the EC-{cube}-qry* pod exceeds a wait time of 5 seconds for a response.
jaqlPool.pingInterval (default value: 2 seconds)
This ping interval will trigger if the EC-{cube}-qry* pod does not respond after the jaqlPool.checkin.Timeout period, and will continue to ping for the period of time specified by the jaqlPool.pingTimeout parameter.
jaqlPool.pingTimeout (default value: 20 seconds)
This timeout will be invoked if the EC-{cube}-qry* pod fails to respond within this time period.
When a JAQL request is issued, an internal check is made to the Elasticube to determine if it is responsive. If, after the time specified by the parameter jaqlPool.checkinTimeout, there is no response, then subsequent checks are made at intervals for some duration, specified by the parameters jaqlPool.pingInterval and jaqlPool.pingTimeout respectively.
If the Elasticube responds within this duration period, then the JAQL request is processed successfully and a response is returned to the requestor.
If the elasticube does not respond after this duration, an error message is generated and displayed in the UI.
Example
In the following example, the user ran a dashboard that eventually generated the error shown in the following image, which required a change of the default values of the JAQL parameters to address.
- With Data security enabled, the dashboard filters are spinning, then followed by a ‘Something has gone wrong’
Reconciling JaqlPool parameters with the galaxy.log
In summary, one can check if these parameters may need adjusting by performing a search of the galaxy log for the following occurrences:
- [Checkin jaql failed because ESOCKETTIMEDOUT]
- failed to get the dashboard from mongo: Error: The EC query pod {cubename} was not responsive for 20s
In the example below, we use the [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] to map the galaxy logs to each of these parameters.
One can obtain the X-request from the Headers section of the Developer Tools HAR file for the dashboard being executed.
The snippets below have been taken from the galaxy log with irrelevant portions removed.
Dashboard request comes into galaxy
2022-07-27T04:01:17.031, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Start handling request => GET /api/dashboards/62d5d8db4813a0002bddf622]
jaqlPool.checkin.Timeout is invoked here at 04:01:17
2022-07-27T04:01:17.206, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Running checkin request for data source Sample ECommerce]
2022-07-27T04:01:17.208 [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [trying to run jaql, datasource: Sample ECommerce, jaql: {"queryGuid":"71A1B-FE60-247C-50E9-D21B-0CC8-55EF-7DEA-5","datasource":{"fullname":"localhost/Sample ECommerce","id":"localhost_aSampleIAAaECommerce","address":"LocalHost","database":"aSampleIAAaECommerce","live":false,"title":"Sample ECommerce"},"metadata":[{"dim":"[Brand.Brand ID]","filter":{"members":["1"]}}]}
jaqlPool.checkin.Timeout ‘times out’ here at 04:01:22 ( 5 seconds later)
2022-07-27T04:01:22.215, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Error: ESOCKETTIMEDOUT
2022-07-27T04:01:22.216, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Error running jaql: ESOCKETTIMEDOUT]
2022-07-27T04:01:22.217, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [runJaql [Brand.Brand ID] executed in: 00:00:05]
2022-07-27T04:01:22.217, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Checkin jaql failed because ESOCKETTIMEDOUT]
jaqlPool.pingTimeout is invoked at 04:01:22
PINGING STARTS EVERY 2 SECONDS FOR 20 SECONDS
2022-07-27T04:01:22.218, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [The pool will ensure that ec query pod is active and retry jaql]
2022-07-27T04:01:24.277, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:26.285, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:28.226, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:30.272, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:32.226, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:34.227, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:36.281, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:38.229, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
2022-07-27T04:01:40.269, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Pinging ec query pod Sample ECommerce, active: false]
CUBE FAILS TO RESPOND WITHIN THE jaqlPool.pingTimeout period
2022-07-27T04:01:42.221, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Failed to apply datasecurity on filters: Error: The EC query pod Sample ECommerce was not responsive for 20s
2022-07-27T04:01:42.222, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [failed to get the dashboard from mongo: Error: The EC query pod Sample ECommerce was not responsive for 20s
RETURN CODE IS 500 WHICH IS SHOWN IN THE UI AS ‘Something has gone wrong’
2022-07-27T04:01:42.224, [X-request:e1337ea8-0649-4d1a-ad25-2937289f4dc3] [Finished handling request => GET /api/dashboards/62d5d8db4813a0002bddf622 500]
Conclusion
In conclusion, the hope is that the information covered in this article will come in handy as you work with the Sisense platform. If you need any additional help, please contact Sisense Support or create a Support Case.
