Does Sisense free up RAM from crashed queries and builds?

[Tim]

I've been looking at RAM use. While experimenting, I sometimes build bad widgets or cubes that use lots of RAM and then fail with the "abnormal memory consumption" message. That's fine, but I noticed after the RAM use climbs up, it remains high. After the failure, shouldn't Sisense free up the RAM? I want to experiment, and I don't mind hogging server resources while building or querying, but I expect those resources to be released for other users after I cancel or crash.

[ihoryokhym · Accepted solution]

Hi Tim,

Thank you for reaching out with your question about RAM usage in Sisense. It's great to see you're actively experimenting and exploring the platform's capabilities.

When a cube build or query consumes a large amount of RAM and subsequently fails, Sisense does not automatically release the RAM immediately. This is because the RAM remains allocated to caching mechanisms for potential reuse, which helps improve performance for subsequent operations.

If you'd like to free up RAM after a failed or canceled build, the best approach is to stop the specific cube. Stopping the cube releases all cached RAM associated with it, ensuring those resources are available for other users or further experiments.

To stop a cube, you can follow these steps:

1. Navigate to the Data tab in the Sisense Web Application.
2. Locate the specific cube.
3. Click the Stop option to terminate it.

Once stopped, you can restart the cube as needed and continue experimenting.

Please let us know if you have additional questions or need further assistance.

Best regards,

Ihor

[Tim]

Hi Ihor, ok, I've used stop and restart to free ram before. You say "remains allocated to caching" - does that mean that if another process/cube/query needs RAM, Sisense will deallocate the crashed query from cache to free up space?

[ihoryokhym]

Thank you for your follow-up question, Tim! I appreciate your curiosity about how Sisense manages RAM allocation.

In Sisense, if a cube build or query fails and RAM remains allocated to caching, that memory is typically reserved for performance purposes. However, Sisense does have mechanisms to manage memory dynamically. If another process, cube, or query requires RAM, the system will attempt to deallocate or reassign cached memory to accommodate the new demand.

That said, the effectiveness of this reallocation depends on overall system usage and configuration. In scenarios where memory pressure is high, manually stopping a failed cube can proactively free up cached RAM and ensure availability for other operations.

Please let me know if you'd like to dive deeper into this or explore any specific examples!

Best regards,
Ihor

[Tim]

Ok.

In simpler terms, sounds like: "A bad query can use a lot of RAM. Caching means it occupies that RAM even after it has finished or crashed. If a new query needs that RAM, Sisense might choose to reallocate RAM. If it doesn't, the new query gets the Memory Pressure error message. You can manually restart a cube to remove its cached queries from RAM. You can see which cube is using the most RAM with 'ps aux --sort=-%mem | head'."

You say "the effectiveness of this reallocation depends". Shouldn't Sisense always clear the oldest cached items when a new query needs RAM? Otherwise, doesn't that mean the system can be crippled by cached junk? Also, if a query has crashed/failed/errored, surely that query can be safely removed from cache, or, at most, leave just its error message in cache?

My situation is: A bad query used 100GB of RAM and then failed. Several days later, a cube build failed with the "Memory Pressure" error. The reason was that the failed query's 100GB was still cached in RAM. I manually identified and restarted the cube, but doing that proactively is not practical day-to-day. So, does this mean "Querying a badly designed cube can make Sisense unoperational until a technician restarts the cube or restarts the server"? I don't like that. I think a bad query should cripple the server for a few minutes, but after it finishes or errors, Sisense should always free up RAM for the next query.

Is there a setting I can alter so Sisense always clears cached items if it needs RAM? Is there a setting so that when a query errors, Sisense immediately clears it from cache (except, perhaps, the error message)?