Forum Discussion

Tim's avatar
Tim
Data Integration
01-20-2025
Solved

Does Sisense free up RAM from crashed queries and builds?

I've been looking at RAM use. While experimenting, I sometimes build bad widgets or cubes that use lots of RAM and then fail with the "abnormal memory consumption" message. That's fine, but I noticed...
dashboards & reporting
Data Models
Performance Help
  • ihoryokhym's avatar
    ihoryokhym
    01-21-2025

    Hi Tim,

    Thank you for reaching out with your question about RAM usage in Sisense. It's great to see you're actively experimenting and exploring the platform's capabilities.

    When a cube build or query consumes a large amount of RAM and subsequently fails, Sisense does not automatically release the RAM immediately. This is because the RAM remains allocated to caching mechanisms for potential reuse, which helps improve performance for subsequent operations.

    If you'd like to free up RAM after a failed or canceled build, the best approach is to stop the specific cube. Stopping the cube releases all cached RAM associated with it, ensuring those resources are available for other users or further experiments.

    To stop a cube, you can follow these steps:

    1. Navigate to the Data tab in the Sisense Web Application.
    2. Locate the specific cube.
    3. Click the Stop option to terminate it.

    Once stopped, you can restart the cube as needed and continue experimenting.

    Please let us know if you have additional questions or need further assistance.

    Best regards,

    Ihor

ihoryokhym's avatar
ihoryokhym
Sisense Employee
01-21-2025

Hi Tim,

Thank you for reaching out with your question about RAM usage in Sisense. It's great to see you're actively experimenting and exploring the platform's capabilities.

When a cube build or query consumes a large amount of RAM and subsequently fails, Sisense does not automatically release the RAM immediately. This is because the RAM remains allocated to caching mechanisms for potential reuse, which helps improve performance for subsequent operations.

If you'd like to free up RAM after a failed or canceled build, the best approach is to stop the specific cube. Stopping the cube releases all cached RAM associated with it, ensuring those resources are available for other users or further experiments.

To stop a cube, you can follow these steps:

  1. Navigate to the Data tab in the Sisense Web Application.
  2. Locate the specific cube.
  3. Click the Stop option to terminate it.

Once stopped, you can restart the cube as needed and continue experimenting.

Please let us know if you have additional questions or need further assistance.

Best regards,

Ihor

  • Tim's avatar
    Tim
    Data Integration
    01-21-2025

    Hi Ihor, ok, I've used stop and restart to free ram before. You say "remains allocated to caching" - does that mean that if another process/cube/query needs RAM, Sisense will deallocate the crashed query from cache to free up space?

    • ihoryokhym's avatar
      ihoryokhym
      Sisense Employee
      01-22-2025

      Thank you for your follow-up question, Tim! I appreciate your curiosity about how Sisense manages RAM allocation.

      In Sisense, if a cube build or query fails and RAM remains allocated to caching, that memory is typically reserved for performance purposes. However, Sisense does have mechanisms to manage memory dynamically. If another process, cube, or query requires RAM, the system will attempt to deallocate or reassign cached memory to accommodate the new demand.

      That said, the effectiveness of this reallocation depends on overall system usage and configuration. In scenarios where memory pressure is high, manually stopping a failed cube can proactively free up cached RAM and ensure availability for other operations.

      Please let me know if you'd like to dive deeper into this or explore any specific examples!

      Best regards,
      Ihor

      • Tim's avatar
        Tim
        Data Integration
        01-22-2025

        Ok.

        In simpler terms, sounds like: "A bad query can use a lot of RAM. Caching means it occupies that RAM even after it has finished or crashed. If a new query needs that RAM, Sisense might choose to reallocate RAM. If it doesn't, the new query gets the Memory Pressure error message. You can manually restart a cube to remove its cached queries from RAM. You can see which cube is using the most RAM with 'ps aux --sort=-%mem | head'."

        You say "the effectiveness of this reallocation depends". Shouldn't Sisense always clear the oldest cached items when a new query needs RAM? Otherwise, doesn't that mean the system can be crippled by cached junk? Also, if a query has crashed/failed/errored, surely that query can be safely removed from cache, or, at most, leave just its error message in cache?

        My situation is: A bad query used 100GB of RAM and then failed. Several days later, a cube build failed with the "Memory Pressure" error. The reason was that the failed query's 100GB was still cached in RAM. I manually identified and restarted the cube, but doing that proactively is not practical day-to-day. So, does this mean "Querying a badly designed cube can make Sisense unoperational until a technician restarts the cube or restarts the server"? I don't like that. I think a bad query should cripple the server for a few minutes, but after it finishes or errors, Sisense should always free up RAM for the next query.

        Is there a setting I can alter so Sisense always clears cached items if it needs RAM? Is there a setting so that when a query errors, Sisense immediately clears it from cache (except, perhaps, the error message)?