EC-QRY Pod CrashLoopBackOff - MonetDB Segmentation Fault When Using UNION in Custom SQL Expression | Data Models

EC-QRY Pod CrashLoopBackOff - MonetDB Segmentation Fault When Using UNION in Custom SQL Expression

Summary

The ElastiCube query pod (ec-qry) enters a CrashLoopBackOff state with a segmentation fault (core dump) when executing a custom SQL expression that uses UNION to combine multiple SELECT statements from the same table. The issue is caused by MonetDB's deduplication sort pass during UNION operations on large datasets. Replacing UNION with UNION ALL resolves the crash.

Applies To

Sisense Linux (all versions, confirmed on L2026.2)
ElastiCube custom SQL expressions using UNION
Self-hosted and cloud deployments

Symptoms

The ec-qry pod enters CrashLoopBackOff state after a query is executed.
Pod logs show MonetDB starts successfully, binds to port 50000, and passes the CT probe - but crashes immediately after MAL plan creation completes.

The pod log contains:

/opt/sisense/monetdb/bin/mserver.sh: line 509: 36 Segmentation fault (core dumped) /opt/sisense/monetdb/bin/mserver5 $ARGS $SUFFIX

The kubectl describe pod output shows:

State: Waiting
  Reason: CrashLoopBackOff
Last State: Terminated
  Reason: Completed
  Exit Code: 0

Readiness probe fails with: dial tcp <pod-ip>:50000: connect: connection refused
The query was previously working without issues on the same or earlier versions.

Root Cause

The UNION operator (without ALL) requires MonetDB to perform a deduplication sort pass across the combined result sets. Under certain conditions - particularly with multiple UNION branches reading from the same table with dimension table JOINs - this sort/dedup operation triggers a segmentation fault in MonetDB's mserver5 process.

This is a known limitation tracked in Jira as SNS-113713.

The crash occurs after MAL (MonetDB Assembly Language) plan creation completes, during the execution phase of the deduplication.

Resolution

Replace all instances of UNION with UNION ALL in the custom SQL expression.

Before (crashes):

Select col1, col2, 'Category1' as category from MyTable
UNION
Select col1, col2, 'Category2' as category from MyTable
UNION
Select col1, col2, 'Category3' as category from MyTable

After (works):

Select col1, col2, 'Category1' as category from MyTable
UNION ALL
Select col1, col2, 'Category2' as category from MyTable
UNION ALL
Select col1, col2, 'Category3' as category from MyTable

Why This Works

UNION removes duplicate rows by performing a sort + deduplication pass across all combined result sets. This is the operation that crashes MonetDB.
UNION ALL returns all rows from all branches without deduplication, bypassing the problematic sort operation entirely.

In most custom SQL use cases where each sub-query selects different literal values (e.g., different category names), the rows are already unique by definition. The deduplication step is unnecessary, making UNION ALL both safer and faster.

Additional Notes

This issue is not caused by insufficient memory or CPU resources.
This issue is not caused by CASE WHEN expressions alone, though UNION combined with CASE WHEN can trigger related MonetDB bugs.
The crash may appear as a regression after an upgrade because changes in the SQL translator or MonetDB optimizer can alter how UNION queries are executed internally.
UNION ALL is generally recommended as a best practice in ElastiCube custom SQL whenever duplicate elimination is not explicitly needed, as it reduces memory usage and improves query performance.

Anton Dyachenko

Posted 1 month ago

0 comments