Query result limit on live models and available workarounds [Linux]

Step-by-Step Guide: 

Understanding the Query Limit

In Sisense, live model configurations include a maximum result limit of 50,000 records per query. This cap is enforced to maintain performance and system integrity when processing large datasets.

Verifying Advanced Query Configuration

• Navigate to Advanced Query Configurations in your Sisense settings.
• Look for the setting: LimitOfDownloadRowsForLive.value. Go to the Amnid tab - Server & Hardware - Configuration - click 5 times on the Sisense logo - Query. By default, this may be set to 0, which allows queries to reach the predefined limit.

• Note that adjusting this value does not override the $ 50,000 cap.

Workarounds for Querying More than 50K Records

If your project requires retrieving more than 50,000 records, consider the following solutions:

• Splitting Data Across Multiple Tables:
• Divide large datasets into smaller chunks or subsets based on logical data segmentation (e.g., date ranges, categories, or other filters).
• Create separate tables to house these subsets, ensuring each table falls within the query limit.
• Creating Multiple Widgets:
• Design multiple widgets to query the split tables. Each widget will handle a portion of the data, effectively circumventing the 50K cap without overloading a single query.
• Aggregate or visualize the data across widgets for a cohesive view.

Conclusion 

The Sisense live model query limit is capped at 50,000 records to ensure smooth system operations. While this limit cannot be overridden, using strategies like splitting your data into multiple tables and creating separate widgets can help you work around the restriction, enabling the retrieval of larger datasets without compromising performance.

References/Related Content 

• Sisense Documentation on Live Models
• Best Practices for Large Dataset Management
• Creating and Managing Widgets