Faster UI for large datamodels

Tim · ‎03-11-2024

I have a data model with lots of tables (more than 30). That makes the UI painfully slow, e.g. when editing relationships. I think the reason is that it’s calculating where to place them on the screen. I’d like to request an alternative, crude interface that doesn’t slow down for lots of tables. Or make it work for lots of tables – e.g. do the positioning calculations when first opening, but not each time I resize or click on something.

fschleich · ‎07-12-2024

Currently, the aggregation function depends on the type of the dimension : - Text or date => Distinct count - Numeric => Sum In our use case, this default assignation is not always accurate as it is sometime required to use Distinct Count with numeric fields. It would be good to hauve the default aggregation function to be choosed by the cube designer, when developping the metadata layer (ElastiCube or Live cube). This way, the dashboard designer will be proposed the accurate function without need to change it. This is particlary interesting with a users population that don't have analytics background and for which choosing the correct agregation function is not obvious. Thank you

soporteparaptx · ‎11-14-2023

Hi, We are facing some issues in the hour timezone related to builds. There's no way to change the UTC setting for the build schedule. It's very confusing and it's generating many issues in timezones when there are changes in the hour due to summer/winter time adjustments. We tried to change the timezone in the upgrade process, but the build schedule only works in UTC. This is a nightmare for data administrators. We need to be able to set the local timezone as the build schedule setting. Best

edtguy · ‎02-22-2024

There are many scenarios where a model contains data for many distinct groups, such as customers, and you want viewers to only have access to the data that pertains to their specific customer. This is easily implemented with row level security, when there is a data column that uniquely identifies each customer. But what if you have a requirement that certain types of viewers should see all columns of data per row, while others should not have access to certain columns? Here is an approach that can be implemented right out of the box in Sisense. The one prerequisite is that any table that contains sensitive data must have a column that uniquely identifies every row, i.e. a primary key column. For every table that contains sensitive data in certain columns, create a new table that contains the original table's primary key column, a binary "switch" column with a name like [Access Allowed] and a column for each of the sensitive columns. Remove the sensitive columns from the original table. Modify the import query to load only the non-sensitive columns in the original table. Modify the import query to load the new table with the primary key and sensitive columns of the original table twice, once with [Access Allowed] set to 1 and the second time with 0 and blank (or null) for all the sensitive columns. Associate the new table to the original table by primary key column Create a dimension table for secure access with the column [Access Allowed]. One row with value 0, one row with value 1. Associate the dimension table with the sensitive data table by the [Access Allowed] column. Set up a data security rule on this column, assigning the secure user group to value 1 and the general group to value 0 If there is a hierarchy greater than two levels of column-level data security, this model can be extended to create additional one-to-one related tables to the original table.

dougnewton · ‎07-25-2023

It seems strange to me that each time I need to add another table to my model, I have to re-enter all the connection (or pick a recent connection). If i have 10 tables in my model, they all have their own individual connection information. If a server name ever gets renamed, it'll be a crazy headache for us. There should be a place where you define the distinct list of actual database connections (one per server or database), give it a name, apply security, etc. And then when you go to add a table to a data model, at that point you pick from the previously defined available list of connections.

Tim · ‎04-10-2024

When you export a datamodel to a .smodel file, the file includes connection information (server name, password, etc.). I would like an option to exclude that information when I click export. The export dialog already offers check boxes for Data Security, Scope Configuration, etc.; this feature could be just another one that says "Connection Information". Ideally, I'd like it to exclude the connection information by default, so users are forced to explicitly ask to include it, which will remind them to consider the file as sensitive. But I understand if you don't want to change default behaviour that other users are used to. When you import a model that has had this information removed, the import should let you to enter new connection information, or leave it as placeholders like Server=0.0.0.0 and password=Password so you can enter the information after importing. Sisense GIT already removes this information; this feature could work in a similar way. You could even deliberately make GIT's "non-tracked entities" all appear on the list of export options. GIT per this page: https://docs.sisense.com/main/SisenseLinux/asset-translation-and-management.htm?tocpath=Git%20Integration%7C_____3#Non-trac

dixitraj · ‎04-02-2024

Hi, I have gone through your connection update process for live models. But As I discussed with sisense team, they don't have any single API for updating connection. As per current scenario we need to follow multiple steps like below:- 1. Get table by: GET /datamodels/datamodelId/schema/datasets/datasetId/tables/tableId. response => tableResult 2. Override table by PATCH /datamodels/datamodelId/schema/datasets/datasetId/tables/tableId} where body = tableResult from the step 1. Only after performing the above steps, you can use PATCH/datamodels?/datamodelId?/schema?/datasets?/datasetId to update the schema for this dataset. So I just want to proposed one idea here that we can use multiple option here like below:- 1. We can provide any Key Vault connectivity for each model where we can store our connection details and we can use them for our model connections. 2. We also can provide a single API where our connection can we updated for all relevant objects & model it self.

Tim · ‎06-23-2024

Two problems: Datamodels store old connection information without the user being aware. That increases the chance of accidentally exposing credentials to a different security context. Sisense R&D said that the reason is to show Recent Sources. When you change a dataset, you can end up with two copies of the dataset, which is confusing. Ideas for improvements (take some or all): Sisense could track the Recent Sources separately. Then Sisense could automatically remove empty datasets. Sisense could show empty datasets under Data Sources in the data model editor UI, and give the user a delete button for them. The Data Model editor could have a drag-and-drop to move tables between data sources. When you click export, there could be a checkbox "Remove data source connection information". More detail about the problems here: https://community.sisense.com/t5/support-portal/bd-p/SupportPortal?page=casedetails&caseId=500Pk00000Es4PyIAJ and here:https://community.sisense.com/t5/help-and-how-to/models-have-empty-datasets/m-p/20632#M222

Community_Admin · ‎04-17-2023

**Please note this idea was migrated to the Product Feedback Forum from older submission areas** User parameters must be set and cannot be used for dynamic customizations. When setting up data models, these parameters cannot be included as part of a filter in the underlying query. We would like a way to reference a global or system variable that includes the username of the currently logged in user.

wallingfordce · ‎06-14-2024

When replacing a server, or testing an upgrade, we often find our clients taking a backup of their Sisense database and restoring it to another server. After updating the server identity in the mongodb, it's pretty much ready to go. However, use dashboards all reference the old server and have to be updated. This is a manual process that the users themselves one dashboard and data source at time. An administrator cannot easily do that work on their behalf. Acknowledging that there is a replace_datasource API call that allows you to apply a new server and data source to a single dashboard, I would love an easier a way to update just the server across all dashboards.

ccoelho · ‎06-12-2024

It would be really useful to have a hierarchy filter displaying parent-child relationship. Currently when we create a filter, it just displays all the members in the entity. If there is a relationship between the members its not possible to identify it directly in 1 filter, the only way around is create dependent filters. But that too would get tedious, as everytime we have go through multiple filters to see the members in each sub filter. Example: If there is a relationship as below: In this case, the table has the name and parent name as 2 separate columns. So the filter could display members as below in a tree format which can be collapsed or expanded, such that there is a check box next to the member and user can select the individual members, or a group of members. It would give the user a clear view of the relationship between different members:

dougnewton · ‎12-22-2023

Right now, if you're making significant changes to an existing cube, the only way to verify that you haven't broken something is to actually do a build. I'd like a Verify or Validate button, next to the Build button, that would do the initial steps of the build process and then stop. For example, determine if the metadata in the table can actually be retrieved from the underlying data source. Or checking datatype mismatches in existing joins. And so on. Just leave out the last step where you actually retrieve the full data set and then build the cube. This would save time during the development process.

bputhiyaveetil · ‎12-17-2023

Hello There, We need to create a data model with 2 different data sources . One databricks live and another one is an excel file with some additional data . We need to create a combined model so that we can join these two tables and get the result. Currently sisense does not support it and other providers like Power BI and Zoho supports this feature. This is a common use case for our scenario

Precisio · ‎09-12-2023

Currently, it appears that Shared Formulas are displayed in the selection dialog based on the order they were created. It would be great if they can be arranged logically, or at least sorted alphabetically, otherwise the list can grow to become disorganized and difficult to use.

adammor · ‎11-16-2023

We need a view that shows us which dashboards are dependent on which table in the datamodel. We want to remove some tables, but because we have many dashboards we don't know where exactly that table data is being used so it's very risky to change/delete tables. The depedency granularity of the data model is good but not enough.

qamardinpal · ‎10-04-2023

Perspectives is a great addition to the application and making only certain parts of the model visible to those dashboards connected to that perspective is excellent. Adding the ability to filter the perspective is something that is required for many of the use cases we have. For example, we have a model that has All Tickets. We have many dashboards that have a locking filter at the top to show only Open Tickets which doesn't look good from a user and design point of view. Being able to create a Perspective and not only be able to refine the columns but also filter to (in this example) Open Tickets only would greatly add to the power and usability of this feature.

Edues · ‎04-17-2024

Currently, whenever a user starts a build in a cube, either everything is updated or only the changes that were made. It would be incredibly helpful to have the cube refresh only certain tables as indicated by the end user.

fschleich · ‎10-13-2023

Hello, It is not possible to rename a cube, but it is possible de copy it. If we then move the existing dashoards to the copied cube, we have workaround to rename the cube. But with Perspective it is not possible to rename nor to copy. => It would be useful to be able to copy (duplicate) a perspective. Thank you ! Franck

gupta3 · ‎02-27-2024

Standard Live and Elasticube connector for AWS EMR: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-connectors.html

Tim · ‎07-09-2023

Summary: Live Model slow, can be fixed by prioritising a filter I use a Live Model to return info about a few records from a big dataset. One filter should efficiently narrow the big dataset down. But the server sometimes applies other filters or joins first, making it unusably slow. I want an option to tell the Live Model to fetch the important filter's items first, and then write those items into the filtering clause when writing the main query. That will cause the server to correctly discover that applying the important filter first is the most efficient query plan. Details (I'm querying SQL Server, but I suspect other sources would also benefit.) Sisense documentation says "Analytical Engine inserts filtering clauses into the innermost SQL statement. This means that filtering is the first operation performed". Sometimes that helps performance. But it depends on the source server. If you apply multiple filters or joins, the server may decide to perform a less efficient one first, or it may parallelise. In my case, my query went from a few seconds to over an hour depending on the server's choice. The problem is that the server can't estimate the cardinality of the "filtering clauses" entered by Sisense. For example: The server doesn't know that my "dimCustomer.[Customer Name] = 'Joe Bloggs'" filter returns only 100 rows from my fact table, whereas "dimDateTime.FiscalYear = 2022" returns 100,000,000 rows from the fact table. There is a solution. Instead of "Where Fact.[Customer ID] In (Select [Customer ID] From dimCustomer Where [Customer Name] = 'Joe Bloggs')", Sisense should first run the query Select [Customer ID]... query to find that Joe's ID is 5. Then put the result into the filtering clause, i.e. "Where Fact.[Customer ID] IN (5)". For a hardcoded ID like that, the server can estimate the cardinality, so it sees that filtering on [Customer ID] first will be more efficient that other query plans. Sisense mustn't just do that for all filters. If it tries prefetching "dimDateTime.FiscalYear = 2022", that'll return all 31,536,000 seconds, and then pass that back in the query as "Where Fact.DateTimeID IN (...)". Solution: The solution I request is: Allow the Live Model developer to set a "Prefetch Filter Items" flag on a field. If it is set, when a widget or dashboard filters that field, then Sisense first creates a query "Select dimSomething.[Relationship column] From dimSomething.FieldWithPrioritisedFilter". Then, when building the main query, change the "filtering clauses" to be "Where Fact.[Relationship column] IN (<put prefetched values here>)". Reference: For reference: https://docs.sisense.com/main/SisenseLinux/analytical-engine-overview.htm: Analytical Engine inserts filtering clauses into the innermost SQL statement. This means that filtering is the first operation performed, minimizing the number of rows that need to be scanned in order to process the query. Complex filters, such as top ranking and filter on measure are significantly improved as a result. Proof of concept: Build a Live Model. FactA joins to DimB on ID_B and to DimC on ID_C. The source is SQL Server. In SQL Server, all three tables have millions of rows, and the FactA has foreign keys to DimB and DimC. DimB.Name has a unique name for each row. DimC.Type is 1 for half the rows, and 0 for half the rows. Create a widget that filters for DimB.Name=<some name> and DimC.Type=1. Use SQL Profiler to find the query your Live Model runs. Run the query; it should take over a minute or time out. Then change the "Where FactA.ID_B IN (Select ID_B From DimB Where DimB.Name IN ('<some name>'))" to instead be "Where FactA.ID_B IN (<copy paste the ID of <some name> here>)". It should run much quicker. The original query might run quickly without change, depending on server settings; if so, mess about with more complicated joins or ask me. In my setting key queries were reliably slow.

Sisense Community

Faster UI for large datamodels

Defining the default aggregation function per dimension

Build schedule in local timezone

Implement column level security out of the box

Reusable/shared connection information for data models

Sanitise .smodel export

Idea for password rotation for a live model connection

Datasets UI

Make user parameters more dynamic

Update server (data source) across all dashboards (e.g. after restoring db to a new server)

Ability to create a Hierarchy tree filter

Validate a cube's design before building

Ability to create hyrbid data model live + elastic cube

Please allow arranging/sorting Shared Formulas in the selection dialog

Dependencies for Tables (Datasets)

Filtered Perspectives

Enable cube refresh on specific tables only

Copying a Perspective

AWS EMR Connector

Priority filter for faster livemodel