What is Data Validation Testing?
This guide describes how to use a sampling approach to test expected results against actuals and how to investigate cases where the two are not in agreement.
Let’s begin!
Prepare Expected Resultset
For each dashboard, start by creating a set of expected data points to check again actual dashboard results across various data security and filter scenarios. To get you started, we have created this template for you to fill out.
Conduct Scenario Tests
Next, test each expected result. No mismatches? Great! Time to move on to the next dashboard.
Investigate Failures
To identify the cause of each discrepancy, start at the dashboard and work your way towards the data source.
1. Review the Formula
Check each formula in the formula editor to ensure each KPI is calculated appropriately. Make sure to check the table and column source as well as the formula.
2. Check filters
What dashboard, widget, or measured value filters are being applied?Are the filters coming from the appropriate tables?
3. Check Data Security
Are there any unexpected data security rules applied to the testing account? The Visualize Widget’s Queries Add-On can help show you what filters and data security rules are being applied.
Haven't identified the source of the discrepancy yet? The next stop is the elasticube or live model.
4. Check Latest Build
If you are validating a dashboard using an elasticube, when was the last successful build? If the data is not up-to-date, this may be the cause of the discrepancy.
5. Check Relationships
Confirm there are no many-to-many relationships between tables and that they are joined on the appropriate keys. Double-check that the elasticube is also the appropriate source. You can use the Visualize Widget’s Queries to identify many-to-many relationships as well.
6. Recreate Joins
Is more than one table used to generate the KPI? Create a custom table with inner joins to confirm that the relationship doesn’t drop records unexpectedly. Use the Visualize Widget’s Queries visual to help guide your query. Begin from the Dimension tables or where filters are applied, and then join to the Fact tables.
7. Check Custom SQL
If you are using any custom SQL, confirm that the logic is accurate and return expected results. Are lookup functions using the right keys and returning results? What about CASE statements?
8. Confirm Records
Using Preview, check that records in the elasticube exist (or would be rendered based on the live connection’s table definition).
9. Review Import Query
In addition to confirming the database and schema of the import query's source, carefully review the logic in each custom import query. Confirm that you are using the same logic to generate the expected results.
10. Confirm Elasticube Source
Double-check to make sure you are sourcing the dashboard results from the same data source you have used to generate your expected results. For example, if you are using a SQL database for the elasticube or live model, make sure the schemas used are the same.
Lastly, check the source itself. Review the queries or processes used to generate the expected results for any cause of the discrepancies.
