I wanted to thank you all for joining the cohort and participating at any level. I know everyone has many obligations pulling them in different directions, and I’m thankful for the time we got to spend together during the cohort! Also, a huge thank you to Sofia and Jacqueline for making the cohort happen. It was great to meet all of you, and I’m looking forward to seeing everyone around in the future! If you ever want to chat about writing, give feedback on the cohort experience, or just stay connected, you can connect with me here on LinkedIn, Substack, Twitter ... View more

The Importance of Thoughtful Design: A Prerequisite for Effective Dashboards When tackling a problem, people tend to become emotionally invested in the solutions they come up with. The more time and effort they dedicate to a particular solution, the more they believe in its effectiveness, even if it may not actually address the core problem adequately. That's precisely why it is crucial to resist the temptation of rushing into creating a dashboard as a solution. We must avoid the creation of ineffective dashboards that won't serve their intended purpose. In my previous posts, I emphasized the importance of finding the right problem before jumping into crafting a solution and gave you the tools to do so. To ensure the success of dashboard design, it is essential to invest time in understanding why we need a dashboard in the first place. Exploring multiple ideas and possibilities before settling on a specific solution is key. The dashboard design process should begin by identifying our stakeholders and their decision-making needs. This understanding will guide us in determining the relevant metrics that support those decisions. The next step involves prototyping the dashboards using simple tools like pen and paper, obtaining feedback and iterating to refine the design. Only after confirming that the prototype aligns with the project's objectives should we proceed to gather the actual data and begin building the dashboard. Lastly, creating a useful dashboard requires sharing it effectively with the intended audience and maintaining it over time. By following this thoughtful approach to dashboard design, we can ensure that the end product becomes a valuable tool for its users. Dashboard Design Process We can summarize this process into four steps: Define Prototype Build Deploy Step 1: The Initial and Crucial Step: Defining Stakeholders and Metrics for the Dashboard The first and foremost step in dashboard creation is gaining absolute clarity regarding the intended audience and the metrics that hold significance to them. This clarity is paramount to ensure the dashboard's practicality and usability. Stakeholders: In this endeavor, four main stakeholders play crucial roles: The Designer (You) - Responsible for crafting the dashboard. The User(s) - Those who will access and utilize the dashboard. The Key-User - An experienced user who provides valuable insights and feedback. The Data Buddy - A team member offering assistance and support in dealing with data-related aspects. Metrics: Working in collaboration with the key-user, you will transition from identifying the decisions that need to be made to determining the metrics that can be effectively queried and tracked. This process necessitates a thorough back-and-forth exchange to distinguish between interesting but non-essential metrics and the mission-critical data required for making informed decisions. Step 2: Prototyping - Crafting a Useful Dashboard After identifying the metrics to be included in the dashboard, the next step is to determine the most effective way to present them to the audience for maximum usefulness. Visualizations: Select visualizations that convey the metrics clearly and accurately. Even during the sketching and prototyping phase, making the right visualization choices can significantly enhance the prototype and feedback process. Dashboards: Utilize best practices for assembling the chosen visualizations into a cohesive dashboard. The composition decisions might even lead you to reconsider your initial choice of optimal visualizations. Sketching and Iteration: At this stage, it is highly recommended to sketch out the visualizations and dashboards either on paper or using a tool like mokkup.ai. This approach allows you to swiftly discard any unsuitable ideas without concerns about time investment. Moreover, it enables you to concentrate on the design itself rather than being preoccupied with validating the accuracy of the numbers. Links: Mokkup.AI Step 3: Building the Dashboard: Transforming Prototype into Reality After finalizing and being content with the prototype, the next step involves bringing the dashboard to life using real data. Locating the Data: This stage can present various challenges. Questions such as where the data is stored, its cleanliness and its availability may arise. Collaborating closely with the development team and the Data Buddy becomes essential in navigating through this phase. Creating Metrics and Dashboard: To build the dashboard, we must generate queries to power the metrics, formulate necessary calculations and transform the data into visually informative charts. Utilizing a framework to log the metrics, formulas and data sources simplifies the process of creating queries and ensures a smoother development experience. Step 4: Deploying and Sustaining the Dashboard: Ensuring Widespread Impact After successfully creating a fully functional dashboard, the next crucial steps involve sharing it with the entire audience, enhancing its effectiveness at scale and ensuring its ongoing maintenance as usage evolves. Sharing: Given that users possess varying levels of data literacy and contextual understanding, it is imperative to ensure that the dashboard provides sufficient context within its design and offers adequate training resources. This approach enables people to easily derive valuable insights from the presented data. Scaling: As the dashboard gains popularity, the number of views and viewers is likely to increase. To accommodate this growth, it is beneficial to incorporate links, interactivity and comprehensive documentation, enabling the dashboard to serve a broader range of use cases and inspire other dashboard creators. Additionally, dedicating time to optimize queries becomes crucial in maintaining the dashboard's usefulness amidst growing demands. Maintenance: Datasources, tables and fields are subject to change over time, necessitating corresponding adjustments in the dashboard. Establishing scheduled review periods is vital in ensuring that the dashboard remains relevant and fully functional. Furthermore, providing a means for the audience to report issues allows for informed improvements and ensures the dashboard's ongoing effectiveness. ... View more

In the dynamic realm of modern business intelligence, real-time reporting has emerged as a pivotal force reshaping the landscape. While traditional approaches often involved generating static dashboards and reports linked to databases refreshed daily or less frequently, the advent of real-time reporting has opened new vistas of opportunity. While this approach remains important in many scenarios, there are instances where real-time insights can provide a competitive edge and drive better decision-making. Traditional vs. Real-Time Reporting Imagine a scenario in warehouse management where operational efficiency is paramount. Traditional reporting methods often fail to detect underutilized stations until the next day's data refresh. This delay in identifying performance issues can result in reduced productivity and missed opportunities for immediate corrective action. However, the adoption of real-time reporting using tools like Sisense introduces a paradigm shift in how businesses can harness the power of data. A Case Study in Warehouse Management To emphasize the impact of real-time reporting, consider this example that vividly demonstrates how our stakeholders from the Warehouse Management team stand to benefit significantly. In this case study, the focal point is the integration of real-time Sisense reporting within warehouse management operations. The data flow originates from warehouse management systems and ultimately populates a data lake that undergoes daily updates. This data forms the basis for constructing comprehensive Sisense dashboards that illuminate the operational performance of different warehouse stations, delving down into the granular performance of individual pickers. The drawback of relying solely on daily data refreshes becomes evident when considering the impact of underutilized stations. Traditionally, declining station performance might go unnoticed until the next day. For the sake of argument, if station xyz were to underperform by 1% at the 5th hour of a 12-hour shift, in a traditional set-up, this issue would only be picked up by the dashboard the following day due to the nature of data refresh. This means that by the end of the shift, there would be a 7% decline in the overall performance of the station. To mitigate this, the proposition to employ Sisense Live models gained traction. With live models and dashboards, underperforming stations can be detected immediately, thereby enabling prompt countermeasures. This approach prevents stations from underperforming for a prolonged period, preserving overall performance from dropping. Harnessing Sisense Live Models The implementation of real-time reporting involves leveraging Sisense Live models, which enable the creation of dashboards with minimal delays. To achieve this, a prerequisite is to have a data source providing real-time data. Once connected, Sisense's live model seamlessly integrates with the data source, enabling dynamic visualization of data in real-time. Additionally, Sisense allows widgets to be set to auto-refresh at specified intervals, making it possible to display these dashboards on factory floors for live monitoring. One key advantage of real-time reporting is enhancing situational awareness and enabling proactive decision-making. By displaying real-time dashboards on factory floors, supervisors and employees can monitor performance at a glance. Additionally, this live monitoring is further augmented by pulse alerts, which provide instant notifications for critical changes or anomalies in performance. For instance, when a station's efficiency drops below a certain threshold, it triggers an alert, enabling immediate intervention. Caching Periods and Sisense Live Models One major challenge we encountered is related to the caching period for Sisense Live Models. Currently, the caching period of live models is set for the whole instance, which removes flexibility when different caching periods are required, such as for exceptionally large models. For example, we have a table that must be ingested via a live connection as it is too large for ElastiCubes. However, we wouldn't want the caching period for this live connection to be too "low" as it would incur high costs through AWS. Conversely, a low caching period is desired for warehouse management low latency reporting, as explained above. We are exploring options and will update the thread once we have identified a solution. In conclusion, real-time reporting is revolutionizing the field of business intelligence, and the warehouse management case study highlights its tangible impact on operational efficiency and productivity. By leveraging tools like Sisense, businesses can transition from reactive to proactive decision-making, ensuring that performance issues are promptly identified and addressed. As the business landscape continues to evolve, real-time reporting stands as a crucial tool for those seeking to stay ahead in a fast-paced world. ... View more

If you torture the data long enough, it will confess to anything: How to avoid misrepresentation There is so much data you have access to within a company. Communicating accurate insights from company data is challenging. This post covers the mental biases and common mistakes that people make when analyzing data, namely Confirmation Bias, Selection Bias and Survivorship Bias Understanding Confirmation Bias Confirmation bias refers to the inclination to actively seek or interpret information that aligns with preexisting beliefs, often disregarding contradictory evidence. This bias tends to overlook opposing data, especially in a business setting where it can lead to ignoring indications that a feature, product, or business element isn't functioning optimally due to a focus on a single favorable metric. This phenomenon urges individuals to downplay negative signals while emphasizing positive aspects. How to Detect Confirmation Bias? There are three common signs that confirmation bias is influencing the data you are looking at. Only good news If the only news is good news confirmation bias is probably hiding some inconvenient metrics. Limited metrics reported Usually people present a few metrics because they are trying to be brief and that is desirable in most cases. Obscure metrics being used Many times, we cannot measure exactly what we want so we have to settle for proxy metrics. If people are confidently sharing convoluted proxy metrics, they are likely looking for ways to find positive signals in the data. Identifying Confirmation Bias Several indicators can signal the presence of confirmation bias in the analysis of data: Selective Positivity: If exclusively positive news is highlighted, it's likely that confirmation bias is obscuring less favorable metrics. Limited Metrics: Presenting only a handful of metrics could indicate an attempt to simplify, yet it might also be concealing contrary data. Complex Proxies: When intricate proxy metrics are touted with confidence, there might be an effort to extract positive signals from the data due to the absence of direct measurements. Counteracting Confirmation Bias To approach confirmation bias systematically, consider these strategies: 1.Academic Approach: Establish a peer review system where a fellow analyst scrutinizes your analysis before presentation. This ensures diverse perspectives and helps validate conclusions. Additionally, replicating tests within the company, if resources permit, can corroborate findings. 2.Third-Party Audit: Leverage data analysts to randomly assess shared analyses. If discrepancies or misleading visuals emerge, engage in a collaborative review with the creator. Once corrections are made, distribute updates to affected stakeholders. 3. Insurance Methodology: Define clear parameters for data claims, restricting certain causal phrases, for instance. Claims beyond established boundaries aren't approved for sharing, aligning with an "insurance" mindset within the company. 4. Cultural Approach: Highlight the company's mission as paramount. Emphasize that the mission's importance surpasses personal satisfaction with product or feature success. Acknowledge that mistakes are inevitable, and they serve as stepping stones for better decision-making in the future. By integrating these strategies, businesses can mitigate the influence of confirmation bias, fostering a more objective and holistic analysis of data. Understanding Selection Bias? Selection bias arises when the group chosen for analysis does not accurately represent the broader population under study. Acquiring an Unrepresentative Group Unintentionally 1. Convenience: This occurs when a group is chosen for analysis due to ease of measurement, leading to a non-representative sample. Example: When evaluating the value of a new feature, asking only three friends for their opinion may not reflect the broader customer base. Common Convenient Biases: Engaged customers Latest user cohort Specific geographic region 2. Self-Selection: This happens when individuals who voluntarily participate in analysis possess traits that differ from the entire population. Example: Distributing a product satisfaction survey could attract highly opinionated or time-wasting respondents, skewing results. Common Biased Self-Selectors: Extremely negative individuals Overly positive individuals Early adopters Power users Selection Bias in Business Context Suppose you’re launching a premium feature in your BI Tool. If you invite only the most active users to try it, their motivations might cloud the analysis: They might test all new features, irrespective of value. They might seek contact with your company. They might aim to suggest new features. Although well-intentioned, their feedback might mislead you. Remedies: Actively engage a representative sample for testing. Employ qualifying questions to select a balanced sample. If many early adopters respond, consider weighing their feedback against typical customers. Grasping Survivorship Bias These biased interpretations of events and data are prevalent within the business landscape. Imagine you’re overseeing a two-week trial phase for a recently launched dashboard/data product. At the midpoint of the trial, you observe only a few remaining active users. Let’s suppose that these users primarily consist of data analysts, and they are progressively generating more intricate analyses using your BI tool. Possible Conclusions and Their Caveats 1. My dashboard/data product resonates with data analysts (inferring a norm): Without investigating the users who ceased engagement with your dashboard, you lack insight into whether the initial trial group included data analysts. If a majority of those who started the trial were indeed data analysts but a larger portion discontinued usage, your initial conclusion would be contradicted. It’s imperative to delve into the non-surviving trial participants before drawing conclusive statements. Engaged data analyst users do not necessarily imply that your product resonates with all data analysts. To reach a more informed conclusion: Analyze the entire trial participant pool to unveil distinct patterns between engaged and non-engaged cohorts. Understanding the characteristics of both groups before deducing a norm is crucial. 2. My dashboard empowers deep analysis (inferring causality): Currently, you lack a point of comparison for your users’ capabilities. While they may be utilizing your dashboard to perform advanced tasks, their proficiency might transcend your product’s design. Their success could be due to their analytical skills rather than your dashboard’s influence. To reach a more informed conclusion: Pre-assess users’ skill levels to gauge the true value added by your product’s features. Compare their performance using other tools for similar tasks. If your product demonstrates clear benefits, you can leverage these results as a testament to its effectiveness. Key Takeaways Analyze the entire trial cohort, encompassing both engaged and non-engaged users. Gauge your product’s impact by comparing user behavior with competitor tools. Evaluate users’ skill levels before attributing success solely to your product. By adopting these approaches, you can mitigate survivorship bias and arrive at more accurate and nuanced conclusions about your product’s performance. ... View more

We're entering the last phase of this journey. Is there a topic you're exploring this week? @Priscillareagan @HamzaJ @Steven88 @jirazazabal ... View more