It’s the Journey, and the Destination: The Never-Ending Data Story
Article Highlights:
After collecting data, it is tempting to jump right to analysis in search of an answer or decision to your research question.
There is an opportunity to derive deeper understanding of your broader research problem by building on your results, regardless of whether or not they support your research question.
Throughout the series, we have emphasized and described a data-driven mindset that can ultimately help drive decisions that blend intuition and objective support (i.e. using data). As we’ve laid out, this includes understanding the limitations of intuition, mitigating biases with a structured approach to examining our assumptions, and finally evaluating the quality of the data we use to test our intuition.
We understand that with data in hand, it is tempting to jump right into analysis to arrive at some insights to shape your decisions. In this blog post, we offer a different perspective and opportunity to make sense of our research question and contextualize our results.
Imagine the ideal scenario—you’re presented with data that supports your hunch. Although it may appear that your initial research question[1] is supported, you should ask yourself: What else have I learned along the way? What additional data can I collect to better shape my understanding of the broader problem? And Would further analysis ultimately enhance the decision I wish to make?
Similarly, in the not-so-ideal scenario where your data doesn’t support your hunch, do you stop there? You might, but there is a lot of learning that comes from results you do not expect. You may challenge yourself and ask: are my assumptions about this problem flawed? What else have I learned along the way? What factors might I have overlooked?
We recognize that taking this time to examine your results in the context of your research question, and broader research problem, is not always possible for leaders who need to make urgent decisions. However, the key is to lean into the ambiguity and understand that regardless of the result of an analysis, leveraging data to make decisions is an ongoing process. At each critical point—including the moment from examining assumptions, evaluating data, and interpreting results—there are opportunities to pause and get ‘feedback’ about how we understand the problem and the factors we perceive to be relevant (e.g. the data source, the data itself).
As implied in the image below, we are describing an iterative rather than linear progression of how our understanding of a problem evolves over time. Think of A, B, C, and D as points in a research process (i.e. identifying the problem, determining the data you need, etc) - at each point, you will learn more about the broader research problem and have the opportunity to incorporate new information into your process thus arriving at a more informed decision.
As we adopt a data-driven mindset, at each step it is important to resist the urge to look at our progress in understanding of a problem as complete. Instead, we should consider the value of iterations along the way to ‘fill in the blanks’ that lead to a more complete understanding of the broader problem.
As we have discussed throughout this series, adopting a data-driven mindset can be helpful in bolstering intuition by combining foresight with an objective process for guarding against biases. However, even more value is realized when organizations have the enabling environments for elevating data into the decision-making process. Our next post will discuss those critical enabling factors.
[1] In this blog series, we use two similar but differentiated terms: 'research problem' and 'research question'. The research problem is the broader challenge. For example, homelessness. The research question, on the other hand, carves out a specific part of the research problem and frames it for exploration: Does an underperforming economy lead to homelessness?
This is a piece in Hawai‘i Data Collaborative’s Data for Good Decisions Series. The purpose of this series is to showcase how to elevate data into important policy and social change decisions to solve challenging problems.
