Three Questions to Ask Before You Start Collecting Data

by Joyce Lee-Ibarra, JLI Consulting LLC

Regardless of the sector you work in, you’ve probably been bombarded in recent years with messages about the importance of data. Whether it’s talk of “big data” in the business world, predictive data analytics in the public sector, or data-informed program strategy among nonprofits, data collection and analysis are increasingly seen as requirements for savvy organizations.

As our reliance on technology continues to grow, you might be feeling a sense of urgency about jumping on the data train. Identifying the right tools for data collection suddenly feels critical to your company’s success. And analyzing the data you’ve already been collecting feels imperative. But where to begin? Should we be using customer satisfaction surveys?, you might ask. Maybe we need to dig into our employee benefits data to better understand how to cut costs. Or perhaps what we really need is to upgrade to Salesforce from our current Excel spreadsheets, so we can better crunch our numbers. So many options to consider…

But wait! Before you fall down the data collection rabbit hole, hit the “pause” button. As with many things in life, the siren song of tactics and bright shiny objects can often lure us away from more critical considerations of strategy. Here are three key questions your organizations can ask itself before it embarks on data collection and analysis:

Photo by Jamie Street on Unsplash

What are we hoping to learn?

Perhaps the most important question any organization can ask itself prior to a data endeavor is, What is our purpose? Asked another way: What are we hoping to learn — or what decisions are we seeking to inform — from the data we plan to collect or analyze? Having a clear purpose provides a north star to guide your organization’s efforts around data. Clarity of purpose is key, too, for engaging staff and supporters in your efforts, and helping them understand why human and financial resources are being tapped for that purpose. If data collection is purely pro forma or undertaken for compliance reasons only, buy-in and investment from staff and stakeholders will be lukewarm at best.

In my own experience assisting nonprofits with their data efforts, the act of clarifying a learning purpose can be transformational. A healthcare organization I advised had grown accustomed to collecting what was essentially monitoring data — number of clients served, number of patient education classes offered, etc. While this data was helpful in that it tracked changes in these quantities over time, it did little to spark service improvements or identify the “why” of observed changes. Taking time to pinpoint what they fundamentally hoped to learn — namely, if and how their services were improving quality of life for patients and their families — helped the organization’s leadership and program staff reimagine the role of data and what it might make possible for their clients.

What questions will best guide our learning?

Baseball great Yogi Berra famously said, “If you don’t know where you’re going, you’ll wind up somewhere else.” Identifying one or two key questions to drive your data collection and analysis is a useful way of pinpointing your learning destination, helping you avoid wayward side trips. Spending time formulating targeted, open-ended questions — rather than yes-or-no questions, or ones that can be answered with simple quantitative answers — tend to be a better investment in an organization’s learning. Certainly, “output” questions may be answered along the way, but many organizations realize that closed-ended questions are typically not enough to drive organizational improvement and learning.

I have found NTEN’s Getting Started with Data-Driven Decision Making: A Workbook to be a useful resource for exploring an organization’s key questions, which they refer to as “action questions.” The workbook does an excellent job of not only walking the reader through a step-by-step process of defining your action question, but also distinguishing Goldilocks questions (not too broad, not too narrow — just right). It then guides considerations of what questions are most meaningful within your organization, how to choose data metrics that are useful and reasonable to collect, and how to define a process for using collected data to inform decisions. And although it is written with nonprofits in mind, its fundamental approach is transferrable to a wide array of organizations.

What is our plan for sharing what we learn?

One of the responsibilities of collecting and analyzing data is having a solid plan in place for sharing the knowledge you gain. Particularly when stakeholders’ input has been part of the data collection process (e.g., through surveys, focus groups, or interviews), doing so not only honors people’s contributions, but also closes the loop and confirms that their perspectives have not gotten lost in the abyss. Even when data collection and analysis are passive or more quantitative in nature, share-outs help foster a culture of transparency and shared learning, and reinforce the aforementioned purpose of data efforts.

In my own experience conducting community research on the performing arts sector on Hawaii Island, plans for public share-outs sessions were discussed early in the scoping process with my client. These share-outs provided the 600+ community members who completed to an online survey and the several dozen focus group participants an opportunity to understand the results of the collected data in sum. Sharing the data findings also generated broader discussion about community priorities for the performing arts on the island, one of the client’s key goals for the project. While not all data collection and analysis will lend themselves to something similar, this example underscores the importance of developing a clear plan to communicate what’s been learned, in a format that is sensible for your target audience.

Ultimately, honing your organization’s learning purpose and plans for sharing knowledge helps ensure that your data — and not the tools, processes, or analysis attached to it — remains centered on the people and places you are seeking to serve. And isn’t that far more important that securing the newest bright, shiny data object?

Joyce Lee-Ibarra is principal of JLI Consulting LLC and a consultant to the Hawaii Data Collaborative.