Collecting qualitative data isn’t the beginning of a researcher’s journey. The real work starts before carrying out a user interview.
Data collection and tagging should go hand-in-hand to set your research up for success. With a little advance preparation, you can save time and reach insights faster.
First, we break down how important tagging is to analyzing qualitative research. We then run through a real-life example of how to effectively tag with a user research tool to conduct a quick and efficient analysis.
The faster you analyze your data, the faster you gain valuable insights about your users.
The User Researcher’s Guide to Analyzing Qualitative Data
Here’s what you can expect to learn:
- Intro to Qualitative Research
- Why Should I Tag My Research Data?
- How to Analyze Your Qualitative Data
- Draft a Research Discussion Guide
- Tagging, Coding or Labeling During Your User Interviews
- Use Graphs to Visualize Your Qualitative Data
- Build Narratives with Your Research Insights
- Create a Report & Share Your User Research Findings
- Data Analysis Tools in Qualitative Research
Introduction to Qualitative Data Analysis
Humans have been counting, performing calculations and using quantitative methodology for thousands of years. Its qualitative counterpart only took off with the advent of the social sciences in the mid 1900’s, but is no less scientific. Today, a large amount of policy making depends on qualitative research.
Quantitative data analysis tests hypotheses:
“Design A is rated more favorably among Management Consultants than Design B.”
Qualitative data analysis is used to understand our environment better, and provides a deeper understanding of what we observe. Asking the same participants:
“What do you think of the design and experience of a research tool?”
Using data, qualitative analysis uses scientific rigor to build theories. These theories must be repeatable — multiple people looking at the same data must come to similar conclusions about it. This adds credibility to the qualitative research study and the narratives we uncover from the data. Crucial to conducting effective qualitative research is tagging.
Why Should I Tag My Research Data?
Tags are the building blocks of qualitative research.
So what is tagging? Tagging (also known as “coding” or “labeling,” which we will use interchangeably throughout the article) essentially breaks down large chunks of qualitative data into smaller observations.
Tags help us turn unstructured data into semi-structured data, taking large and complex datasets and turning them into more manageable pieces. By tagging data, we provide context, delving deeper into understanding and analyzing these bite-sized observations.
Tagging in research has several benefits.
Tags make research data:
- Analyzable – easy to look for patterns; tease out and report these patterns to build out narratives to make decisions. The science we alluded to earlier lies in extracting these patterns and narratives.
- Searchable & Discoverable – even if you are not directly involved in the study, you can always look at the data at any point in the future. Tagging gives us the opportunity to quickly slice and dice data and tie together observations from various different places. This helps broaden our research perspective.
- Shareable – by making observations accessible to your peers, you elevate the voice of the user and create user empathy across your organization. This also allows people to challenge your assumptions and builds confidence in research results.
Let’s sink our teeth into a real-life example of how to use tagging to conduct quick and effective qualitative research.
During the testing of our product (we promise this won’t get too meta), we interviewed several Management Consultants about their expectations and challenges using qualitative data collection tools. This is how we approached it.
[DISCLAIMER: The following is a suggested method for conducting quick user research in Marvin. To each their own if you prefer to conduct your research with other qualitative data analysis tools. But we quite like our own tool, which was built with advanced AI features.]
How to Analyze Your Qualitative Data
We suggest you start your qualitative data analysis with the output first.
Sounds odd, doesn’t it? All scientific inquiry begins with a question (or questions). You don’t necessarily have to have a well-formed hypothesis when you begin your qualitative research — it can be unclear. However, when you craft questions, you’ll have a ballpark idea of potential answers that may emerge from your research.
Going into our example study, we knew certain challenges would be more common for consultants than other professionals. We had no idea of this output ahead of time.
We also knew that certain features would be more useful to consultants. Here, we knew the features we had built. For this output, we were able to anticipate some answers that we might receive — this helped us to prepare the data before beginning our study.
We suggest thinking deeply about the output you have in mind. The more thought you put into how your project will be structured, i.e. the questions you are going to ask and the output that you may get, the easier it becomes during analysis.
Draft a Discussion Guide for User Interviews
Good output comes from good input. A discussion guide is a set of questions, tasks or topics that a researcher will address during the research process. These topics enable us to better understand participants’ experience and help guide research sessions. A discussion guide isn’t simply a set of questions, it’s also foundational to making the tagging or coding process easier along the way.
The best discussion guides move from easy, specific questions to difficult, higher level ones. Guides help facilitate conversations and act as a reference tool for the researcher. Their purpose is to keep the research focused on the output we mentioned earlier.
Example of How to Use a Research Discussion Guide
Q: If I already have a hypothesis in mind, how do I create questions around it?
A: Turning a hypothesis into questions is an art form, whereby you’re not trying to expose too much of that hypothesis to your participants. Validating or invalidating the hypothesis is your job, not the participants. So, ask questions that go into the context of the hypothesis, not the actual hypothesis itself.
Our hypothesis: Our AI Note Taking app is very powerful and will resonate with our users.
We showed participants a design with and without “Live Notes” and asked the following questions:
- What is your process like?
- Do you take notes during a conversation?
- Does this design make your workflow easier?
Tagging, Coding or Labeling During Your User Interviews
Pre-Interview Prep
Tags, codes, or labels are a powerful and flexible tool that make qualitative analysis more useful.
One of our biggest time savers? Tag your questions!
Starting with questions gives you a launch-off point on how to group and categorize your labels. From your discussion guide, you can start to build out labels in advance that you know will aid your analysis.
During our initial research, we coded our labels with the following rules (we lied about going meta):
- Use foundational labels based on overarching topics or subtopics. “Goal,” “Background” and “Process” are labels we knew would come in handy during our interviews.
- Use structured and semi-structured questions as labels in advance. We showed our participants several designs and asked them to provide feedback with responses on a 5-point Likert scale. So the question we tagged was “Does this design make your workflow easier?” and the relevant labels were “Likert 1” “Likert 2” “Likert 3” “Likert 4” and “Likert 5.” We also included a “Likert Other” label to capture potential outlier feedback.
- Create a simple label for each user persona to help with mapping each user interview. For example, we wanted to tag any mention of our partners so we included that specific persona as a label in our codebook.
- Establish and build a label taxonomy and hierarchy. To build on the user persona example, we built parent and child relationships so we could get super-granular and view data based on which partner the interviewee was talking about. The parent = Partner Persona; the child = Partner Company Name.
We suggest creating templates to make life easier and save your teammates precious time in future projects. Many modern qualitative data analysis tools offer this functionality to help you save time and effort. (Surprise! HeyMarvin is one of them.)
What are User Research Templates?
Templates create standardization across the company and make label taxonomy easy to import into a project. One of the best reasons to use a template is when you have recurring interview questions / responses across interviews and projects.
“More is greater than less” — this rather obvious statement applies to labeling, especially early on. At the beginning of your project, use as many labels as you can. Labels are a flexible and powerful tool that you should continue to modify and refine as you dig deeper into your analysis.
Remember to treat your label hierarchy and codebook as living documents. You will likely need to update and refine your tagging taxonomy as you gather more data and gain deeper insights into the themes and concepts you are exploring.
Tagging Live during Interviews
For every hour of interview recorded, a qualitative researcher spends 3-7 hours analyzing and coding the data afterward.
This is another reason UX researchers are turning to qualitative data analysis tools to automate the heavy lifting. One example? Using a verbatim transcript of every interview that’s easily downloadable in a time-stamped text format. No more manual text transcription or coding after the fact.
Marvin’s Live Notes is a collaborative note-taking functionality that encourages researchers to annotate a transcript in real time, alongside their peers. We spoke earlier about adding context to your observations – this is where the magic happens! Live Notes frees up your time to add key details right when your interviewee says something important.
Steps for Analyzing Research Data in Real Time
Add a note, a label, tag a question from the discussion guide, or simply add a bookmark at key moments during your interview.
You can create new questions and labels or lean on the existing library you’ve already created. Revisit these moments to annotate and add notes later. Every note you make will be appropriately time-stamped according to when you created it during the user interview.
Finding and maintaining consistency in your labels can be confusing while you’re leading a user interview. A significant amount of tagging can be done in real time by tagging questions.
Using the discussion guide (we told you it was worth it) is an easy way to tag questions at the beginning without having to revisit the data frequently.
Tagging with Live Notes helps organize your data for analysis. And even better — it saves you from rewatching hours of footage to remember the context.
How to Analyze Qualitative Data After the User interview
Traditionally, researchers needed to revisit an interview recording multiple times before coding any data. This might still be the case if you don’t have access to data collection and analysis software.
But if you followed the previous steps, you’re already ahead of the curve!
There are two methods for analyzing your qualitative data.
You may prefer to interact with and label one user interview at a time. You can highlight parts of the transcript and add your labels, questions and notes to the appropriate location.
This method enables the researcher to conduct an in-depth review of individual recordings, but isn’t necessarily time-efficient. Personally, we use this view to look for new tags that haven’t been created in advance.
We recommend starting from a view that showcases all your labels, questions and notes in one spot. You can do this with a spreadsheet if you don’t have access to a user research analysis tool yet.
Luckily, the team at Marvin did have a solid user interview tool. And thank goodness, because going into 377 notes would have been incredibly daunting.
Coding Your User Interview Questions
Remember when we said “Tag your questions”? Here’s where the fruits of our labor emerged.
Marvin categorized the 377 notes within our question tags, which gave us an immediate clean, global view of all our user interview data. We then had a data playground to tinker with existing labels and refine our taxonomy as we looked through the material one topic at a time.
Tagging user interview questions at the outset is helpful as it makes it easy to go through the data without having to go through entire recordings.
Refining Your Tags
We added a new label called “Recommendation” after reviewing a few different participant responses.
We then went back to our codebook and nested the newly created “Recommendation” label under the foundational “Goal” label.
This interplay between Labels and Analyze sections is responsible for the bulk of our qualitative data analysis. Looking at data through the lens of questions gives us perspective on how to create and group labels together and establish hierarchies and relationships between labels.
Going question-by-question is a systematic way to deepen our understanding of the data. As you go through the questions, look out for recurrences of old patterns and potential new patterns as well.
Start with certain labels and as you go through the data, you will re-use and refine old ones. This constant back and forth establishes consistency in your approach to labeling data.
We recommend starting on this step after your second or third interview. By then, you should already have a substantial enough dataset to begin your analysis. Plus the user interviews will still be fresh in your mind.
Use Graphs to Visualize Your Qualitative Data
Once you begin building relationships and tagging your data, resume thinking about the output of your study. Graphs aren’t used exclusively in quantitative research – they’re a great way to visualize your qualitative data.
Numbers tell stories (even when they aren’t representing statistical significance). Viewing your data in a graphical representation can help crystallize your thinking so you can build out narratives.
People can quickly look at graphs and arrive at conclusions about answer distributions. Our graph showed us that the most used tools and methodologies are qualitative interviews and data visualization.
Quantify Your Qualitative Research with Data Visualizations
Overlapping labels can help build overarching narratives for your project. In our case, we were wondering which of the designs that we showed participants would resonate the most with them and make their lives easier. We’re interested in a distribution of results, charting which design features performed best.
We graphed by labels and selected “Strongly Agree” from the Likert scale responses. This is because we only wanted to see the responses that were overwhelmingly positive. We also checked all “Product Feature” tags because we want to see which performed best. The “Relation between labels” operator AND is selected to signify that both conditions must be met.
After clicking through, the graph clearly showed us:
Among Likert scale responses of 5 (Strongly agree), the Transcription tool seems to be most popular in terms of making consultants’ lives easier.
There are infinite ways to slice the data, but keeping your output in mind and viewing results through a distribution lens helps build out narratives.
Reader, beware — you should aim at getting a minimum amount of data before drawing any conclusions or telling any stories with your data. On the bright side, 6-8 interviews is generally enough to find valuable insights that apply to your customer base.
Build a Narrative with Your Research Insights
Once you’ve begun analyzing the data, you’ll start to see potential narratives. This is the meat of qualitative analysis — building out these arguments or narratives and validating them with data points.
If you see unifying themes across interviews, you can tie them together to make your point hit home. Highlight reels are a common way for researchers to elevate participant quotes and provide reasons why they reach a certain conclusion.
Marvin’s Research Narrative: “Live Notes is a generally positive feature that management consultants like to use.”
We then created a highlight reel of the participants who shared positive feedback about the feature.
This playlist became a shareable artifact that validates our story and backs it up with underlying data:
- We immediately showed it to our whole company in Slack.
- We used it as a value prop when pitching our services to other consultants.
- We embedded it in PowerPoint and Notion for staff and board meetings.
- We posted it on social media to spotlight one of our favorite product capabilities.
The more we share our research findings, the more value our audience gets.
This is true for our customers as well. The more they share their qualitative data insights, the more value their stakeholders place on the power of user research.
By using the playlist feature, we build proof points behind our narrative or recommendation.
Create a Report & Share Your User Research Findings
If you’re conducting research, you expect to create a report at the end. But it doesn’t have to be boring.
New data analysis software allow you to assimilate all your data and build an interactive report. If you’re creating graphs, playlists and clips throughout the analysis stage, you already have elements that truly highlight your customers’ voice.
- Playlists or highlight reels are fundamental to your recommendations — embed playlists throughout the report to lend credibility to your claim.
- Add graphs to your report to give it some numerical nous. Simply take a screenshot of a graph and paste it directly into the document.
- Copy transcript quotes within the report to make it more memorable for the reader
Write reports to share your narrative (backed by source data) with teams so they can use it to build out their own narratives or help you clarify your own. If someone looks through your report and disagrees with some of your assumptions, it sparks healthy debate.
This is a best practice of qualitative research and helps build confidence in the conclusions you’ve made. Multiple pairs of eyes looking at the data, challenging your conclusions and assumptions, with you defending your stance, helps your whole company view the data through different lenses.
All of which is important in building the best possible product for your users.
Data Analysis Tools in Qualitative Research
It’s nearly impossible to code your research without a certain set of data collection tools.
Here are four basics you need:
- Recording software. If you’re hosting user interviews through videoconferencing technology, hit record and capture the raw feedback.
- Transcription tools. There’s good automated technology for transcription now. Save your time (and sanity), and use it.
- Research repository. Make sure everyone on your team is storing their recordings in an easy-to-search, easy-to-share location. (Honestly? We love Google Drive. Your stakeholder probably already know how it works!)
- Codebook or tagging taxonomy. Spreadsheets are fine, just make sure you’re documenting your labels somewhere central.
Once you’ve established this data collection framework, you’re ready for the data analysis and collaboration tools in your research toolbox.
You may have guessed by now… we have a preferred qualitative data analysis tool. And we’d love to show you HeyMarvin when you’re ready.
But more than anything, we want you to have access to the tools and best practices to prove the value of user research to your stakeholders. Once they’re on board, they’ll be your biggest champions for qualitative data analysis.
Photo by Abigail Lynn on Unsplash
