The paranoia of confronting a future with AI hasn’t gone away for UX researchers. It’s just shifted from “Will AI replace me?” to “How do I learn this AI thing so I don’t get left in the lurch?”
If you want to hop on the AI express, we’ve got just the ticket.
Our CEO Prayag sat down with our Chief Product Officer Chirag Narula to discuss effectively using AI in research. Marvin’s co-founders shared their thoughts and advice on research tasks you can automate using off-the-shelf, freely available AI technologies. All aboard!
Here’s a tl;dr summary of what you’ll learn about AI in research below:
- Using AI as your Research Assistant
- Parts of the research process where you can use AI
- Preparation
- Field Work
- Analysis
- Crafting the perfect AI prompt
Watch the entire conversation about AI in research with Marvin’s co-founders.
AI as a Research Assistant
If you’ve tied your shoelaces with a loop, swoop and pull all your life, it’d sure be difficult to learn a new method now.
Once humans learn how to perform a task, it’s borderline impossible to reprogram the way we think about it. Researchers who’ve historically carried out tasks manually, will undoubtedly be wary of using and depending on AI in their research.
Chirag alleviated any lingering AI-related doubts: “It’s a research assistant — it’s meant to make research more efficient. It doesn’t mean it’s going to take your job — it’ll help you do more projects in less time,” he said.
Rather than replacing junior researchers, AI can automate tedious tasks and give them more time to focus on analysis and exploratory research. This undoubtedly closes the skill gap between them and senior researchers.
Let AI help you tackle the beast of burden that is qualitative research. It all boils down to when, i.e., at what stage, you can leverage the abilities of AI in your research. To understand the when of it all, we must establish the research process.
The Research Process
Chirag separated the research process into three broad categories, and shared some research activities that AI can assist with:
| Preparation | Field Work | Post-Game Analysis |
| Ideation & Lit Review Question Guides | Transcripts Notes Synthetic Users | Coding |
DISCLAIMER: This is by no means an exhaustive list of tasks that researchers undertake. It covers some crucial areas where you can leverage AI. If we’ve missed something important to you, drop us a line!
Depending on what phase you find yourself in, there are plenty of opportunities to incorporate AI into your workflow.
Using AI in Prep Work
Ideation and Lit Review
All research begins with a question, or a goal. And for the time being, that responsibility rests solely on the shoulders of researchers.
Where does the idea come from? Researchers perform due diligence to find the question they want answers to. They conduct Lit Reviews, scouring research papers from the past to identify any opportunities to create a hypothesis. Traditionally, lit reviews and peer reviews have been conducted by individually scanning each research paper one by one.
AI seeks to automate this process — studies testing AI during the document screening process show promise. While they cannot replicate the entirety of the screening process, they can certainly expedite proceedings. The authors express their opinion that given the right parameters, AI screening can become a helpful and valuable tool for researchers.
Don’t waste time going extensively through research papers and journals — use AI to summarize them for you. Assembling summaries of hundreds of papers can help you (quickly) identify gaps in existing research, and hone in on the right research question with which to begin your scientific inquiry. Ask AI to interrogate and summarize highlights of research papers and articles.
Bounce ideas off ChatGPT or Bard to get a new perspective.
“What are you trying to prove or not prove? What are the business goals in and out of context? If you don’t give complete context, you’re not going to get good solutions,” Chirag said.
For junior researchers and people new to the field, Chirag suggested using AI to suggest or validate your methodology. You may even ask AI what methodologies to use in the first place.
Use AI to Iterate Question Guides
The iterative nature of AI is invaluable while drafting questions for your discussion guides.
Ask AI to critique your survey questions — are they open-ended enough? Is it a leading question? How would AI rephrase it? Chirag suggested using AI to trim the fat from your interview questions. Use prompts like “Trim down these 20 questions so that people can answer in 15 minutes.”
Repeatedly interacting with AI also lets you be more iterative in your design. Say you are designing a button — ask AI for 10 ideas on what to name your button. If you’re not happy with the list, ask for 10 more. Pester AI with questions until you get the response you’re happy with.
(Editor’s Note: Our marketing team does this with blog headlines! The headline you see above is the result of three different iterations, using input from two different AI tools.)
Using AI in the Field
Transcription & Notetaking with AI
Turns out, AI has replaced one key role in the research process — human transcription.
Note taking has changed. No longer do you have to hang on someone’s every word, trying to capture exactly what they say. Use an AI note taker to record and transcribe every word. It frees up your time and mindspace for you to concentrate on supplementing your transcript with deeper thoughts and insights.
Transcription also improves data collection and organization — easily search through transcripts with accuracy and fly through pages of interviews while analyzing your data (more below).
This one’s an absolute must-have as part of your AI research toolkit.
Synthetic Users
Not all studies are made equal. Some have bigger resources than others, and smaller studies can struggle to extrapolate their findings to a wider population. Synthetic users (exactly what it sounds like) are AI-enabled bots who can help bring minor studies to the statistically significant level.
We don’t love this concept (you’ve likely heard our views on ethical AI in the past)… but we addressed it anyway.
Chirag expressed some concern over the state of AI and whether it can really simulate responses to questions with human-like accuracy. He’s worried the results conform to stereotypes and are biased in nature incapable of wholly capturing the socioeconomic and psychological state of a human. While it’s a good starting off point from which to gain some understanding of users, it’s important to stay mindful when asking AI to simulate answers or responses.
“Be assured, you’re not going to get the best result from it — take it with a pinch of salt,” he said.
The Library at Northwestern University assembled this informative guide to help you understand the applications of using generative AI, and importantly its limitations. The fact that LLMs are trained on large and inherently biased datasets is a serious drawback, for instance.
Using AI during Analysis
Post data collection is where a researcher or analyst’s work really begins. Sit back and let AI perform a preliminary analysis on your data. Its capacity to analyze large datasets with a quick turnaround time, means you can process more data and uncover insights you otherwise would have missed. It’s the perfect starting point for beginning your analysis.
AI and Coding
We’ve previously touched upon the importance of tagging or coding data — identifying patterns and themes from a dataset is a researcher’s craft.
Researchers from MIT and Kozminski University Poland explored how to conduct qualitative analysis without data coding. They use several free AI tools to scrape data off the web and perform sentiment analysis using quantitative methodology. All without writing a single line of programming code.
There are some things that AI just can’t do by itself. Deloitte’s study illustrates why coding must be conducted by a human being.
“It’s different from how a human would conduct thematic analysis,” Chirag said.
When tasked with coding qualitative data, a machine’s output was produced infinitely faster than a human being. However, the machine failed to unearth complex themes and make sensible connections.
While humans coding data cannot be replaced entirely, you can leverage AI to assist with some part of the process. Check out CODY, a ML tool designed to help qualitative researchers code their data. Establish rules and logic so the system learns, enabling a smoother coding experience.
Incorporating AI into preliminary qualitative analysis reduces man hours, allowing for voluminous data processing and informs decision making.
Watch more highlights from our co-founders’ fireside chat
Perfecting the Prompt
Make way, data scientists. You’ve been dethroned as the most sought-after roles on the job market. Enter, prompt engineers — professionals hired to train AI tools to deliver a specific output with accurate responses. They need not be engineers or computer scientists, and they can make up to $335,000 per year.
Before going gung-ho and quitting your job to become a prompt engineer, it’s imperative to understand the time, effort and skill that goes into creating that perfect prompt.
With AI at such a nascent stage in its development, people are scrambling to get their hands on all the information they possibly can. There aren’t any go-to resources that all-of-a-sudden will make you an AI expert overnight. Prompts are perfected over time, with trial and error; prompt engineers wrestle with this reality every day. Imagine spending a week writing just one prompt, over and over again. Rain check on those grand quitting plans?
Without putting too much of a damper on proceedings, here are some helpful resources to get you started:
The CLEAR Framework
Leo S. Lo of the University of New Mexico created CLEAR – a framework for drafting a successful prompt. Lo’s detailed article is not available for public use, but Mike Wolfe summarized the framework. We took the best of both resources and added insight from our own research to articulate how to interact with Large Language Models (LLMs):
- CONCISE – Keep your prompts clear, direct and concise. Avoid ambiguity. Be very deliberate with your word choice – if any one word isn’t clarifying your content, then it’s confusing the AI bot. Use declarative sentences to provide context and imperative sentences while asking something of the bot.
- LOGICAL – Begin with a clear goal. What does your final output look like? Provide clear instructions with easy-to-follow steps. Give the LLM some context like persona (who the output is coming from), the task to be carried out and the format and tone of the output. Your output is only as good as your input.
- EXPLICIT – Don’t assume that the LLM knows what you’re asking about. Interact with it like you would a child. Don’t leave anything up for interpretation. If you’ve dabbled with generative AI already, you must be aware that if you ask the same question several times, it gives you a new answer every time. Be specific in your instructions and use keywords to refine your queries.
- ADAPTIVE – Don’t settle for the first response or output you get. Provide feedback to the model – it remembers and learns from past prompts and data points. Leverage this to modify your instruction until you get the answers you desire.
- REFLECTIVE – Each prompt teaches you something about how to draft the next one. Think about each interaction you’ve had with an LLM. What worked? What didn’t? Much like qualitative research, creating a prompt is an iterative process.
Prompt Engineering Resources
Not satisfied with just one framework? We’ve got you covered. These helpful additional resources will help you master the art of the right prompt:
P.S. At Marvin, we love a good acronym. We’ve coined a few of our own:
- SAAS ASPIRE: From our informative guide on the UX Research Repository Landscape
- POINT CHASE: From AI Note-Takers: Your Shortcut to Smarter UX Research
Get Started with AI in Research
Simple parting advice from Chirag:
“Give it a try — play around with AI. Try with a smaller data set and you’ll realize how tricky it can be with bigger data. (You’ll understand) how to write a better prompt, what is good context and what isn’t. It’s just about trying out a lot of stuff,” he said.
Prayag is bullish on AI.
“It’s never going to replace researchers, but as an AI assistant, it’ll definitely bring more joy to your work,” he said.
Photo by Konstantin Planinski on Unsplash
