Is AI the existential threat we think it is, set to replace all human UX researchers? Or can designers and researchers harness and incorporate its mind-boggling capabilities into their work?
In 2023, global AI market revenue was $196.6 billion. Over the next five to ten years, that number is expected to grow at 37% annually. Or 31%. Or 19%, depending on your source.(1)(2)(3)
Our point is, AI’s influence is already BIG. And it’s only going to get BIGGER.
It’s forced a rethink of research and design roles, and the organizational design and dynamics that surround them.
In this article, we’ll explore fundamental concepts of AI and how it’s permeating the UX Research and Design fields.
TL;DR – Pros and Cons of AI in UX Research
AI is revolutionizing the UX research process. But of course, we have to be mindful of the dark side, too.
Here’s a quick glimpse at the pros and cons of AI in UX research:
| Pros | Cons |
| Optimize User Workflow | Baked-in Bias |
| Conduct Large Scale Analysis | Less Context |
| Consistent & Reliable | Lost Human Touch |
| Insights Discovery | Noisy Data |
| Increase Collaboration | Danger of Overreliance |
At Marvin, we’re bullish about the power of AI in your research workflows. Find out how to cut your time in half with an AI research assistant.
What is the Role of AI in UX Research?
Researchers, breathe easy. AI will NOT replace you.
It will only help you do your job better.
AI streamlines the research process. It enhances productivity by automating data collection, synthesis, and analysis. Additionally, AI expedites content generation and can surface predictive insights from large datasets.
Removing manual and cumbersome tasks frees up a researcher’s time. They can focus wholly on analysis and influence strategic decision-making.
Researchers must examine how AI augments their workflow. As with any new technology, it’s essential to understand its use cases — what it can or can’t do.
First, we recommend you become acquainted with various abbreviations associated with AI.
Here’s a quick explanation of the different types of technology:
Marvin’s Guide to AI Terminology
| Phrase | Abr. | Description |
| Artificial Intelligence | AI | Umbrella term for the simulation of human intelligence by a machine. Intelligence = perceiving, synthesizing and inferring information. |
| Machine Learning | ML | Subset of AI, ML uses algorithms that enable computers to learn patterns or make predictions based on a dataset. |
| Natural Language Processing | NLP | Type of AI that conducts interactions between humans and computers. Machines understand, interpret and generate human-esque responses. Initially trained with real-life data, they use pattern recognition to come up with responses. |
| Large Language Models | LLM | AI algorithm that uses language as their input and output. Trained on extremely large datasets to understand, summarize, generate and predict new content. |
| Generative AI | GenAI | Type of AI that can create images, video, audio, text and 3-D models. |
| Application Programming Interface | API | Software that facilitates communication between different applications. |
| Neural Network Model | NNM | AI model that teaches computers to process data like a human brain, so it can make decisions on its own. Unlike ML (which makes decisions based on what it learned from data). |
| Generative Pre-Trained Transformers | GPT | Series of neural network models that learn context and meaning by tracking relationships in sequential data. Think about how auto-complete knows what to write next in messaging apps. |
Key Applications of AI in User Research
You can use AI at every stage of the research process.
To name a few:
- Content / Idea Generation
- Automated Transcription
- Survey Generation
- User Testing & Behavioral Tracking
- Preliminary Analysis
AI research tools can also shed light on customer behavior and sentiment, and provide insight into the competitive landscape and market trends.
Let’s further examine how various AI technologies fit into the UX workflow.
Optimized Data Collection and Analysis
Two approaches to data collection here — passive and active. Active studies are conducted to answer a specific research question. Passive data collection involves collecting data on an ongoing basis.
To collect data from active research projects, Marvin’s AI uses NLP and LLM to transcribe your user interviews. Get a verbatim transcript within minutes. No more frantic note-taking.
Learn more about the differences between passive and active research.
Let’s talk about passive data collection. The prevalence of APIs means that apps (Google Analytics and the like) supply a continuous data stream to companies. Once in a research repository, ML helps researchers unearth trends and patterns from the dataset. AI tools for data analysis ensure that critical insights don’t go overlooked.
Researchers can use AI to conduct rapid analysis, processing volumes of data from complex datasets in real-time. The benefits are threefold:
- Aggregating and interpreting user data in real-time brings adaptability. In an ever-evolving digital landscape, it helps researchers unearth “in-the-moment” insights. These insights provide companies with feedback for on-the-fly decision-making, eliminating guesswork.
- AI’s ability to handle large volumes of data enables researchers to take on more extensive projects without increasing the time and resources spent.
- Another upside is scalability. AI allows researchers to meet increased data processing needs (previously impossible if conducted by humans). AI allows companies to ramp up their research efforts rapidly and efficiently.
AI can collect and analyze data round the clock for you — 24 hours a day, 7 days a week…you get the idea.
Learn more tips and tricks to incorporate the use of AI in your research.
Customized Personalization and User Experiences
Ever scrolled through a friend’s YouTube feed? You likely noticed it’s entirely different from yours. A prime example of a customized experience.
UX will always be centered around designing an experience that delights each and every user. Designers and researchers’ primary aim is to create tailored experiences that improve customer:
- Adoption
- Engagement
- Satisfaction
- Retention
Using data about how users interact with an application, AI can decipher individual user patterns and preferences. To craft well-rounded user personas, AI uses NLP to understand people’s attitudes and emotions from social media posts, forums, and customer reviews.
AI is capable of dynamically adapting interfaces and suggestions based on these user personas.
This opens up the door for more inclusivity and accessibility. Using AI-driven tools, researchers and designers can create applications and processes that cater to a more diverse user base, considering their abilities, backgrounds and general product use. UX researchers can then study the effectiveness of personalized features and iterate on design choices to improve user satisfaction.
AI will eventually tailor experiences to each individual user. This results in highly customized user experiences based on an individual’s preferences.
Imagine a day when ALL your apps look and feel completely different from someone else’s. With AI, that level of personalization is right around the corner.
Advanced AI for Research Analytics and Design
Ever wondered how Amazon knows exactly when you’re out of dishwasher tablets or detergent?
It’s down to predictive analytics.
Amazon’s AI examines your consumption and purchase history, and it uses ML to estimate when you’ll run out. That’s when it automatically sends you the eerie notification.
Predictive analytics uses statistical algorithms and ML techniques to enable a deeper understanding of user behavior. Designers can identify patterns in data and troubleshoot user issues or pain points to improve UX design. It enables UX professionals to make better-informed design decisions.
AI analytics can alert professionals to anomalies and issues. However, it still requires a human being to interpret results and understand their implications.
AI has forged its way into visual design as well. AI features expedite design tasks. It can help create a plethora of prototypes or wireframes that weren’t humanly possible before.
Companies such as Adobe have introduced AI features to help designers build visually appealing and effective user interfaces. To name a few:
- Visual asset analysis — comparing attention across various creative elements
- Text-to-image generation
- Color & font pairing
- Cropping or resizing
- Removal of items/objects
Implementing advanced AI in research analytics and design results in well-designed user interfaces. AI can also help gather and analyze usability metrics to understand what’s working and what isn’t. It makes the feedback cycle a lot shorter.
Chatbots, Virtual Assistants and Conversational Interfaces
ChatGPT’s release was revolutionary. It changed how college students worked (or avoided work), how we access information, and brought efficiency to the workplace.
Our friends at User Interviews conducted a study with over 1,000 UX professionals and found that 77% of respondents already use some form of AI in their work.
NLP is the underlying technology behind applications like ChatGPT. It’s essential for chatbots, voice assistants (such as Siri or Alexa), and sentiment analysis. UX researchers can leverage NLP to conduct surveys and analyze user feedback. This allows them to iterate and improve the conversational aspect of a user interface.
If you haven’t jumped on the AI chatbot bandwagon already, FAQPrime assembled a beginner’s list of prompts to get you started. Tinker with these commands to see what suits your workflow best. Remember to follow these basic guidelines to follow when interacting with a chatbot or virtual assistant:
- Provide ample context
- Ask for multiple options
- Iterate on the output
- Build a prompt library
Search Your Own Research the Way You Search the Web
Marvin’s Ask AI product is the ChatGPT of Research. Trying to garner insights from old studies? A perennial pain in the behind, no more! Ask any question to begin searching your research repository and get the answers you need in seconds.
Aside from transcribing calls, Marvin’s end-to-end research assistant conducts foundational qualitative analysis for you. It creates automatic notes during and after interviews and is capable of articulating the gist of mundane and long conversations with AI generated summaries. AI synthesis allows researchers to annotate on the fly.
*Marvin UX Research Products visual of AI Research Assistant
AI frees up a researcher’s time to focus fully on the interview and delve deeper into user pain points. Remember, using AI as a sidekick is like working with an intern. While fully capable of understanding instructions and completing tasks, you still have to check their work.
Learn about Marvin’s most popular products launched in 2023.
Actionable Insights Through AI Analysis
With AI’s help, designers can conduct comprehensive design reviews. Observing how users interact with a system (or predicting how they will interact) provides insight into the user journey. This allows researchers and designers to scrutinize UIs to identify usability issues and other areas that need improvement.
AI’s ability to handle vast and complex datasets accelerates the shift from raw data to actionable insights. AI extracts patterns and insights much faster than a human would. Using AI to conduct preliminary analysis and synthesis creates an immediacy in results. This helps answer questions like:
What changes can we make to the product based on this information?
It all leads to the development of designing smarter, modular design interfaces that are user-friendly.
Data-driven decision-making improves the accuracy and quality of research. It provides insights using real-time data to identify areas of interest. Researchers and designers can then make informed choices based on the underlying data.
AI adds efficiency to the design process. However, its findings or results need to be interpreted by someone(human). You still (and always will) need a human being on hand to establish the direction of research and make sense of it all.
Benefits of AI in UX Research
Following its $10 billion investment into OpenAI*, Microsoft announced an upcoming AI co-pilot soon to be deployed in Microsoft’s Office suite. Co-pilot boasts the ability to quickly analyze data, generate reports and create presentations (among many). Working professionals (including yours truly) were left in limbo, worrying for their long term job prospects.
Sounds too close to home? Let us alleviate your fears.
Microsoft called it “co-pilot” implying someone else is flying the plane: You.
AI may be ground-breaking technology, but at the end of the day, it’s a technology that exists to serve humans. As directors of technology, it becomes increasingly important for us to understand how to effectively use AI during analysis, and for what. Learn how AI can become your ultimate UX research sidekick (or your copilot or the Robin to your Batman or… you get the idea).
Optimize UX Workflow
Qualitative research provides a wholesome understanding of phenomenon through rich insights. Arriving at those insights involves a tedious, cumbersome data annotation process called coding or tagging.
Traditionally, UX researchers toiled away, spending hours transcribing and tagging interview data. Not anymore. Research tools like Marvin use analytical AI to create transcripts of virtually any interview recording you throw at them. By automating these mechanical tasks, it frees up a researcher’s time to focus wholly on the participant and conduct deeper analysis. Marvin users spend 60% less time analyzing UX research data. Set your research up for success — explore our guide on how to use tags to reach insights faster.
One thing that’s indisputable is that AI is bound to make workers more efficient. With AI in their toolkit, UX researchers and product designers will spend less time wrestling with tricky and convoluted data, and more time on analysis.
Enable Large-Scale Analysis with AI in UX Research
Improved efficiency unlocks the door to conduct data analysis en masse. Deconstructing an interview transcript is a complex process. Researchers can suffer from the dreaded data overload — there’s simply too much to unpack from unstructured data from individual interviews and group sessions, verbal and non-verbal cues and overlapping themes.
AI facilitates the analysis of vast amounts of data — sit back and let it pore through mountains of historical data. It has the ability to predict possible customer behavior and evaluate how they interact with designs. The amount of predictive analysis AI is capable of was not physically or mentally possible before the introduction of AI (not by a human, anyway!).
Start your analysis on the right foot. Marvin uses generative AI to create synopses of interviews, condensing key points from hours of interview time into a paragraph or two. Far preferable to get a quick gist than reading a twenty page long transcript. Marvin’s AI generates auto-notes for your recordings. A solid structure and foundation lays the path for deeper analysis. By delegating the heavy lifting to AI, researchers can do more with all this newfound time on their hands.
[DISCLAIMER: We encourage you not to think of the output as the final product, but a starting point for your analysis.]
Deliver Consistency & Reliability
What machines do better than humans, is follow instructions to the tee. Researchers can define an interpretive grid (a coding scheme) and set AI models to work, so that they perform some initial heavy lifting. Any failure cases based on the algorithm can then be further explicated.
Bias is inherent in every study — unfortunately that’s attributable to the researcher. AI can be trained and corrected to interpret and eradicate bias. This makes studies more equitable across the board.
Privacy is a fundamental right. Protecting user privacy is of utmost importance while conducting user research. We obsess over protecting participants’ Personal and Identifiable Information (PII) such as their names, contact details, gender and occupation. We teamed up with Assembly AI to pioneer a first-of-its-kind PII redaction model that automatically strips away participant PII from audio and video files. Powered with AI; more peace of mind for everyone.
Uncover New Patterns with AI in UX Research
Analytical AI has the capacity to unearth unexpected and interesting insights. Given the right prompt, it can detect patterns and themes in textual data, and generate insights that even researchers may have missed.
AI levels the playing field enabling multilingual analysis and promoting cultural diversity in research. With text mining & natural language processing (NLP), research tools like Marvin can translate and transcribe numerous languages. This breaks down barriers to communication, overcoming language limitations, form, styles and varying academic conventions.
With AI, a refreshing and novel idea may be just round the corner.
Facilitate Collaboration
AI originally found itself catering to fields of computer science and engineering, but now permeates through various disciplines. It now transcends healthcare, business and finance, psychology and neuroscience.
At Marvin, we wanted a platform to centralize all user insights, so they all live in one place. No more duplicative efforts, harness the power of a centralized research repository. We also want users across disciplines to continue with existing tools that they already use. Share playlists, clips and insights with your peers. Whether they’re researchers or not – everyone benefits from first-hand feedback from end users.
LiveNotes, our collaborative note taking tool, enables you to create time-stamped insights, so you can quickly bookmark and annotate important parts of an interview while conducting it. Your findings are always accessible and editable upon later viewing — synthesize them live or post with colleagues. Integrate your video conferencing platform of choice (Zoom, Meet, Teams) and document your observations with your peers.
Our core values ring true: Elevate the user voice across your organization. Create a customer-centric culture.
Risks of AI in UX Research
On the road to implementing new tech, there’s bound to be rough and bumpy bits along the way. AI will only get better with each iteration — Chat GPT-4 is far more advanced and capable than its predecessors.
Call them usability enhancements or bug fixes, the fact is we learn more about the nuances of a particular application with time as we use it. It’s vital to understand the limitations of AI in UX research work — drawing the line under what it can help you with versus what it can’t.
Baked-in Bias
Bias is a double-edged sword. We’re all biased, whether we care to admit it or not. Above, we looked at how AI can reduce human bias by automating certain procedures. Consider this — any AI model is coded by a developer. Developers may unknowingly and inadvertently bake in bias into their model, resulting in skewed analysis and reinforcing their own existing social prejudices, stereotypes or inequalities.
Plenty of recent examples illustrate how big tech companies failed to detect inherent bias in their systems. Companies such as Amazon, Microsoft and Google have been in the news for the wrong reasons — their AI algorithms unknowingly exhibited racial and gender bias.
Google Research Scientist Rida Qadri recently conducted a study on the (poor) representation of South Asian cultures in text-to-image AI models. Rida suggests that since these models are based on the internet as a collective archive of information, they are largely representative of the western world. As a result, South Asian cultures are often poorly represented, if at all.
There are serious ethical considerations to take into account when developing new AI technologies. Researchers must delve deeper into AI output and seek to identify and correct any biases. Award-winning researcher Mary Gray questions whether we are failing certain groups in society. She touts qualitative research as key to building more ethically responsible AI.
Less Context
How does AI perform in providing context in your qualitative insights?
Check out these two studies:
Here’s a quick tl;dr recap:
Actual human researchers faced off with a machine to unearth qualitative insights from an open-ended question. While the machine took considerably less time to conduct its analysis, the human analysis was more in-depth and comprehensive. The machine’s categorizations were simplistic and unhelpful. In a sentiment analysis, machines missed the crux of participant responses by isolating each word individually, failing to group them into coherent themes at all.
As humans, the researchers had an understanding of the question and the reasoning behind it. While classifying responses, they can be relied on to provide the right context, group phrases together and tag data intuitively.
We understand other humans — what their responses mean and what they are alluding to. That’s what separates man from machine.
Lost Human Touch
“AI is limited by its inability to possess human-level understanding of the social world” – Hubert. L Dreyfus, 1992.
There’s no getting around it — the responsibility of creating a study and all conclusions drawn from it will always rest with human researchers. No matter how comprehensive results or insights are, AI cannot be accorded with any ownership or authorship of research. AI has no common sense, an inability to learn from experience and a lack of understanding of social and cultural nuances.
Qualitative research relies a lot on forming inferences from an interaction. When you interview a participant, you choose how to navigate the interview, forming new questions and interpretations along the way.
AI lacks that human touch — the intuition and experience that a lifetime of interactions has given us. We spoke earlier about giving interviewees your full attention – AI wouldn’t pick up on subtleties such as a change in facial expression or a shift in the tone of voice.
Humans can gauge sentiment. Fidelity’s VP of Design Ben Little spoke to us about how craftsmanship is making a comeback. A techno-optimist, Ben says that the mass adoption of AI will only increase the novelty of human-crafted design. There are some things you can’t replace; AI just doesn’t have lived experience, imagination and empathy — three inherent human traits.
Noisy Data
Ever heard of GIGO? A computer science wordplay on the concept of the first-in first-out (FIFO) inventory accounting method (yawn), it stands for “Garbage in, garbage out.”
Quite simply, what you put into something, you get out of it. We’re not in the business of doling out life lessons, but the same logic applies to any AI model. The quality of output is dependent on the quality of input. Put garbage in, and don’t be surprised when garbage comes out.
Ask any data analyst, product designer or researcher — the bulk of their time is spent cleaning and sanitizing data. Output from AI models are dependent on pre-processed, accurate data. AI struggles to deal with inconsistent or ambiguous data.
Mary talked passionately about ensuring models have a complete and comprehensive dataset to begin with. Rida explained that this isn’t so easy. It’s hard to shake off the indelible Western influence over technology. Since AI uses the web as the foundational library of all information, it’s important to remember that it’s not all-encompassing.
Continually ask yourself — are we training AI models on the right dataset?
Overreliance on Technology
Should you ask any youngster today a general knowledge or simple math question, watch them consult their smartphones before their brain. Spend a day at work without your phone and observe how disconnected and naked you feel. Surprise, surprise, we’re all a bit too reliant on technology.
This cuts the same way for UX researchers. If AI is doing all the work for you, your neurons aren’t likely firing on all cylinders. Going through the motions, researchers can miss insights, lose interest in the automated tasks which diminishes their critical thinking and analytical skills.
Mechanization culls creativity. Design is an artistic field, a realm of imagination and innovation. Leaning too heavily on tech can make us lazy, complacent and boring. Does churning out something that’s been done to death work?
Likely not.
Our Favorite AI Tools for UX Research
With so many tools name-dropping “AI” on their promotional material, where do you get started? How do you discern between powerful tools and lemons?
Good news.
Like AI, we’ve done the heavy lifting for you. Use these five AI tools to boost your research efforts:
Marvin
Supercharge your research with Marvin, your AI-powered research sidekick.
Here’s a list of Marvin’s industry-leading AI features:
- Accurate Transcription. Marvin converts interviews into transcripts in over 40 languages. Add industry-specific terms, and Marvin learns from that data.
- AI Note-Taking. Free yourself from vigorous note-taking. Marvin summarizes interviews and generates a small write-up for quick review. In addition, it creates time-stamped insights. Collaborative note-taking helps get multiple viewpoints in real-time.
- Ask AI. Think of this as a search engine for your research repository. Ask questions about your data and interrogate research across projects. It makes your data inherently searchable.
- Survey Analysis. Upload surveys such as NPS and let Marvin take over the analysis. In minutes, it generates helpful visualizations that help you better understand your data. Begin your analysis with Marvin.
Marvin integrates with tools researchers and designers love. With GDPR, SOC2, and HIPAA compliance, you can rest easy knowing your user data is always protected.
With AI-enabled smart workflows, reach insights faster with Marvin. Book a demo today!
Maze
Maze has a host of AI features that expedite research. In addition to transcribing interviews, AI conducts analysis on unmoderated testing. Create prototypes, and Maze will analyze how users interact with the UI.
Maze also auto-generates a report of findings from interviews. It can create follow-up questions for different insights. Use sentiment analysis to evaluate overall customer satisfaction with a product. AI creates themes from qualitative questions, allowing researchers to dive deeper into analysis.
QoQo
Powered by OpenAI’s GPT, QoQo helps you better understand users. Visualize how users interact with products and create user journey maps using QoQo. Identify challenges and risks for designs and prioritize design changes.
Use the Figma plugin to generate well-defined user personas. QoQo creates cards representing user goals, needs, motivations, frustrations, and tasks.
QoQo expedites affinity mapping, helping researchers sort through and organize large datasets. Its AI can even generate entire interview scripts for you. Its Figma integration means that it doesn’t interrupt the design workflow.
QoQo’s output serves as a starting point for analysis. Use the application in tandem with other user research tools.
Uizard
Uizard is a highly popular prototyping and wireframe creation tool. You can generate wireframes from text prompts or hand-draw a sketch. Uizard converts your design into an editable mockup. For all the coders out there, it generates the underlying code from the sketch, too.
Generate attention heatmaps to predict where users will focus in an application.
The application uses design best practices to create wireframes—a great choice for efficient, AI-powered UX design. Iterate endlessly and create stunning, user-friendly designs with Uizard.
ChatGPT
ChatGPT uses NLP to generate content based on written prompts. It can be used at different stages of the research process.
Continuously brainstorm research project ideas, generate and refine survey questions, and create narratives for different user journeys. ChatGPT analyzes text data in seconds. Create summaries of lengthy documents and evaluate the sentiment from bodies of text.
Analyze documents to conduct in-depth research using ChatGPT. Collect statistics (verify these first!) about competitors and customer demographics. Craft and personalize emails for marketing campaigns.
It’s vital to cross-check its output before using it. While using ChatGPT, be mindful of the fact that:
- Trained on the internet, ChatGPT output may be inherently biased.
- It has a knowledge cutoff—it’s only up-to-date for a certain time period. If you’re looking for real-time data, use different sources.
- LLMs like this can hallucinate — if it has no knowledge of a topic, it begins to fabricate answers.
- Your data is not secure if you’re not using a premium version of ChatGPT.
Will AI Replace the Researchers?
AI will augment researchers’ work, not replace researchers themselves.
We’re more bullish on this assertion today than ever before.
Marvin CEO Prayag Narula believes AI is the perfect research assistant. AI-generated summaries help with a superficial understanding of the big picture. They also save researchers precious time ordinarily spent rewatching long videos and taking notes.
Our users tells us Marvin’s AI transcription is game-changing. It frees up their time and mindspace to focus on deepening their understanding of users.
AI in UX Research Technology to Watch For
However, transcripts miss important context — they don’t capture emotion. People don’t verbalize all their actions and describe everything that they’re thinking.
To tackle these shortcomings, some AI tech is still in the works:
- Biometric technology is (theoretically) capable of capturing human emotion from video or visual artifacts. Its current output is confusing, messy, and misleading — it requires more training data. Emotion recognition has the potential to provide valuable insights into users.
- Synthetic users are largely unpopular among the UX community. Designers argue that they build for humans, not synthetic or artificial users. However, there are instances when it can be useful. During early days of a research practice, it can help smaller teams increase the scale of their work.
With AI technology developing rapidly, it might be a short time before these features are added to its growing arsenal.
To understand more about the UX industry’s continued evolution, Lattice’s VP of Design stopped by to share his predictions for 2024. Jared reminded researchers and designers to focus on:
- Efficiency — or how to do more with less. Companies will harness a combination of researchers, designers, and AI capabilities to maximize research output.
- AI literacy is now a prerequisite. It moves from being a “nice-to-have” to a “must-have” skill for researchers and designers. Companies will look to hire UX professionals who are proficient or at least competent in AI.
Best Practices for Implementing AI in User Research
Implementing AI in User Research is no small task. It requires a company-wide effort and buy-in from key stakeholders.
Run through this checklist to ensure you get the most out of AI in your research workflow:
- Define Goals & Questions. What are you trying to learn? What research questions are you looking to answer? What tools and techniques will you use? Where can AI be of assistance?
- Try before you Buy. Researchers will love this. Thoroughly test & evaluate AI tools before choosing one (more on this below).
- Soft Launch. Put tools to work to see which ones best integrate with your existing workflow. Remember to use high-quality data (accurate, complete, and consistent) to evaluate AI’s performance.
- Maintain Human Oversight. Double check AI’s analysis. Validate its output and compare it with manual analysis to test its reliability. Remember – AI is not always accurate and doesn’t replace human empathy and creativity.
- Safeguard User Data. Prioritize user privacy. Anonymize user data and ensure it’s encrypted. Establish guidelines for scrubbing Personal Identifiable Information (PII) and for data storage. Ensure compliance with local data privacy laws.
- Stay Up-to-Date. UX Tools constantly roll out improvements and new features. Keep yourself informed of new features and capabilities. Stay ahead of the curve.
- Be Transparent. Inform the rest of your organization about the adoption of AI tech. What does it mean for different users? Send out emails or announcements detailing how implementation will take place. Keeping everyone in the loop establishes trust.
- Communicate & Educate. Create resources for users — establish guidelines and AI best practices. Dispel misconceptions about AI and outline proper use of the technology. They must learn how to use the tool, and analyze and interpret AI output. Set up training for new users.
- Review & Iterate. Spend time with teams to understand their optimal use of AI tools. Existing users will reveal how they are actually using AI. To bridge the gap between the two, periodically monitor progress and tweak the approach. Is the tool helping us meet business objectives?
Don’t over-rely on AI’s output. Remember, AI is an enabler. A research assistant, and not the primary researcher itself. Ask yourself whether it can assist you in projects that you undertake.
What is the Final Verdict on AI in UX Research?
[Admittedly, we dropped the ball with the title of this section. We wanted to replicate the Clint Eastwood western classic, but we couldn’t paint the future as ‘ugly’. Read on to find out why.]
What does the future hold for UX researchers and AI? We’re quite bullish on AI in research. Our two cents (literally) is two factors to constantly keep in mind as you build and introduce new products into the world:
Consider the Community Impact
Technology has a transformational impact on society. Computers used to occupy the size of a room, but now sit in our pockets. With a few swipes, you can have groceries delivered home, learn a new language or hail a taxi.
Rida Qadri examined how technology is continually shaped by social contexts. She spent time in Jakarta examining mobility platforms and how they adapted to the existing mobility landscape. Uber revolutionized travel and mobility in the West, but their approach doesn’t translate across the globe. As designers, we don’t get to tell users how to use our technology. Noone can predict user behavior.
We may have the best intentions in releasing groundbreaking AI technologies. However, we would be doing communities and wider humanity a disservice if we don’t spend a substantial amount of time trying to understand the impact AI will have on these communities.
Try understanding the impact these technologies have not on the immediate, but wider communities, even ones you might not be examining today.
Maintain User-Centric Design
Throughout the product design process, the most important questions designers must ask themselves are:
- Why are we building this?
- What problem(s) does it solve for our customers?
- What are any potential challenges we may encounter?
This is the crux of establishing user empathy. Never lose sight of these core questions, constantly circle back to them.
And remember, a customer-centric culture is not something that AI can build for you.
Learn more tips and tricks to incorporate the use of AI in your research.
Frequently Asked Questions (FAQs)
Let’s tackle some frequently asked questions about AI in User Research:
How Do I Choose the Right AI Solution for My User Research Needs?
We’re going with a researcher’s favorite answer — it depends. Every business has different needs and use cases for AI in UX research.
Consider these business requirements before making your choice:
- Goals. What strategic objectives will the tool serve? What do regular projects or studies look like? How will this tool bring efficiency to these processes? Who will be the primary users of the software?
- Functionality. What functions do users need? Distinguish between essentials and nice-to-haves. Do AI features work as advertised?
- Budget. How much can you spend on new software? Consider how many user licenses you’ll need. Use free trials to give AI tools a test drive. Try before you buy.
- Timelines. Are there any deadlines for the implementation of the new tool? Phase out software implementation to bring everyone up to speed by a certain date.
- Additional Features. Does the tool support other functionalities? What secondary features augment the UX research process?
- Ease of Use. A user-friendly UI helps users learn the ropes easily. Look for tools with a shallow learning curve.
- Data Security. Safeguarding Personal Identifiable Information (PII) is essential. Look for the ability to anonymize and encrypt user data. Choose tools that protect user data in line with local guidelines and regulations.
- Integrations. Consider your existing tool stack. What applications must integrate with the new software? Is this a seamless or complex process?
- Scalability. Can the tool scale with research operations? AI UX Tools must be flexible and adapt to the evolving needs of the research process. Future-proof your tool choice.
- Customer Support. What resources are available to users when they are stuck? Are there self-serve guides and forums? Is there a dedicated support team? How responsive are they?
Want a tool that checks ALL the boxes? Marvin.
Learn more about how Marvin helps you convert user feedback into useful insights.
What Are the Costs Associated with Implementing AI in User Research?
Companies underestimate the cost of deploying and operating AI. This leads to overshot budgets and disappointing AI implementation.
We examine costs in terms of expenditure on the ‘big three’ resources – money, time and effort:
- Financial. Recurring costs include the cost of ownership – license fees depend on the number of users. Companies must factor in one-time expenses like implementation costs and ongoing maintenance. Will AI setup and management be conducted in-house or outsourced? Will you need new hires?
- Time. What’s the deadline to get AI in UX research up and running? The shorter the timeline, the costlier implementation will be. Factor in the time it will take to on-board everyone and get them up to speed with the new software.
- Effort. Transition management is undertaken by one or two designated employees who oversee the process of moving to a new tool. They conduct training and iterate on feedback. Not to mention dealing with resistance to change. Choose a solution that fits your current or planned in-house expertise.
Remember the big three when you think about the true cost of implementing AI in user research.
What Are the Key Metrics to Evaluate AI Performance in User Research?
Measuring the success of initiatives is crucial. It helps companies determine whether AI implementations are achieving business goals.
Evaluate four overarching metrics, each with its own set of Key Performance Indicators (KPIs):
- Accuracy. Measure the correctness and precision of AI output.
- Process Time
- Error Rates
- Automation Levels
- Efficiency. Has AI reduced spend on the big three resources – money, time, and effort?
- Performance. Measure the effectiveness of AI applications.
- Financial. What are the economic benefits that can be attributed to AI?
- Contribution to Revenue
- ROI.
When spending money on anything, C-suite executives and decision makers will ALWAYS revert to one metric – ROI. What’s the return on investment?
Remember, treat these as a longitudinal study by tracking them over time. Chart areas where AI improves business performance and look for places where it can perform better.
Research is complex and informs decision-making and strategy across departments. Thus, measuring its impact is not easy.
Conclusion
There’s no reason why UX professionals and AI can’t enjoy a peaceful co-existence. AI helps facilitate rapid and thorough analysis, giving designers and researchers the gift of time. It comes with its fair share of limitations, which must be understood fully while using it. As with any nascent technology, plenty of kinks to be ironed out.
Central to leveraging AI will always be the human behind it. Technology is here to serve humans, not the other way around. The rise of AI is not necessarily about replacing researchers and designers, but empowering them to become more productive. Think of AI less as a threat and more of an apparatus in the UX toolkit, one that supercharges your productivity.
Future Trends and Innovations
AI will augment researchers’ work, not replace researchers themselves.
We’re more bullish on this assertion today than ever before.
Marvin CEO Prayag Narula believes AI is the perfect research assistant. AI-generated summaries help with a superficial understanding of the big picture. They also save researchers precious time ordinarily spent rewatching long videos and taking notes.
Our users tell us Marvin’s AI transcription is game-changing. It frees up their time and mind space to focus on deepening their understanding of users.
Learn More About G2’s Top-Rated UX Research Repositor
Find out how to integrate Marvin’s AI features into your UX research workflow.
Book a free demo today.
*Quick note that Sam Altman is one of Marvin’s lead investors. We’re big fans of his work and grateful for his support of our user research platform!
