Categories
Articles

8 UX research methods to find insights for A/B testing

Introduction

Using UX research methods to find problems on websites can be really daunting because there are so many out there. You need to decide which UX research method fits within your budget and will have to highest probability to generate key insights for future experiments. 

This article will help you to answer this question by describing what UX research methods are. Furthermore, it will describe the difference between qualitative and quantitative research. In addition, there will be a brief section about the attitudinal and behavior dimension within UX research methods. 

Finally this article will describe 8 different UX research methods and link different tools & resources you can use to get started straight away. 

What are UX research methods?

UX research methods are ways of generating insights about how and why users interact with your product or service. It is the key to optimizing your products and services to meet your user’s needs because it guides you in your decision making instead of trying to experiment on your gut feeling.

The list of insights you can gain by doing research also gives you the opportunity to choose what the improve first. Prioritizing this list correctly will give you a higher chance of success. Understanding the differences between research methods will help you decide what kind of research you need.

What are the differences between qualitative and quantitative research?

Differences between qualitative and quantitative research
Differences between qualitative and quantitative research – 2020

I think it’s important to know if the UX research method you are using is qualitative of quantitative because they play into different goals.

Qualitative research is to gain insights into the needs and motivations of people. Usually this is done with a few people to really dive deep into their thoughts and learn from them. It takes a step away from numerical data and focuses on a person itself. It can help you uncover things you would probably never come up with yourself. The results of this kind of research are mostly subjective but can be used for future research. There is usually no set way of collecting data and the way really depends on the type of qualitative research.

Quantitative research is to test hypotheses and research questions. This kind of research depends on measuring and testing and is a lot less subjective because the researcher is only observing. Collecting data needs a structured approach with a specific set of questions, categories, or metrics. For this method you will need a large sample size to get significant results so the research will have the same results when repeated. Insights are gained by analyzing data and no real contact between customer and researcher is needed.


Over 250+ professionals have already signed up!

Join GrowthPenguin to receive the latest blogs in your inbox every week!

Processing…
Success! You're on the list.


What are the differences between the attitudinal vs behavioral dimension?

Another important difference between research methods is the attitudinal and behavioral dimensions.

The attitudinal dimension is defined by what people say. This means that UX research methods like interviews and surveys fit this category. The behavioral dimension is defined by what people do. UX research methods like ab testing and eye-tracking fit in this dimension.

The research you chose to do first can be decided by looking at these dimensions. I found this infographic on a really nice article from the Nielsen Norman Group.

Nielsen Norman Group a landscape of user research methods
Nielsen Norman Group a landscape of user research methods – 2020

Here you can see different methods based on whether the research is quantitative or qualitative and behavioral and attitudinal. Knowing this information might help you decide what UX research methods you need depending on the kind of answers you are missing.

What kind of UX research methods are out there?

In this section I will discuss 10 different UX research methods. I will describe how much time they cost and what tools you can use to start testing. I will also look at the costs of these tools and base my list on my personal preference with UX research.

UX research method 1#: AB testing

UX research method: AB testing
A is black and B is red – 2020

Undoubtedly my favorite kind of research method is AB testing because the concept is simple but powerful. The main idea of this kind of testing is to test control and a variant with the goal of validating a testing hypothesis. This kind of testing is quantitative because its results are determined by the outcome of numbers. You need a fairly sized sample to get significant results. This UX research method is behavioral because it focuses on what people do instead of what they think. 

Testing with this method takes at least two to six weeks per test cycle to even have a chance at a valid and significant result. The key here is to always have a test running in which you prioritize the tests that have the most potential impact first. This is definitely a long term goal with will only yield results if done for a longer period of time. Therefore the costs are high and can become even higher with development costs and tools.

Examples of tools are:

  1. Google Optimize
  2. VWO
  3. Google Tag Manager
  4. AB Tasty
  5. Firebase
  6. Optimizely
  7. Adobe Target
  8. Leadformly
  9. Unbounce
  10. Sitespect

UX research method #2: Eye-tracking

Research method: Eye-tracking
Eyetracking penguin – 2020

Eye-tracking uses specialized equipment to track users’ eye movements while looking at a screen. This kind of research can give you an idea of where users focus their attention when landing on a page and decide what designs work the best for guiding people’s vision towards your site goals. 

This kind of research will probably be more on the qualitative side because it’s costly to do it with large amounts of people and insights can also be obtained with smaller sample sizes. Eye-tracking is more towards the attitudinal side because the researchers give the subject a set of instructions to complete on a website and will give minimal to no instructions.

The equipment you need for this kind of testing can be quite costly so it’s recommended to work with companies that supply these tools and to only research webpages that will not have many changes in the long run. It can prove very cost in-effective to use eye-tracking on a homepage that changes frequently for example. The time take it takes to complete this research depends on how many participants you use in your research. 

You need to keep in mind that you need to make the instructions, get participants, set up a controlled environment, run the test, and analyze the data. Outsourcing this entirely to an expert at eye-tracking is recommended. An alternative to eye-tracking is mouse tracking and is a lot cheaper and faster than eye-tracking.

Examples of tools are:

  1. Tobii Pro: Eye-tracking technology for research
  2. Imotions: Eye Tracking
  3. EyeTracking – The Eye Tracking Experts
  4. Noldus: Eye-tracking + eye trackers

UX research method #3: Card sorting

UX research method: Card sorting
Penguin doing some card sorting – 2020

Card sorting is a UX research method that uses cards to create a category tree with experts. It can be used for the structure of your website because the results can make it easier for users to navigate through your website. It can be done by users as well with the guidance of an expert in the field of the subject or website. There are many tools online you can use the start of this research method or use good old fashioned post-it notes on a wall. I used the technique for many websites and it usually gave me the insights I needed about a site’s structure. 

This UX research method can be qualitative and quantitative depending on your approach. If you decide to let thousands of people sort your cards online for money then you can get results from numerical data. Whilst the low-tech approach will focus more on qualitative insights. The same goes for attitudinal versus behavioral. The online mass approach will give you more behavioral insights whilst offline can give you the start of a deep conversation with your users.

It doesn’t cost to much time because you can set it up and complete it within a day of research so if you are in the middle of changing your website navigation you should definitely give this a go. The costs really depend on the tool you use because online solutions will usually have a freemium business model which will require you to pay if you want a certain amount of cards. 

Examples of tools are:

  1. Uxtweak – Powerful UX research & user testing tool
  2. OptimalSort
  3. UserZoom | User Experience (UX) Research Platform
  4. xSort – Free card sorting application for Mac
  5. Usabilitest: Card Sorting online tool with built-in data analytics

UX research method #4: Focus groups

Research method: Focus groups
Focus penguin group – 2020

Focus groups are small representations of your target group that you can use to gather insights about a product or service. Usually there will be one researcher that talks and asks questions about the product or service while someone else writes down all the answers for later analysis. This can be done offline or online and there are companies that can supply you with people for this kind of research. You can also reach out to your target group in exchange for a fee of some kind. It can be challenging to contact a diverse focus group and get data because they need to know what’s in it for them.

A Focus group can be defined as qualitative because it focuses on such a small sample size of around 6 to 10 people. The results are hardly numerical and will usually require more research afterward. This kind of research is attitudinal because it focuses on what people say and think instead of what they would do on your product or service.

Depending on the reward for the participants it can be costly to get it all set up. It also depends if you use an agency to gather the people and if you set up the focus group offline or online. The “event” itself could be done in a day if done right.

Examples of tools are:

  1. FocusGroup.com — Paid Focus Group and Market Research Opportunities
  2. Find Paid Focus Groups in Your City – FocusGroups.org
  3. SurveySwap | Find Survey Participants Today
  4. Respondent.io

UX research method #5: Interviews

UX research method: Interviews
Interviewing another penguin – 2020

Interviews are one on one conversations between the researcher and the participant. In the interview multiple questions are asked and answered while the whole process is being recorded. The recorded sessions can be transcribed for further analysis in research papers for example. This is still one of the easiest ways of finding insights about your users. Simply asking them why they respond to certain questions will help you pin down the underlying behavior behind what people say.

Interviews are qualitative because it focuses on one participant. This kind of research is attitudinal because it focuses on what the participants say to answer all the questions. The time and cost will be relatively low because you conduct multiple interviews in a day. Most time goes into contacting and preparing the interviews. I could only find one tool about interviews because most people just write a set of questions on the laptop or notebook. I recommend using a notebook because then the participant can actually look at you instead of your laptop stickers.

Example tools:

  1. Interview Creation Tool

UX research method #6: Co-design

Research method: Co-design
Winter is coming! – 2020

Co-design or co-creation is the process of creating or optimizing a product or service with a small section of your target group. Developers of video games will ask their users if certain features will be a good addition for the game and decide if it will be developed on the number of votes. Another way of doing it is by actually sitting with your customers and brainstorming ways of improving or adding to your current service or product. This can be done by designing social experiments or just getting a lot of post-it notes and using a brainstorming technique. 

This type of UX research method is qualitative when it’s with small groups of people in an offline setting and can be quantitative when doing online research with bigger sample sizes. Co-design is an attitudinal research method because it’s based on people’s responses in most settings but can be behavioral with social experiments for example.

Example tools & techniques are:

  1. Hotjar: Website Heatmaps & Behavior Analytics Tools (for surveys)
  2. Service Design Tools
  3. Google Forms: Free Online Surveys for Personal Use
  4. Design Kit
  5. 19 Top Brainstorming Techniques to Generate Ideas for Every Situation

UX research method #7: E-mail surveys

UX research method: E-mail surveys
Penguin sending out an e-mail survey – 2020

Another way of gettings insights is by sending out e-mail surveys to customers & subscribers to ask for feedback. This can be done after someone has bought a product on your website for example. Asking them how they experienced the checkout process can give you some insights you would have never found on your own. Decide where you need to focus your research and send open simple questions to get answers other than Yes and No. A technique for getting more out of a question is first asking closed questions and afterward asking for elaboration on the answer. This uses the commitment and consistency principle of Cialdini and will give you more insights to act on. 

This kind of research is qualitative because the answers will vary per participant and will probably yield no significant patterns. It could be more towards quantitative research by using a large sample size with closed questions. E-mail surveys are attitudinal because it focuses on what people say.

Example tools are:

  1. SurveyMonkey
  2. SoGoSurvey: Enterprise Online Survey Software & Tools for Businesses
  3. SurveyPlanet: Create Free Online Surveys | Easy to use Survey Maker
  4. SurveyGizmo: Enterprise Online Survey Software & Tools
  5. Google Forms: Free Online Surveys for Personal Use
  6. Typeform

UX research method #8: Form analysis

UX research method: Form analysis
Penguin researching forms – 2020

Form analysis is the method of trying to find what fields in a form are problematic or hard for visitors on your website. You can find where to optimize your forms looking at the time, refills, and drop-offs. This research is quantitative because you try to base your data on as many sessions as you can for a valid report. Try to base your forms on at least 5000 sessions for the best results. Form analysis is behavioral because it focuses on what people do within your forms. Try analyzing your checkout form first to get some insights that could make a big impact on the number of people that finish the checkout process.

Example tools are:

  1. Hotjar: Website Heatmaps & Behavior Analytics Tools
  2. VWO: A/B Testing & Conversion Rate Optimization Tool
  3. Zuko – Product Features
  4. Woopra: Customer Journey Analytics for Web, Mobile, Product and Beyond
  5. Clicktale: Digital Experience Analytics | Customer Behavior Analysis

Conclusion

I hope this article will help you understand and choose the UX research methods you need for your product or service. There are a lot more research methods available then the ones listed here so make sure to keep on researching for more ways of getting insights to experiment on. The key is to get insights that you can actually use for your next step in optimization. Good luck with making your product or service fit the customer’s needs!

Leave a Reply

Your email address will not be published. Required fields are marked *