C07: Using Questionnaires and Surveys in UX Research

Abstract

Surveys are a widely used method to collect user feedback concerning the user experience of products or services, as they allow to collect data from larger groups of users with low effort. A survey allows to collect data concerning demographics and usage behavior (for example, frequency of use or experience with a product), quantitative measurements of UX quality over standardized UX questionnaires and user comments concerning various aspects of the user interaction.

The Course describes the typical research questions that can be answered through a UX survey. It will be discussed in which cases alternative methods (for example, inspection methods or usability tests) may be more appropriate and how a survey can be combined with other UX methods.

Some common standardized UX questionnaires are described in detail to provide an overview of the various existing measurement concepts, item formats and response scales.

We will discuss in detail the practical setup of a UX survey. How to find the most suitable UX questionnaire for the research questions? How to formulate effective survey questions? Which channels can be used to distribute the survey and what are their advantages and disadvantages? Which legal and ethical requirements must be considered, for example data privacy regulations? How to analyze the structured data with suitable statistical methods? What are typical pitfalls in data analysis and how to avoid them? How to analyze larger volumes of user comments efficiently using GenAI? How can the results be interpreted? How are the findings presented to managers and software developers in industry or in research? How can UX insights collected over surveys be transferred successfully into UX improvements.

Course Content

Participants in this Course will gain the ability to:

• Understand common UX research questions that can be addressed using surveys.
• Understand the advantages and disadvantages of UX surveys compared to other popular UX methods, such as interviews, usability tests or inspection methods.
• Understand the types of information that can be collected through surveys, including demographic data, usage behaviour data, user comments and UX metrics from standardized questionnaires.
• Understand the structure and use cases of popular standardized UX questionnaires that enable quantitative measurement of UX.
• Understand the key quality criteria (objectivity, reliability, validity) for standardized UX questionnaires.
• Learn how to set up a UX survey, including writing clear instructions, selecting appropriate and easy to understand questions, etc.
• Learn about various channels for collecting user feedback through surveys and the biases associated with each channel.
• Learn how to analyse structured data and avoid common pitfalls when drawing conclusions from such data.
• Discover the potential and associated risks of leveraging GenAI tools for analyzing unstructured data, specifically user comments collected from surveys.
• Receive tips on delivering a successful presentation of survey results.
• Get some insights how to establish a UX quality control process based on survey data in a larger enterprise, especially what is important to turn UX insights collected over surveys into concrete product improvements.
• Several exercises will help participants to get a deeper understanding of the covered topics.

Table of Contents

1. Basic concepts
2. Example of a typical UX survey
3. Typical application scenarios of UX surveys
4. Some popular standardized UX questionnaires
5. Comparison of surveys to other popular UX evaluation methods
6. How to define effective items?
7. Which UX aspects should be measured?
8. Setting up the survey and collecting data
9. Analyzing UX scores, demographic data and comments
10. Presenting the results and turn insights into action
11. Course summary and concluding remarks