Red Hat Research Quarterly

Mental models: Qualitative research to design for Red Hat OpenShift users

Red Hat Research Quarterly

Mental models: Qualitative research to design for Red Hat OpenShift users

about the author

Carl Pearson

Carl Pearson joined Red Hat as a user researcher after earning his PhD in human factors psychology at North Carolina State University. He employs a mixed-method research approach to uncover the core needs of OpenShift operators and how they use OpenShift.

about the author

Brian Dellascio

Brian Dellascio is a principal user experience visual designer at Red Hat. He feels that beautiful design gets out of the way and becomes a vehicle to disseminate large/complex data sets.

about the author

Sarahjane Clark

Sarahjane Clark is a senior user experience researcher at Red Hat, supporting Red Hat OpenShift. Her focus on finding deep insights through qualitative research studies, backed by solid quantitative data, helps teams design and build products that make operators, system administrators, and developers successful in their work.

Article featured in

To design effectively for our users, we need to learn more about them. If we don’t, we may make a product that our users can’t be efficient in, or worse, a product that our users have no need for in the first place. Our group within Red Hat’s User Experience Design (UXD) team works specifically on the part of Red Hat OpenShift that operators (or system administrators) use to deliver the OpenShift platform on which developers create applications. We wanted to get a firmer understanding of who our users are. What do the users do every day? What motivates them? What challenges do they face? The operator’s problem space is complex, especially to those without a background in system administration.

One way that our UXD team can be sure we’re starting with our users in the design process is by using a set of personas. A persona is a summarization of a segment of users or customers. Each persona segment typically gets a name, a picture, generic attributes (such as income and age), and highlights about their attitudes, needs, and beliefs. 

The marketing field has used personas for decades in their work. For our UXD team, we needed to go even deeper into our users’ backgrounds and work than a typical marketing approach. Because the UXD team designs all minute interactions within a visual interface, we decided to use a specialized persona creation process that prioritizes day-to-day tasks of our users over high-level attributes and beliefs. The mental model approach by Indi Young (indiyoung.com/examples) is a robust technique to visualize the complexity of a job like that of an OpenShift operator.

Mental model research process

The mental model approach uses qualitative interviewing to uncover nearly all tasks related to a user’s main job (in this case, the job of delivering a cloud platform). A task in this case is simply something a user completes for their job that focuses on their goal and not the underlying technology used to do it (write documentation, create a cluster, send an outage message to a customer, and more). 

The interview approach we used focused on exploration rather than confirmation. We aimed to uncover tasks that our users regularly complete that we didn’t already know about. For this reason, our interviews were semi-structured: they had loose prompts but not a checklist of questions we asked. This allowed users to drive the interviews towards what they do in a naturalistic way. We interviewed nine OpenShift users, with a mix of titles and backgrounds, but all worked on OpenShift or Kubernetes platforms. 

To analyze the data, we manually extracted every single task that was mentioned in an interview. We reduced hundreds of tasks to remove duplicates and ended up with over 200 tasks. Scanning a list of 200 tasks is nearly impossible to make sense of, so we grouped the tasks into “task towers” and “mental spaces.” A task tower is a set of highly related tasks (platform health monitoring or scaling the platform). Task towers then fall into a broader grouping of a mental space. This is higher level categorization, such as platform management. Ultimately, the final mental model map is a rich, organized set of all the tasks our users complete in their work. The mental spaces and task towers can be used to navigate the complex space that our users work in. Figure 1  (below) shows our mental model map.

Figure 1.

The mental model map

In this map, each card is a task. Each vertical stack is a task tower, and each gray horizontal bar is a mental space that groups tasks towers. This map was originally all gray until we incorporated our next step, personas. 

The mental model map can be used in a number of ways by designers, product managers, or anyone who wants to explore the full context of users. One typical activity is to map existing product features vertically above each tower where a user goal is met. This can illuminate areas where new features may be needed. Another activity would be a design thinking workshop, a group activity where designers and engineers create solutions to problems. In order to create good solutions, it helps to know where real problems are, and this map can help design thinking workshop participants understand where the problems might be.

The persona process

The mental model alone doesn’t segment the map into who may be more or less likely to spend time on each task. We also developed a set of personas to segment the tasks and clarify how certain users may vary across the map. This is critical for our UXD team because we know that OpenShift users have a variety of goals for typical tasks. We wanted to ensure we could properly design for unique use cases depending on the goals of certain groups of users, for example, developing platform features more often or monitoring platform operations more often.

Using themes drawn from the task towers and mental spaces, we created a set of behavioral variables (from Alan Cooper’s About Face) that represented the spectrum of possible behaviors across our participants. We placed each participant on the variable spectrums. Then we qualitatively assessed how our participants clustered together. These formed the basis for how many personas we developed and what they represented.

Figure 2.

Behavioral variable examples

After splitting up our participants into three personas, we dove back into verbatim quotes to form typical background characteristics, technical environments, goals, challenges, learning strategies, and team networks. We also coded the tasks from the mental model map to show which personas were more likely to complete each task.

Below, we give a snapshot of each persona, but it’s best to get into the full PDF to learn about all the details of a typical OpenShift operator.

Putting mental models to work

We built this set of personas based on data from real users and a rigorous qualitative data analysis process. These personas haven’t been quantitatively validated, but they’re rooted in the real experience of those using OpenShift/Kubernetes. So far, the UXD team has used these results to jump start ideation work in the creation of new, innovative features for our OpenShift operator customers. The mental models and personas have also begun to serve as a platform for quickly bringing new designers up to speed in our highly technical space.

As with any good research project, our work opened up new questions. Our follow up is to explore a gap with an Operations persona, someone mostly concerned with monitoring and troubleshooting production clusters rather than building new cluster features. It’s likely they will not have any fundamentally different atomic tasks to add to our mental model, but will result in a new persona based on how often they do certain tasks.

The mental model map and subsequent personas are a useful set of methods, especially for anyone designing in a technical user space. Our evergreen results speak to the broader nature of being a system administrator and will hold true for many years to come.

Figure 3.

Example layout from The Generalist persona


The Builder

This persona is most concerned with initial building of the cluster and how it will fit into a company environment (consultant is a common role).

Key goals: researching new distributions and features to create PoC clusters or initial configurations to hand off to operators.

Key challenges: working to support an operations team that may lack their OpenShift sophistication and having to keep up with documentation writing.

The Generalist

This persona is who we may think of most commonly as an OpenShift operator. They are capable of building initial clusters and being the expert in their operations as well (SRE is a common role).

Key goals: automating everything possible to open up time for more platform development and to ensure their platform is developed and operated in compliance with the security organization’s requirements.

Key challenges: ensuring developers are enabled to be successful on the platform and navigating scheduling with dozens of different teams using their platform during upgrades/patches.

The Manager

This persona leads a team of SREs and sometimes dedicated operations engineers. They are tied closely to the business needs around their platform but also able to get hands-on in the UI/CLI when necessary.

Key goals:  guiding their team to prioritize automation and to advocate for technology choices with VP level management.

Key challenges: keeping their high expertise SREs from having to spend too much time troubleshooting small issues and keeping costs balanced with optimal platform processes. 

SHARE THIS ARTICLE

More like this