Problem Overview
As a top game streaming service, Twitch has millions of users watching streams every day. And as a growing and ubiquitous service, it is essential that its chat system is inclusive and provides an optimal experience for diverse users.
A significant benefit is that the channel community on Twitch allows everyone to have an equal opportunity to contribute to the conversation in chat. However, chats in popular live streams tend to move rapidly with increasing viewers participating in the discussion, which leads to the chat overloading with different messages. This behavior denies the opportunity for viewers to have a meaningful conversation with the streamer. As a result, chat participants disengage from the streaming service.
Project Goals
To understand the experience of Twitch viewers who struggle reading the chat to design potential improvement features.
My Roles
UX Design Researcher
Tools
Miro
MS Excel
Webex
Figma
Qualtrics XM
My Contributions
I conducted a literature review to understand the current chat readability challenges and previous work to address such issues. I also designed and analyzed our surveys, semi-structured interviews, usability testing, and expert evaluations. Additionally, I translated the discussions and usability sessions from Spanish to English. Also, I collaborated with design ideas and sketches.
Team
Maria-Paula Lengua
Maxwell Nelson
Stacy Lee
Ramisa Murshed
Timeline
August 2021- December 2021
Phase 1: Research
Survey
Objectives:
- Understand Twitch users' goals and pain points while reading the chat.
- Recruit participants for interviews
Rationale: The survey allowed us to retrieve information in a short period of time and help us prepare interview questions.
Method:
-We developed a 17 questions survey using Qualtrics XM.
-We distributed our survey link on three Subreddit groups, one Facebook group, and three Discord servers.
Analysis
We received 49 responses within a week of distribution. Out of the 49 respondents, 47 identified as Twitch users, we only counted their responses for the rest of our analysis.
Findings
-More than 50% of respondents selected that the chat was the service they use the most.
-Participants stated that the chat goes too fast and they are not able to read anything.
-More than half of the respondents identified as Twitch viewers.
-More than half of the respondents identified as Twitch viewers.
-More than 50% of respondents selected that the chat was the service they use the most.
Participants stated that the chat goes too fast and they are not able to read anything.
Semi-Structured Interview
Objectives:
To further understand users’ reading difficulties, as discussed in their survey responses, asked interviewees about their experiences having difficulties reading Twitch chat
Rationale: Semi-structured interview will allow us to ask follow-up questions to get a deeper understanding of users’ insights.
Methods:
We conducted remote interviews through Webex and Discord.
Analysis
We entered our interview notes and transcripts into a spreadsheet. Then, we added them to Miro and created an affinity diagram. Individually, we walked the wall, and we came back together to discuss further implications. See below our diagram.
Findings
We identified the following themes from our user interviews:
Phase 2: Design Implications
Based on our research findings and insights from phase 1, we developed the design implications for our future prototype including our user persona and empathy map.
User Persona
Empathy Map
Phase 3: Design
Based on our design implications, we brainstormed ideas for our future prototype. See below our design timeline
Prototype
Feature 1: Chat Settings Redesign
Our initial research showed that users were unaware of the existing customization capabilities that Twitch provided for the chat view. The font customization feature is currently hidden in the chat feature of Twitch, and we restructured the UI so that the chat setting is at the primary level.
Additionally, having a clear chat setting provides a centralized place where users can easily discover other chat functionalities. To increase legibility, we also included the ability to change font style, to better accommodate readers who may have a harder time reading certain fonts.
Feature 2: Chat Playback Speed
Our research showed that users had varying experiences keeping up with the Twitch chat feeds based on which channels they followed.
Chats of channels with fewer viewers and chat participants were easier to keep up with, however, chats of channels with a large number of viewers were hard to read. By having a speed customization feature within each channel chat, the user can customize speed settings based on each channel.
When doing usability testing, we noticed that it was difficult for some users to navigate directly to the chat playback speed so we moved it over to the main chat settings page for easier access. By adding this option to the main chat settings screen, we aim to assist users in saving time to change the playback speed of their chat.
Feature 3: Emoji and Rules Revamp
Our research showed that users wanted to feel connected to the chat while watching a video gaming streamer. In the new navigation bar added to the bottom of the screen, a button was added to display the chat rules and other chat-specific information. A list of stream-specific chat rules would be displayed in addition to the emote library which described each emotion each individual emoji conveyed.
Feature 4: Chat Highlights: Understand what the general chat is saying
Users emphasized wanting to understand the general gist of chat in a manner that took a short amount of time. To address this want, we created a new feature to add to the Twitch chat called CrowdSpeak AI. This functionality took what users were saying in chat and described it in a short blurb that was visible to all users viewing the chat. For example, if users were talking about how to slay a certain mob in Minecraft multiple times in a short time span, the chat highlighting functionality powered by an AI would summarize in a short line of text for users to quickly scan and understand what is going on.
When “What has the chat been saying…” button is clicked, the messages that helped the AI create the Chat Highlight are shown in a list based on the time they were said. In future iterations, we would add a search bar functionality that allows users to search for specific keywords that were mentioned in the chat.
Feature 5: Icon-based user interface added to the bottom of the chat
After noticing that we needed more space in the chat area to perform new functions, our team designed a fresh look for the overall UI of the chat window. Rearranging the existing buttons into a central bottom location allows for easier access. Icons were added near the bottom of the chat window to indicate what pop-up would appear on top of the chat. This allowed users to still see what was being said in the chat while looking up information related to their experience. Top gifters were moved from the top to the bottom of the chat window.
At the top of the chat window, a slider was added to scroll through limited events that are happening in the chat. The arrow on the slider is used to scroll to other limited events such as Chat Highlights or chat surveys.
Phase 4: Design Evaluation
To evaluate our prototype we conducted expert evaluations and usability testing.
Expert Evaluation
Objective: to uncover the common usability issues that experts can quickly identify following the heuristic principles.
Rationale: Expert heuristic evaluation helped us identify usability problems that non-experts might not notice.
Method:
-Three Experts evaluated our prototype according to Nielsen Norman’s 10 Heuristics.
-We asked experts to think aloud while going through our prototype and we asked follow-up questions.
Analysis
To analyze the heuristic evaluation, we grouped the three expert responses into a spreadsheet to analyze and compare each expert’s scores and comments.
Findings
-The three experts agreed that there were cosmetic issues with our prototype but should be given less priority.
-Experts suggested that giving the users to ability to preview the setting could improve our visibility of system status.
Usability Testing
Objective: To test whether we were creating a user interface that is simple, intuitive, and easy to understand for our target user group as well as meeting their needs.
Rationale: we wanted to test our prototype with Twitch users to gather feedback from our target population.
Method
-We used a task-based usability test with embedded semi-structured interviews after each task.
-We used Figma to display the prototype to participants, allowing them to click through
and interact with the prototype on their own.
-We used WebEx to conduct virtual usability tests.
-We asked participants to fill out a System Usability Scale (SUS) survey once they finished all the tasks.
Analysis
-We analyzed the SUS scores of each participant following the SUS guidelines
-We organized the notes from our usability study to Google Sheets, and then we transferred to a Miro Board.
-We divided the notes by each task, and we collectively worked to find common themes that participants brought up during the session.
Findings
-The average score from all participants was 71 which is close to the average for a system that is likely to be recommended to use by others according to the SUS guidelines.
-We found a common issue that participants had with the information architecture of our design.
Recommendations
After conducting usability sessions and expert evaluations, we were able to draw significant
conclusions about how the design of our prototype should change going into further iterations.
We identified three key issues that our design would need to address. We received great information that allowed us to look at our prototype differently.
Discussion
Watch our final presentation to learn about our work!
