Time
Three months
My Role
UX designer
Team
Rachel Celestine, Chia-Hua Li, Feifei Guo, Matthew Seita
Methods
User-centered design, Interview, Method card, Sketch, Paper prototype, User testing
Tools
Sketch, Principle
Project Goal
Create more seamless real-time communication between DHH and hearing individuals.
How deaf and hard of hearing (DHH) and hearing people communicate in real time?
User Research
Participants: two DHH users and one hearing user
Personas
DHH user’s pain point while using current ASR technology:
Hearing user’s pain point:
Sketches & Brainstorm
Design Concept
This proposed design is meant to accommodate a wide range of conversation types, from impromptu 1-on-1 conversations to group discussion. We decided our product would be a mobile-based messaging application utilizing speech-to-text technology along with other unique features, such as conversation control and live video. Our solution aims to empower users by improving their capability to perceive the emotion and tone of their group.
Low-Fidelity Prototypes
Create a chat room to start a conversation
Open/Adjust the live video
Caption correction
User feedback
Some users can’t find the caption correction feature.
The live feed icon is confusing. It is similar to the video call icon in other message apps.
The position of the resize icon and close icon on the small video are odd and might block the video.
High-Fidelity Prototypes
Create a chat room
Iteration
Add tooltips to introduce unique features to first-time users.
Add a quick chat mode
Open/Adjust the live video
Iteration
Change live feed icon
Modify the live video layout
Caption Correction
Iteration
Add new edit option so users can correct the wrong words manually
User feedback
Some users prefer a smaller font to avoid scrolling up frequently to find the previous conversation.
Some users mentioned that there were too many words in tooltips, they might tap “got it” without reading them.
Final Design
Feature 1: Create a group chatroom and quick chatroom
A group chatroom
Member can create a chatroom and invite other participants (members or guests) in the meeting. (guests can only join a room)
Feature 2: Use the live video
Live video
It’s a live video of the speaker by using the front camera on the phone. Users can tap on the ‘live’ button on the top-right corner to open the live feed feature.
It gives DHH individuals the opportunity to view the speakers facial expression, nonverbal responses, and overall reaction in a group settings.
Feature 3: Caption correction
Correction Suggestion
Correct manually
When users tap on a specific word or a phrase, recommended words or phrases will pop up and they are able to replace it by choosing the correct one. Users can select the ‘edit’ icon to type if the recommended words are not correct.
Feature 4: Conversation Control
Take turn talking
The “take turn talking” feature aims to prevent the system from recording two users’ voice at the same time, so users have to wait in line for speaking. Only one device’s microphone can be turned on at the same time.