Remote learning and remote working have forced nearly all interactions between students and prospective employers online. With virtual meetings, there is a whole new set of impression factors to consider. How is the audio coming across? How is the video connection? Is what’s within the frame of the camera influencing one person’s impression of the other?

Jumpstart, an early career recruiting platform for students and companies in tech, recently posed a challenge for students around the country, with a chance to win a $10,000 fellowship: Consider the problem that thousands of students are facing right now—internships, campus events and career fairs have been canceled—what product or initiative would you create to help students connect with companies during these tough times? 

Jazz Ang, Ana Liu, and Della Sigrest, first-year Master’s students in the Department of Human Centered Design & Engineering, understood the challenge all too well. But where many saw constraints, they saw opportunities to improve online meetings across the board.

To begin, the team wanted to make sure they were focusing on the right question.

“From talking with our peers, we discovered that connecting isn’t really the problem,” Sigrest said. “There are already online platforms out there and networking opportunities to introduce people. We learned that relationship building is the real challenge. And so we decided to ask how can we improve video communication platforms to help people present their best selves online.”

Virtually Better

Mockup of the Virtually Better prototype shows the Action Center, a pop-up that overlays the video call when the user first enters a video call. The pop-up displays recent insights such as eye contact, filler words, and speech pace. From this action center, the user can navigate to view past statistics, personalize, their experience, or manage their account.

To Jumpstart’s Fellowship, the HCDE team proposed Virtually Better, a plug-in to work with video conferencing programs and provide virtual insights to help people improve their communication.

Features the team is exploring include on-screen and auditory feedback about how the speaker’s voice is coming across—from the audio connection to the pace of their speech—to how movements or background distractions are detracting from the conversation.

The team wants to ensure their product is accessible to the widest range of users, bringing in features to help blind and low vision users and those who are deaf or hard of hearing. “For non-visual users, we hope to provide speech-based feedback to let people know how they are presented on the screen, Ang said. “And for people who are deaf or hard of hearing, we propose a hands-free note taking feature.”

“Our overall goal is to democratize virtual insights into your video conversations to benefit everyone. Not just students, but for everyone else who is navigating this new world of conducting meetings virtually,” Ang said.

$10,000 and 6 weeks

In early May, Jumpstart announced that Ang, Liu, and Sigrest are one of three teams to receive the Work Together fellowship, granting them $10,000 to work on their idea during the summer, with the guidance and support of industry experts. 

Liu is the team’s technical lead, bringing a background in computer science, and the team is actively searching for other machine learning engineers to partner with. “We’re starting our testing with different APIs, to evaluate them first before we decide what we will build into our project,” Liu said. “At each stage we’re developing our conceptual design ideas and testing those out with our peers, and continue iterating as we go.”

At the end of the six weeks, the team hopes to have their proof of concept developed, while continuing to seek other fellowships and accelerators that can help to bring their project to life.

The HCDE way

Sigrest credits her team’s success with the fellowship because they used many of the methods and skills they’ve learned in the HCDE program. “I know the reason we succeeded in getting this fellowship is because we turned to our users first to ask what their needs are here, instead of just running with the prompt that assumed what the user wanted,” she said. 

Ang cited HCDE’s course on User-Centered Design (HCDE 518) as influential in their process. “The whole foundation of HCDE 518 really taught us how to go back to the drawing board, reframe the design question, go back to the user, frame our pitch, and go back to the user again.”

“Our time in HCDE has also taught us how to generate conceptual design ideas after interviewing our target audiences,” said Liu. “We’re now able to effectively present these concepts in a way that makes sense with people, and can help generate new ideas.”