May 22, 2024

February 15, 2023

Undergraduate student in the Rochester Human-Computer Interaction lab seen with a computer screen displaying an AI avatar.Computer science major Sammy Potter ’25 interacts with SAPIEN, an avatar built by the team in the Rochester Human-Computer Interaction lab. Being part of the lab has given Potter exposure to “a new field that I might not have considered otherwise,” he says. (University of Rochester photo / J. Adam Fenster)

With faculty and graduate student mentorship, undergraduate researchers thrive in the Rochester Human-Computer Interaction lab.

Stela Ciko ’25 chose to attend the University of Rochester because of its reputation as a Tier 1 research institution.

“I knew it would offer a lot of research opportunities in whatever field I chose to study, and I wanted research to be a big part of my undergraduate experience,” says the computer science major.

She was able to start substantive work in the Rochester Human-Computer Interaction lab the summer after her first year.

“I have been very impressed with the quality of undergraduate students and the commitment of our graduate students to mentor them.”

—Ehsan Hoque

That’s not unusual at the lab, known as ROC (“Rock”) H-C-I. Housed in the Department of Computer Science, it has given students like Ciko invaluable opportunities to apply computer science to projects that directly help people.

Sammy Potter ’25 didn’t think he would get a chance work in a research lab for at least a couple of years. Then, during his first year at the University, he met Masum Hasan through the Google Developer Student Club. Hasan, a PhD student in the ROC HCI lab, was looking for someone with experience in 3D technology, something Potter had. “I jumped at the opportunity,” Potter says.

Since then, “I’ve had a great experience as an undergraduate hire,” he adds. “ROC HCI is a very friendly community, and I feel respected and that I have an equal voice.” The lab also helped him see new possibilities, giving him exposure to “a new field that I might not have considered otherwise.”

ROC HCI seeks a diverse representation of undergraduates

The Rochester Human-Computer Interaction lab is co-led by associate professor Ehsan Hoque and assistant professor Zhen Bai.

Hoque, who joined the University in 2013, gained international recognition after his PhD thesis at MIT demonstrated for the first time how humans could improve their face-to-face interpersonal skills with a virtual assistant. He has received multiple awards, and more than $9.6 million as a principal investigator to support a broad range of projects.

Bai, who joined the University in 2018, earned a PhD at Cambridge University and did postdoctoral work at Carnegie-Mellon. Her research includes developing learning technologies to support AI (artificial intelligence) literacy for K–12 students and teachers, assistive technology for deaf child communication, and human-AI collaboration in social reasoning.

Both place a high priority on bringing a diverse representation of undergraduates into the lab. There are multiple reasons for doing so, they say.

  • The continuing underrepresentation of women and minorities in computer science hurts the field. “Imagine a lab dominated by one particular group of individuals, and not recognizing the problems that females or minorities might associate with a technology that we are developing,” Hoque says. “Or not having someone with a disability coming in and helping us understand how a technology may create more problems for them.”
  • The interdisciplinary nature of the research makes it “really important for us to bring in students from a lot of background, such as psychology, brain and cognitive sciences, digital media studies, and economics,” Bai says.
  • Engaging undergraduates in research helps build a pipeline to address the shortage of US students applying for PhD programs in the field.
Diptych of Rochester Human-Computer Interaction lab professors Zhen Bail and Ehsan Hoque.

Housed in the Department of Computer Science, ROC HCI is co-led by Rochester faculty Zhen Bai (left) and Ehsan Hoque. (University of Rochester photos / J. Adam Fenster)

Undergraduates at ROC HCI have coauthored papers

Human-computer interaction (HCI) is broadly defined as the design and implementation of computer technology to improve and provide novel interfaces between computers and the people who use them.

Projects that ROC HCI undergraduates have participated in include:

  • Multiple interactive platforms, including LISSA and ROC Speak, to help individuals improve their social skills by simulating face-to-face conversations
  • SOPHIE, an avatar to help physicians and patients communicate better, especially about end-of-life decisions
  • A program analyzing facial gestures to detect whether people are lying
  • Coaching to elevate the creativity of social networks
  • Remote screening of potential cases of PTSD based on free-hand sketches
  • An online framework that can automatically screen for Parkinson’s disease using a standard webcam

These human-computer interaction projects have resulted in more than 50 publications in the last five years alone. Many include undergraduate students as coauthors.

For example, two recent ROC HCI papers—“Assistive Video Filters for People with Parkinson’s Disease to Remove Tremors and Adjust Voice,” presented at the 10th International Conference on Affective Computing and Intelligent Interaction (ACII), and “Systematic Feature Isolation for Bias Research Using Deepfakes,” presented at the 30th ACM International Conference on Multimedia (ACM MM)—received best paper nominations. More than half of the coauthors on these papers are current or former undergraduates. One of the students, Adira Blumenthal ’24 (T5), recently received an honorable mention in the Computing Research Association’s (CRA) Outstanding Undergraduate Researcher Award program for 2023.

Rochester Human-Computer Interaction lab graduate student explains concepts to undergraduates, two o whom are seen out of focus in the background.

PhD student Masum Hasan meets with undergraduates in the Rochester Human-Computer Interaction lab discussing presentations and new ideas. (University of Rochester photo / J. Adam Fenster)

Undergraduates work closely with graduate student mentors

“I have been very impressed with the quality of undergraduate students and the commitment of our graduate students to mentor them,” Hoque says.

Day to day, students work mostly with the graduate students in charge of projects.

Ciko, for example, works with PhD student Adiba Proma on a project that involves building a website where shared computational data about climate change and natural disasters from labs around the world can be gathered in one place for easy access by other researchers.

The project involves reading papers that involve machine learning, an area new to Ciko. Proma “was very good at guiding me through the entire process,” she says.

Potter works with Hasan primarily on two projects in the lab: creating a more sophisticated hand tracking system for Parkinson’s disease detection and a more versatile natural acting virtual avatar than the one used in SOPHIE for conversations with patients.

“As an undergrad I was surprised by the amount of freedom he gave me to figure things out on my own,” Potter says. “I appreciate being able to bring my own ideas to our projects.”

Vertical diptych featuring photos of Luke Gerstner and Caleb Wohn.

Luke Gerstner ’20 (top) and Caleb Wohn.

Luke Gerstner ’20, a data science major, joined ROC HCI to fulfill a requirement to do summer research at the University as part of his participation in the McNair Scholars program. He ended up working in the lab during his last two years at the University, contributing to a couple of published papers. One of them involved extracting facial expressions from videos to compare the expressions by gender to detect deception. Gerstner worked closely with Gazi Naven, a fellow undergraduate.

“We went through many revisions, and spent some late nights working in the lab, working with graduate students, and getting feedback from Ehsan on the paper. It was a really great project. And a great learning experience,” Gerstner says.

Gerstner is now a data scientist and software engineer with Rosen USA, part of an international company that specializes in inspection devices for pipelines and other complex technical systems.

Weekly ROC HCI team meetings provide another valuable learning experience for undergraduates.

Caleb Wohn ’22, now an implementation consultant at Fast Enterprise LLC, says, “I think I learned even more from the weekly lab meetings and especially the paper readings, where a grad student would present a paper relevant to the lab’s research and explain all its important concepts and lessons.”

And, he adds, “I learned how to communicate in the research community.”

The academy or industry?

Just as an internship can help students decide if want to pursue careers in industry, a research experience can similarly help someone determine if they foresee a future in academia.

Close headshot of smiling student against blue background.

Adira Blumenthal ’24

Blumenthal, for example, was attracted to the lab because of her interest in accessibility and assistive technologies. One of her projects involved interviewing 177 people with Parkinson’s disease, asking if they would approve of the use of filters to remove the tremors from their body or voice during videoconferencing, and whether this would raise ethical concerns.

However, despite having won an award for a paper, she often felt there was not enough time to broaden the scope and potential impact of projects, because of pressure to get papers published quickly. She plans to look for a job as a software engineer either in the production of assistive technology or in making general products more accessible.

Still, the opportunity to talk directly to people with Parkinson’s about their concerns and involve them in the development process was “meaningful and impactful, and something that will be really important going forward,” Blumenthal says.

Skills ‘you can’t get from a class, or even an internship’

Wohn has already experienced how skills he gained at ROC HCI translate to private industry.

“I just finished working on a report that is automatically generated for users, showing information they need to make decisions and work through certain processes,” he says. “This project was pretty much exactly like what I did for the SOPHIE project.”

In both projects, he met with experts in a non-technical field to figure out how he could build a tool that would provide them with useful information. “Communicating with non-technical people, interpreting feedback, and presenting my work turned out to be extremely useful skills I learned from working with the lab.”

There’s one more life lesson he learned there. Projects in the lab usually take months, sometimes even a year or more to complete. They entail patience and planning many undergraduates don’t practice anywhere else.

Says Wohn: “When you’re an active participant in planning out a project and have to consider things a year or two years down the line, that’s something you can’t get from a class, or even an internship.”

Read more

Student holding camera stands in front of a case full of human skulls at a dark tourism destination.The ethics of dark tourism

Julia Granato ’23 crisscrossed Europe to study human bone collection and display sites. Now she’s pondering what it means to display and visit human remains.


Tags: artificial intelligence, Department of Computer Science, Ehsan Hoque, Hajim School of Engineering and Applied Sciences, undergraduate research, Zhen Bai

Category: Campus Life