Platform teaches nonexperts to use machine learning

Machine-learning algorithms are used to find patterns in data that humans wouldn’t otherwise notice, and are being deployed to help inform decisions big and small—from COVID-19 vaccination development to Netflix recommendations.

New award-winning research from the Cornell Ann S. Bowers College of Computing and Information Science explores how to help nonexperts effectively, efficiently and ethically use machine-learning algorithms to better enable industries beyond the computing field to harness the power of AI.

“We don’t know much about how nonexperts in machine learning come to learn algorithmic tools,” said Swati Mishra, a Ph.D. student in the field of information science. “The reason is that there’s a hype that’s developed that suggests machine learning is for the ordained.”

Mishra is lead author of “Designing Interactive Transfer Learning Tools for ML Non-Experts,” which received a Best Paper Award at the annual ACM CHI Virtual Conference on Human Factors in Computing Systems, held in May.

As machine learning has entered fields and industries traditionally outside of computing, the need for research and effective, accessible tools to enable new users in leveraging artificial intelligence is unprecedented, Mishra said.

Existing research into these interactive machine-learning systems has mostly focused on understanding the users and the challenges they face when navigating the tools. Mishra’s latest research—including the development of her own interactive machine-learning platform—breaks fresh ground by investigating the inverse: How to better design the system so that users with limited algorithmic expertise but vast domain expertise can learn to integrate preexisting models into their own work.

“When you do a task, you know what parts need manual fixing and what needs automation,” said Mishra, a 2021-2022 Bloomberg Data Science Ph.D. fellow. “If we design machine-learning tools correctly and give enough agency to people to use them, we can ensure their knowledge gets integrated into the machine-learning model.”

Mishra takes an unconventional approach with this research by turning to a complex process called “transfer learning” as a jumping-off point to initiate nonexperts into machine learning. Transfer learning is a high-level and powerful machine-learning technique typically reserved for experts, wherein users repurpose and tweak existing, pretrained machine-learning models for new tasks.

The technique alleviates the need to build a model from scratch, which requires lots of training data, allowing the user to repurpose a model trained to identify images of dogs, say, into a model that can identify cats or, with the right expertise, even skin cancers.

“By intentionally focusing on appropriating existing models into new tasks, Swati’s work helps novices not only use machine learning to solve complex tasks, but also take advantage of machine-learning experts’ continuing developments,” said Jeff Rzeszotarski, assistant professor in the Department of Information Science and the paper’s senior author. “While our eventual goal is to help novices become advanced machine-learning users, providing some ‘training wheels’ through transfer learning can help novices immediately employ machine learning for their own tasks.”

Mishra’s research exposes transfer learning’s inner computational workings through an interactive platform so nonexperts can better understand how machines crunch datasets and make decisions. Through a corresponding lab study with people with no background in machine-learning development, Mishra was able to pinpoint precisely where beginners lost their way, what their rationales were for making certain tweaks to the model and what approaches were most successful or unsuccessful.

In the end, the duo found participating nonexperts were able to successfully use transfer learning and alter existing models for their own purposes. However, researchers discovered that inaccurate perceptions of machine intelligence frequently slowed learning among nonexperts. Machines don’t learn like humans do, Mishra said.

“We’re used to a human-like learning style, and intuitively we tend to employ strategies that are familiar to us,” she said. “If the tools do not explicitly convey this difference, the machines may never really learn. We as researchers and designers have to mitigate user perceptions of what machine learning is. Any interactive tool must help us manage our expectations.”