UH Hilo data scientist Travis Mandel finds intellectual freedom in tenure

Associate Professor of Computer Science Travis Mandel, who arrived at UH Hilo in 2017 and was awarded tenure this year, exposes his students to real-world research in both computer science and data science.

Travis Mandel
Travis Mandel

By Susan Enright.

This story is part of a series on newly tenured faculty.

Opportunities for students to do hands-on research in the fields of computer science and data science are not easily found on Hawai‘i Island. But Travis Mandel, an associate professor of computer science at the University of Hawai‘i at Hilo, decided to do something about that.

Mandel, who arrived at UH Hilo in 2017 and was awarded tenure this year, makes it a priority to expose his students to real-world research in both computer science and data science. Currently, he has six UH Hilo undergraduates working on research projects involving these fields, and he has plans to involve at least another six to 10 students next summer.

“I think my most important contribution [to UH Hilo] has been exposing students to what real-world research in computer science and data science looks like,” he says. “Before I started there were very few opportunities for UH Hilo students to get involved in computer science research or internships on this island. This is a unique problem, as at schools on the mainland students can often just drive to the nearest major city for a plethora of computer science related internships or research experiences.”

In addition to working directly with students, Mandel also is continuing to improve curriculum at UH Hilo through developing a data science major; data science is currently offered only as a certificate program. Data scientists solve challenging real-world problems in a variety of fields of study through collecting, analyzing, and visualizing complex datasets. It is a versatile field because almost every branch of science collects loads of data—big data—and has processes for analyzing all the information. These analytical skills are in  high demand in the workforce and also something many students find interesting. He is hopeful the major will be offered soon.

Meanwhile, Mandel’s teaching style and his inclusion of students in research is highly successful in guiding budding scientists to produce high-quality projects. In 2020, Mandel published a journal article at a top human-computer interaction journal co-authored with six UH Hilo undergraduates, and in 2021 he published a conference paper at a top AI conference co-authored with four UH Hilo undergraduates. Just recently, he published the journal article, “Detection confidence driven multi-object tracking to recover reliable tracks from unreliable detections,” in a top computer vision journal co-authored with another six UH Hilo undergraduate students.


Travis Mandel (red aloha shirt) stands for photo in garden with two women and three men.
Travis Mandel (center in red shirt) stands with students from one of his summer classes on artificial intelligence. From left, Ryp Ring, Ethan Sick, Travis Mandel, Olivia Jarvis, Jennifer Nakano, and Rodel Tagalicud. The course, a Human-in-the-Loop AI summer research experience sponsored by Mandel’s National Science Foundation CAREER award, was held this past summer. (Courtesy photo from Travis Mandel)
Five students at computer stations in classroom. Prof. Mandel is showing students information on an overhead screen.
Travis Mandel teaches students in a 2022 summer class focusing on research problems in human-in-the-loop artificial intelligence or AI. (Courtesy photo from Travis Mandel)

Even for his students who do not participate in his research directly, Mandel tries as much as he can to connect what he’s teaching in the classroom to real-world issues he encounters in his research.

“There is a misconception that professors can only talk about their research in highly-specialized upper-level courses,” he says. “I try my best to avoid this by weaving my research into 100-level introductory courses, as well as courses such as systems programming which on face value seem far from my research area of human-in-the-loop artificial intelligence. It helps students recognize the importance of the material being taught in class when they connect it to a real-world computer science problem that I encountered in my recent research.”

Research

Mandel is an expert in Human-In-the-Loop Artificial Intelligence (AI), a data-driven process that improves models and algorithms through human intervention and contribution to create better and more accurate AI.

In his research, Mandel asks the question, “How do we create AI systems that work effectively with human teammates to achieve better results than either could individually?” This area of research lies at the intersection of two fields: AI and Human-Computer Interaction or HCI.

“I am particularly interested in exploring how AI can most effectively work together with human scientists, identifying and addressing common problems that arise in a number of different scientific domains,” he explains on his website. “This has led me to do new work in the field of Computer Vision (CV) to meet the needs of Hawai‘i Island scientists who wish to extract important scientific data from images and video.”

In 2020 Mandel was awarded a National Science Foundation CAREER Award to advance this research agenda, and in 2022 he joined a statewide program funded by the National Science Foundation to help develop new Human-in-the-Loop AI techniques for climate science.

Mandel’s recent research includes published work in which he and his research team developed new computer vision algorithms that allow computers to track fish effectively from video feeds collected by real-world divers off of Hilo Bay. The algorithm outperformed a wide variety of competitors across multiple datasets. The researchers are now working on next steps, which are understanding how people actually interact with such systems. “We have built an Android app which runs a version of our tracker at near real-time on Android phones. We hope that this can be deployed underwater in the near future,” he says.


Image above shows new fish counting algorithm, developed by data scientist Travis Mandel and his students, is working well on video from Hilo Bay. The development of this AI algorithm was recently published in a top computer vision journal. (Courtesy photo from Travis Mandel)
Data scientist Travis Mandel and a team of UH Hilo students built a fish counting Android app that runs a version of an underwater tracker at near real-time on Android phones. (Courtesy photo from Travis Mandel)

In other work, Mandel is helping better annotate invasive species data in collaboration with UH Hilo geographer Ryan Perroy who is investigating Rapid ‘Ōhi‘a Death in Hawai‘i forests. Mandel and Perroy are in the process of developing software that shows the potential to save up to 40% of user’s time by adjusting to both the user and the machine learning model. Mandel says this can help accelerate management of ROD and other harmful invasive species in Hawai‘i.

Mandel also is working on several collaborative projects with a wide variety of climate scientists from across the UH System funded by a statewide grant from the National Science Foundation. “We are working on problems involving important factors that affect climate change such as carbon sequestration, cloud water interception, and soil moisture,” Mandel explains. “The goal is to build Human-in-the-Loop AI systems that help these scientists more easily collect and label data, which will result in tools that help individuals and organize understand the impact their behavior has on climate and hopefully make a positive change.”

Benefit to the community

In terms of reach, since coming to UH Hilo, Mandel has adjusted his research agenda to better meet the needs of Hawai‘i, and Hawai‘i Island in particular.

“When I started, I had just completed my PhD in which I worked on using AI to make educational videogames more entertaining and effective,” he says. “But when starting here, it became clear that there was a huge unmet need for AI and data science skills in the natural sciences. Of particular interest was computer vision, due to the large need to analyze images and video of coral reefs, marine life, and invasive species.”

Despite never having done any research in computer vision before, he took on the challenge of learning the field so as to better meet the massive need for this field among scientists at work on Hawai‘i Island. “This has led to many fruitful collaborations,” Mandel says. “For instance, in one of my first forays into computer vision, I was able to help Dr. Ryan’s Perroy’s lab greatly increase their accuracy at identifying Rapid ‘Ōhi‘a Death, which has greatly eased management efforts.”


Visualization of rapid ohia death in a swath of trees.
Data scientist Travis Mandel helped geographer Ryan Perroy increase his lab’s accuracy in identifying Rapid ‘Ōhi‘a Death, which greatly eased management efforts. In the above image, trees in the forest affected by ROD are shown as red dots. (Screenshot from video on Rapid ‘Ōhi‘a Death Data Visualization Research/UH Hilo)

The freedom of tenure

Mandel has found intellectual freedom in tenure, awarded to him earlier this year.

In navigating AI ethics, he has observed a growing resistance in the AI community to being open-minded about what problems need to be solved and how to go about solving them. His approach is to identify real-world problems and come up with innovative and effective general-purpose solutions to those problems.

“I am open to exploring all methodologies to solving these problems, and am also open to identifying new problems if existing problem formulations are not a good fit,” he says.  “This is challenging because it conflicts with how some people think modern AI research should be conducted. Tenure gives me the freedom to explore these high-risk and potentially disruptive ideas without fearing retaliation.”

Tenure also gives him the freedom to teach challenging topics to students using innovative curriculum.

“For instance, I am scheduled to teach a new course on computer vision in the spring.  Although I think the class will be a lot of fun and very well received, there are no up-to-date textbooks for the class and so it will take a lot of work to develop it. Tenure gives me the freedom to take risks like teaching this class, coming up with new activities and course content, etc. without fearing that one semester of poor student course evaluations will cause me to be dismissed from my position.”

What’s ahead

Looking to the future of his career, Mandel looks at AI contextually, seeing that the field continues to become more and more capable, achieving incredible feats such as generating professional-looking artwork or essentially solving the protein folding problem in biology. But, he observes, these advances have been driven by a massive amount of data, and there are a huge number of important problems where there is no data available yet.

“In these cases, the challenge becomes how to collect that data effectively, how to make the best use of human effort to label the data, and how to make AI systems that work as well as they can with such a small amount of data,” he explains. “All these things involve collaboration between the AI as well as humans who are making decisions about data collection, annotation, and policy. I am excited to continue to explore this exciting space of problems, forming new interdisciplinary collaborations while sustaining the existing ones.”


Story by Susan Enright, a public information specialist for the Office of the Chancellor and editor of UH Hilo Stories. She received her bachelor of arts in English and certificate in women’s studies from UH Hilo.