From the rainforests of Hawaiʻi Island to UH Hilo’s new AI lab, student researchers explore AI connections to natural science
This summer’s AI research group was the first ever student research experience at UH Hilo’s new AI Lab, with the first field trip to test equipment and appreciate the natural world under study.
By Susan Enright/UH Hilo Stories.
Over the summer, nine students with their mentor, Associate Professor of Computer Science Travis Mandel, investigated various artificial intelligence research problems using the natural environment of Hawaiʻi Island and a new AI lab at the University of Hawaiʻi at Hilo for their research. The eight-week program, based at UH Hilo’s newly renovated and renamed Scientist Centered Artificial Intelligence Laboratory or SCAIL, is sponsored by the National Science Foundation.
“The students have been working hard on four different projects,” says Mandel, whose summer programs focus on the use of human-in-the-loop AI and its connections to natural science. The 2026 projects were presented at a gathering on campus July 31 with topics ranging from AI-assisted coding to the efficiency of annotation AIs.
Almost every summer since 2018 (skipping 2024), Mandel has immersed a new cohort of students into human-in-the-loop AI research, each session supported by the NSF. This year’s program included two exciting firsts.
This summer’s AI program was the first ever research experience at UH Hilo’s new SCAIL facility. Mandel’s AI research group, established in 2017, fully settled into the new Scientist-Centered Artificial Intelligence Laboratory over the last few months with the support of a $5,000 grant from the Office of the Chancellor to rebrand and upgrade the lab to its new identity.
- With an infusion of funds, UH Hilo’s data science lab is transforming into a scientist-centered AI lab (March 11, 2026, UH Hilo Stories)
“The name helps to let students know this is an important center of research on campus,” says Mandel. “The name reflects the fact that we’re very focused on not just AI, but AI that helps scientists do their job better.”
This year’s program was also the first time Mandel’s summer students went on a field trip during their AI research experience.
“They went to Hawaiʻi Volcanoes National Park to test computer vision software and to learn more about the natural environment they were studying,” says Mandel. “Many students had not been to the Big Island before the experience.”
The students
The nine undergraduate or recently graduated students are from three universities: UH Hilo; Gonzaga University, Spokane, Washington; and Lewis & Clark College, Portland, Oregon. Here are their names, schools and majors:
UH Hilo
- Alex Bridges (computer science)
- Josh Kralewski (data science)
- Paige Matheson (astronomy)
- Richard Oliver II (data science)
- Raine Kyro Visaya (data science)
Gonzaga University
- Federico Brown (applied mathematics)
- Abraham Meshesha (computer science)
- Bert Collins (computer science)
Lewis & Clark College
- Xavier Huang (computer science)
The projects
Finishing Your Sentences: Personalized AI Code Completion for All
Presenters: Paige Matheson, Bert Collins, Abraham Meshesha
Abstract: As AI assistants roll out into every corner of the internet, the constant question is how to make them more efficient and helpful, especially for science. One particularly challenging domain is AI-assisted coding, where AI can sometimes actually increase the time taken to solve a problem. Over the last eight weeks, our team contributed to a coding interface that autocompletes code line by line with AI-generated suggestions as a testbed for an AI filtering algorithm that seeks to limit AI output to places where it is most useful. Previously the interface relied on external AI providers with models that were neither fully open nor geared for coding; however, we implemented a backend using a recent model from Allen Institute for Artificial Intelligence (AI2), after a rigorous performance evaluation. We developed a new metric for comparing code similarity and compared its performance on a large dataset. We also added and improved a variety of things from adding new test problems from a standard object-oriented coding dataset to designing a robust line-tracking system and even patching critical security vulnerabilities. Finally, with the code for running the experiment now nearing completion, we have prepared the forms for the Institutional Review Board to review.
Metrics: Towards a novel way to measure AI success
Presenters: Federico Brown, Raine Kyro Visaya
Abstract: Model development often optimizes for performance metrics like accuracy and helpfulness. Yet these measures say little about whether a model’s uncertainty estimates are themselves reliable, a gap that is especially acute in real-world settings. To address this, we contribute a framework for evaluating the reliability of uncertainty estimates in AI systems rather than model accuracy alone. The framework is built around two testbeds spanning distinct scientific domains, namely spatial interpolation of Hawaii climate data and factual question answering. Both are assessed through a unified metric layer that combines standard calibration measures with novel utility and confidence scores. We evaluate a range of AI systems, including recent uncertainty-quantification methods and large language models from the Allen Institute for Artificial Intelligence (Ai2).
The Rise of FishNet: Enhanced Neural Network Based Multi-Object Tracking
Presenters: Alex Bridges, Xavier Huang
While multi-object tracking AI models have advanced rapidly alongside technologies like self-driving cars and autonomous robots, applying them to a specific field — such as our current focus on tracking fish — reveals significant challenges. One of the major problems the field of multi-object tracking currently faces is the lack of effective evaluation metrics. The industry-standard metrics, HOTA and MOTA, are good at evaluating how well boxes match up, but may not reflect how much an AI helps with actual tasks. To highlight the real world performance of these two systems and a baseline for future analysis, we have contributed to a platform for evaluating MOT systems in a realistic fish tracking setting. This summer we implemented a simple yet robust AI user simulator designed to mimic human player interactions, such as searching for visual targets (primarily bounding boxes of fish) and interacting with on-screen UI elements. To support these systems, the entire codebase underwent significant changes across both the Unity frontend and the Python backend, prioritizing modularity and readability. To improve detection performance, we explored new object detection systems, including from Ai2, and implemented a new module using native Android libraries and device-specific compilation to further optimize real-time performance.
If It Can’t Walk, then It Can’t Run: Analyzing The Efficiency of Annotation AIs
Presenters: Josh Kralewski, Richard Oliver II
One of the grand challenges of AI is building AI systems that can process almost any kind of input in an efficient way, quickly learning about important aspects of the data based on user feedback. Despite recent progress in the field, computer vision systems often fall into two categories: Those that are designed to process only one particular format of image data, and Large Vision Models (LVMs), which in theory can process many formats, but are typically evaluated with only one format of data at a time. Our goal is to evaluate the capabilities of AI systems when fed many different formats of user feedback, for instance boxes around objects or tracing the outlines of objects. This presents many challenges from a software engineering perspective, as it requires a robust pipeline that can convert between many different formats of data and coordinate many different AI systems and interfaces. This summer, we implemented a carefully designed object-oriented paradigm, which enabled us to identify bugs, redundancies and efficiencies, as well as providing a more extensible platform for future development. Additionally, we created a comprehensive test harness for evaluating the learning process of AI systems, enabling us to better identify the cause of weaknesses in the learning performance. Finally, we explored integrating new LVMs from the Allen Institute for Artificial Intelligence (Ai2).
Support
The research was sponsored by the National Science Foundation:
- Award #RI2-2413244 (funding principle investigator, students, materials and supplies)
- Award #OIA-214913 (funding students, equipment, field trip, materials and supplies)
- Award #HCC-1942229 (funding students, materials and supplies)
Other support came from the NSF ACCESS Program (computational resources) and Jetstream2 (computational resources).
Campus support included the UH Hilo Research Council (award to support food and desks), Office of the Vice Chancellor (lights and furniture), Department of Astronomy (field trip transportation), UH Hilo Housing (student housing), and Vegan Shop (food).
Related posts
Students work on interface improvement of artificial intelligence
Story by Susan Enright, public information specialist for the Office of the Chancellor and editor of UH Hilo Stories. She received her bachelor of arts in English and certificate in women’s studies from UH Hilo.
