On Tuesday, Sept. 2, University of Idaho professor Bert Baumgaertner kicked off the fall semester’s Malcolm Renfrew Interdisciplinary Colloquium with “Easy for Humans, Hard for AI: Metaphor Abstraction and Reasoning Corpus.”
Baumgaertner is a professor of philosophy and studies cognitive science and the ways belief spreads. In his presentation, he outlined the research behind his new campus-wide Metaphor Abstraction and Reasoning Corpus competition, which is based on the idea that artificial intelligence struggles with metaphor.
“These large language models that undergird AI systems have a kind of super knowledge. They’ve been exposed to more information than any single human,” Baumgaertner said. “But they’re not intelligent. Why?”
Baumgaertner explained that what artificial intelligence does, rather than adapt to novelty or create new solutions, is pull from memorized facts to solve a problem. “The key feature of intelligence, as I noted earlier, is novelty, that [humans] are able to adapt to novel situations … in a very, very efficient way.”
“There’s a view in cognitive science and in philosophy that metaphors aren’t just these literary devices, they’re actually conceptual,” Baumgaertner said. “You’re [thinking] in a way that allows you to now understand something that you wouldn’t have otherwise understood if you were thinking about this literally.”
Baumgaertner said that the instinct is to create harder problems and brand-new, unanswered questions for the AI to solve, with the benchmark being the difficulty of the problems. Instead, he references Francois Chollet’s 2019 book “On the Measure of Intelligence” and strives to create puzzles that are easy for humans, but hard for AI. This is called the abstraction and reasoning corpus.
Each of Baumgaertner’s example problems contain an input image, an output image and a metaphor. The input and output images are simple pixel art. There is some sort of connection between the input and the output, and participants attempt to create an output from a final input. The metaphor serves as an abstract hint toward the solution.
“I should be involving as many people as I possibly can, and that’s what brings us to the MARC project,” Baumgaertner said. “To create this metaphor, abstraction and reasoning corpus, I want to build a benchmark, with the help of the UI community, that we can use in either the AI industry or in cognitive science.” The MARC benchmark could have many different uses, including basic AI literacy, according to Baumgaertner.
Anyone can submit a MARC puzzle on the competition’s website, marc.nkn.uidaho.edu/, by creating a metaphor and example inputs and outputs.
The competition features four different prizes: $500 for each of the top five submissions from philosophy or political science majors; $500 for the best student submission, as judged by the library; $500 for the best faculty or staff submission, as judged by the library; and, Baumgaertner’s favorite, a 3D-printed Bragging Rights trophy for the best submission by senior leadership.
Each of the submissions will be evaluated based on creativity, ease for humans to solve and difficulty for AI to solve. The competition runs from Sept. 1-Nov. 30 at 11:59 p.m., with the results announced Jan. 16, 2026.
MARC is sponsored by the Department of Politics and Philosophy, the Institute for Interdisciplinary Data Science, the Institute for Modeling Collaboration and Innovation, the UI Library and Bert Baumgaertner. Those who want to sponsor a prize should reach out to Baumgaertner at [email protected].
The next Renfrew Colloquium presentation will be hosted by Nels Reese, a professor emeritus of architecture. Reese will outline the 50-year history of the P1FCU Kibbie Dome in “The Kibbie Dome Becomes History” on Tuesday, Sept. 9.
The Renfrew Colloquium began in 2001 and is sponsored by the College of Letters, Arts and Social Sciences and the UI Library. Presentations are hosted weekly on Tuesdays from 12:30-1:30 p.m. on the first floor of the library.
For more information on the Renfrew Colloquium and the semester schedule, visit www.uidaho.edu/letters-arts-social-sciences/news-events/renfrew-colloquium.
Dakota Steffen can be reached at [email protected].