The Florida Museum of Natural History, nestled on the edge of the University of Florida's Gainesville campus, is home to more than 40 million artifacts.
It's a treasure trove of remnants from eras past — and a neverending task in data collection. Manually noting information about the phenotype, or the observable characteristics, of one specimen can be time-intensive. Multiply that by tens of millions of specimens, and you've got yourself a backlog in analysis.
For Arthur Porto, an evolutionary biologist, that bottleneck has been at the forefront of his research ever since he took on a postdoctoral project with 10,000 images of marine invertebrates. He calculated how long it would take him to manually measure their skeletal traits: 250 days.
"At the time, I was like, 'There must be a better way to do this.' And I started exploring," Porto says. He eventually landed on computer vision, developing a program that could use artificial intelligence to identify skeletal structures in images. The algorithm analyzed all 10,000 photos in two hours. "That really took my career in another trajectory ... This was the tool we were missing. This is the tool we need."
In 2023, Porto became the Florida Museum's first curator of artificial intelligence for natural history and biodiversity. He now creates AI models from scratch that help in-house researchers — like himself — extract information from digitized natural history collections. So far, he estimates he has analyzed more than 1.4 million images of specimens with AI tools. He works alongside Nicolas Gauthier, the museum's assistant curator of artificial intelligence for cultural and biological diversity.
Their biggest project of late is a collaboration with UF's NVIDIA AI Technology Center to connect the museum's digital collections to users. A mixture of AI models can "read" images of artifacts and automatically assign them tags according to their traits, such as their color or species name. That way, users can easily sort through large digital collections and narrow down their selection with queries.
Other AI projects are in progress at the Florida Museum. One tries to quantify levels of coloration within a specimen to learn more about prey-predator dynamics. Another leverages generative AI to help fill gaps in incomplete fossil records, reconstructing fragmented artifacts within a level of certainty. In the future, Porto says he'd love to see what data he could scrape from the museum's collection of 16,000-plus 3D scans of vertebrate skeletons.
"There are billions of objects all around the world with so much historic information about things that were alive 100 years ago, 10,000 years ago, 10 million years ago," he says. "We can finally have the tools to get that information from them, which is really exciting to me."