Please select your home edition
Upffront 2020 Foredeck Club SW LEADERBOARD

Computers now "see" animals on the ocean bottom

by NOAA Fisheries 20 Sep 15:09 UTC
Researchers at the Northeast Fisheries Science Center are using a machine learning software to teach computers how to identify the sea life in millions of images of the ocean floor. © NOAA Fisheries

The Northeast Fisheries Science Center's annual sea scallop research survey uses a towed sampling device called the HabCam. It collects approximately five million images of the ocean bottom off the Northeast United States. Scientists and volunteers then manually examine an astonishing 100,000 of these images, roughly two percent of the number gathered. They focus on identifying just four targets: sea scallops, fish, crabs, and whelks.

So, a wealth of data is going uncollected owing to the sheer volume available and just how labor-intensive pulling it out of images can be.

Researchers have turned toward finding ways for machines to help identify sea life in these images, faster and more efficiently than humans can. This would improve population data for sea scallops. By more thoroughly examining each image, all kinds of information about other sea life and their habitats can also be captured.

Enter Dvora Hart, an operations research analyst at the Northeast Fisheries Science Center. Hart has been at the forefront of estimating sea scallop populations numbers from images taken by devices like the HabCam. The upcoming 2020 sea scallop assessment will once again use improved population data collected from images.

"HabCam gives us photos of animals in their natural environment without disturbing them; however, much of the information in the images is not collected because human annotators can only mark a small percentage of the available images," said Hart. "Automated annotators can mark all the images and, given proper training, can identify a multitude of different targets—not just sea scallops, fish, crabs and whelks."

Hart is part of an interdisciplinary team that developed the world's first advanced automated image analysis software for the marine environment. The Video and Image Analytics for the Marine Environment, VIAME for short, uses convolutional neural networks—a recent advance in artificial intelligence. These networks teach computers to recognize species and features of their habitats in the images taken by the HabCam.

The work is so significant that Hart and her team won a 2019 Department of Commerce gold medal for their work. This recognition is the highest honor award offered to department employees.

How to train your machine

A computer can "see" an image, but it must be trained to identify what it is "looking" at. To train VIAME to read the HabCam images, human analysts draw a box around the sea life they find in each image. They then identify what's in the box (an "object" to VIAME) with a label. The team uploads these labeled, or "annotated," images, and VIAME is trained by showing it what each image represents.

Initially VIAME gets a training set of about 40,000 annotated images. It takes the program between 1 and 7 days to "learn" to recognize the objects in the images, then it can identify these objects in new images.

This year, Hart brought in reinforcements to help annotate the training images. Ayinde Best and Hayden Stuart, students participating in this year's Partnership in Education Program, created a new set of more than 4,000 densely annotated images to enrich VIAME's learning.

They focused on objects that have not been priority targets, such as sea stars, bryozoans, anemones, hermit crabs, moon snails, and skate egg cases. This work is a start on a new collection of data on important aspects of the marine environment. For example, it will look at large populations of the sea star Astropecten americanus, a predator of young sea scallops.

Smoothing "Fuzzy" data

In population assessments for fish, data underlying the analyses are primarily collected directly from the fish that are caught and examined. Scientists have a strong basis for assuming that the sample is 100 percent reliable. Images annotated by humans are also highly accurate as samples go, but few in comparison to the large number of images available.

VIAME produces samples that are more "fuzzy" than those produced from the images by human analysts. Many marine organisms, flounders and squid for example, can blend into their surroundings by adjusting their appearance to match it. Sea scallops can be covered with organisms that live on their shells, providing a convenient disguise. Several objects in an image can be located near one another; the water may be cloudy; the bottom may have a lot of rocks. There are any number of factors that can obscure the objects. This can result in VIAME categorizing a boxed area as empty when it isn't.

This raises an interesting question: How do you create a population estimate from a group of imprecise or "fuzzy" variables? Answer: statistics to the rescue.

Led by Jui Han Chang, Hart and other colleagues compared the data collected from the same images by human analysts and by VIAME. Then they analyzed four different ways of calculating an estimate of objects in an image, combining data collected by human analysts and by VIAME. The results were used to get a corrected estimate of the abundance of organisms. That estimate is then applied to all of the collected images across a range of habitats and environmental conditions.

"Automated image analysis has the potential to reduce labor costs, since automated annotators can reduce requirements for manual annotators," says Hart. "But it also can do it faster—several images per second compared to 1-2 per minute for manual annotators—and annotate many more images for more potential targets, not only scallops and fish, but many other organisms on the seafloor that are not usually surveyed. This will improve our understanding of the seafloor communities and help guide ecosystem-based fisheries management."

Related Articles

Celebrate seafood month in the Pacific Islands
Join the conversation about sustainable seafood and share your seafood stories throughout the month In the Pacific Islands region, our connection to seafood is deeply rooted in our diverse island cultures. Seafood continues to be a regular part of our lives, and it will remain an important part of our food-secure future. Posted on 17 Oct
Nine things you didn't know about aquaculture
U.S. aquaculture supplies sustainable seafood, provides jobs and enhances ecosystems Many aquaculture producers in the United States don't raise fish, despite the industry's popular image of fish farming. In fact, oysters were the most commercially valuable domestic farmed marine species in recent years. Posted on 11 Oct
Sports medicine leads to discovery about mussels
Feeding rates of blue mussels slow down under ocean acidification conditions Shannon Meseck, a NOAA Fisheries research chemist and marathon runner, was initially interested in how ultra-runners can tolerate higher levels of carbon dioxide than non-athletes. Posted on 6 Oct
The impacts of ghost nets on coral reefs
Researchers use structure from motion photogrammetry to measure the damage Ghost nets are silently drifting through the Northwestern Hawaiian Islands, snagging on coral reefs and entangling wildlife. Posted on 5 Oct
Saving the most endangered marine animals
Watch and learn about the progress the Species in the Spotlight initiative has made in 5 years The Species in the Spotlight initiative focuses on the immediate actions we can take to stabilize endangered marine life populations that are most at-risk of extinction. Posted on 4 Oct
Rare Albino fur seal pup stands out
This pale pup contrasts with its dark-furred companions The only information about how rare this sighting is comes from literature published in 1958. At the time, Scientist Vic Scheffer reported that the chances of encountering a fur seal with this coloration is one in 100,000. Posted on 27 Sep
Killer whale predation on bowhead whales
A new study provides essential information to conserve an endangered species For the first time, scientists have direct evidence that killer whales are preying on bowhead whales in the U.S. Pacific Arctic. A dramatic loss of sea ice in recent years may be leaving bowheads more vulnerable to killer whale predation. Posted on 19 Sep
Loggerhead turtles record a passing hurricane
Changes in dive behavior and movement patterns as storm passes In early June 2011, NOAA Fisheries researchers and colleagues placed satellite tags on 26 loggerhead sea turtles in the Mid-Atlantic Bight. The tagging was part of ongoing studies of loggerhead movements and behavior. Posted on 9 Sep
A disentanglement tale
Saving a humpback whale, removing nearly 4,000 pounds of fishing gear On Monday, July 27 trained responders began a 4-day effort to disentangle a humpback whale from submerged fishing gear near New York City. Multiple partners brought expertise and large boats to haul the heavy gear up and free the whale. Posted on 8 Sep
Give pregnant killer whales space to forage
Washington regulations require boaters to stay 300 yards from killer whales Washington regulations require boaters to stay 300 yards from Southern Resident killer whales, 400 yards in front and behind. Posted on 23 Aug
Grapefruit Graphics 2019 - FooterVaikobi 2019AUG - Footer 3North Sails 2019 - NSVictoryList - Footer