Please select your home edition
OLAS 2020 - Leaderboard

Eyes and ears at sea: US Coast Guard to test Saildrone autonomous MDA capabilities

by Saildrone 19 Jul 15:21 UTC
Saildrone is using a specially built camera system and advanced acoustic technology combined with machine learning to identify activity at sea. © Saildrone

At sea, a human of average height can see about five kilometers (three miles) on a clear day. The distance across the Pacific Ocean from San Francisco to Tokyo is about 8,260 kilometers (5,133 miles)—a seemingly impossible distance for any law enforcement organization to monitor.

Maritime domain awareness (MDA) is the effective understanding of anything associated with the safety and security of the global maritime domain, including illegal fishing, drug enforcement, and limiting intrusion into protected marine sanctuaries.

Congress has tasked the United States Coast Guard (USCG) with examining the feasibility, costs, and benefits of improving maritime domain awareness in the remote Pacific Ocean using a low-cost unmanned surface system. Saildrone unmanned surface vehicles (USVs) have conducted extensive data collection missions around the world, from fisheries missions in the Arctic to bathymetry missions in the Gulf of Mexico, demonstrating their significant potential as a tool for MDA in any area of the ocean. Being completely silent and capable of missions up to 12 months in duration, Sailrone USVs effectively provide eyes and ears at sea, supporting a variety of ISR (intelligence, surveillance, reconnaissance) objectives.

Saildrone has been awarded a $1.1 million contract by the USCG Research and Development Center (RDC) to conduct a 30-day demonstration of ISR/MDA capabilities in the Central Pacific Ocean. The goal of the demonstration is to assess low-cost, commercially available autonomous solutions to improve maritime domain awareness in remote regions. The demonstration will investigate detection ranges, appropriate sensor packages to provide desired outcomes, and the flow of communication between the vehicles and command centers.

Saildrone's MDA solutions consist of three parts. First, the vehicles themselves, designed for long-duration missions at sea. Second, an array of detection sensors including optical cameras, automated identification system (AIS) receivers, and optional radar or infrared cameras for night-time capabilities. Third, the crucial AI/ML software, which fuses the data from all sensors, recognizes targets of interest, and alerts the end-user appropriately.

Saildrone has built an unprecedented proprietary data set of some four million images, representing years of data collection by saildrones at sea. Just as the ImageNet data set was instrumental in the development of ML algorithms for visual object detection on land, this data set is unlocking new capabilities at sea, a challenging environment where all pixels are moving across each frame.

"Machine learning is teaching computers to memorize patterns. In order to memorize a pattern, you need a lot of examples of that pattern. The data set that we have built is one of the crucial ingredients, allowing us to do what we're doing," explained Cory Schillaci, senior machine learning engineer at Saildrone.

Computer vision is based on deep neural networks, also known as artificial neural networks. The patterns within the collected data set are represented by numbers, which in turn are mathematically mapped to define a model that a computer can be trained to recognize. Saildrone achieved this over several years, leveraging the industrial-strength large-scale cloud-based compute infrastructure provided by Amazon Web Services (AWS).

"We are excited to see public sector customers continue to utilize the AWS Cloud to drive innovation and spur solutions that allow for missions to be performed better, faster, and in a more secure manner," said Brett McMillen, general manager of the US Federal Civilian & Ground Station for AWS. "Using AWS, Saildrone developed autonomous maritime domain awareness solutions that leverage machine learning and help support the US Coast Guard's efforts to monitor activity from the surface layer to the deep ocean."

When it comes to machine learning, people often think that the exciting part is in training the model, but in reality, the work is in the data collection and annotations, and efficiently deploying the model on an embedded system—in this case, an autonomous vehicle with a limited supply of solar power.

Typically, neural networks are run using specialized hardware called a graphics processing unit (GPU), which consume hundreds of watts of power. Saildrone's USVs are powered exclusively by solar energy, which is shared between data collection, storage, navigation, and communication operations. "A significant part of the work that we've done is in making our models run in a low-power environment," said Schillaci.

The "eyes" of the Saildrone MDA solution are in a specially built 360 degrees camera system, integrated with a GPU. The cameras capture images on a very high frequency and the ML model scans the images looking for one of the patterns it's been trained to find, such as an illegal fishing vessel off the coast of Hawaii. When a vessel has been identified, the vehicle sends an alert to users in real time. Advanced acoustic technology provides the "ears" for sub-surface maritime domain awareness.

The Saildrone MDA solution integrates machine learning with automatic identification system (AIS) information to not only identify the presence of a vessel but identify that vessel using its Maritime Mobile Service Identity (MMSI) number including registration and country of origin.

This proof-of-concept mission on behalf of the USCG will take place in a 52-square-kilometer (20 sq. mi.) area of the Central Pacific about 30 nautical miles south of Oahu, Hawaii. Three Saildrone SUSVs will operate in a "picket line" formation, essentially creating an invisible fence to detect any passing vessels. The primary objectives of the mission are to demonstrate the operational capability of the Saildrone solution and investigate how a small USV system can be effectively used to improve maritime domain awareness in remote areas of the Pacific Ocean.

Saildrone also offers an enhanced surface MDA solution, using its 22-meter (72-foot) medium unmanned surface vehicle (MUSV). This larger platform offers higher patrol speed and wider detection range, due to significantly higher placement of the sensor array at a height of 15 meters (50 feet). In addition to the optical cameras and AIS receivers, the MUSV also carries radar and infra-red (IR) cameras, offering night-time detection capabilities. It comes packaged with the same high-performance AI/ML onboard detection algorithms and alerting system.

When combined with its advanced acoustics sensing suite, the Saildrone MUSV solution offers persistent eyes and ears above and below the sea surface anywhere in the world, redefining ISR/MDA to combat illegal fishing, secure borders, and protect infrastructure.

Resources: United States Coast Guard, "Coast Guard Awards Contracts for Maritime Domain Awareness Study," press release, February 25, 2020

Related Articles

Saildrones could support US Coast Guard
A fleet of six were launched from Hawaii to demonstrate Maritime domain awareness (MDA) is the effective understanding of anything associated with the safety and security of the global maritime domain, including illegal fishing, drug enforcement, and limiting intrusion into protected areas. Posted on 18 Oct
Saildrone completes first Arctic mapping mission
Four saildrones sailed 8,000nm round trip from San Francisco to the Canadian border Accurate bathymetry of the Arctic seafloor is key to maritime safety, economic development, and sustainability efforts in the region. Posted on 16 Oct
Ecosystem changes put pressure on data collection
Saildrone USVs enabled Alaska Fisheries Science Center to perform the 2020 Alaska pollock survey Saildrone's fleet of unmanned surface vehicles (USVs) is a cost-effective tool to help fisheries managers meet and expand their mission. Posted on 22 Aug
A historic 5,000+ nautical mile voyage for science
Demonstrating how fit-for-purpose technology can be used to increase ocean observation This historic nine-month voyage began in Cabo Verde, off the coast of West Africa, entered the Mediterranean Sea through the Strait of Gibraltar, and finished in Trieste, at the top of the Adriatic Sea. Posted on 22 Jul
Mapping arctic seafloor with autonomous vehicles
Testing ability of Saildrone USVs to map the US exclusive economic zone off Alaska's North Slope It was a treacherous journey—along the west coast of Greenland, weaving through Canada's Arctic islands, and then southwest along Alaska's North Slope and through the Bering Strait—and even in the summer, it was mostly blocked by impenetrable sea ice. Posted on 12 Jun
Saildrone data quality validated
Establishing confidence in Saildrone fit for purpose data Saildrone's global fleet of unmanned surface vehicles (USVs) are designed to sail the world's oceans in the most extreme conditions collecting atmospheric and oceanographic data above and below the sea surface. Posted on 12 May
Saildrone enters Med through Strait of Gibraltar
First unmanned passage reverses the path of the ancient mariners At 16:16 UTC, two Saildrone unmanned surface vehicles (USVs) reversed the path of the ancient mariners entering the Mediterranean Sea through the Pillars of Hercules. Posted on 9 Mar
GJW Direct 2020 FOOTERLizard 2020 - FOOTEROLAS 2020 - Footer