Eyes and ears at sea: US Coast Guard to test Saildrone autonomous MDA capabilities
by Saildrone 19 Jul 2020 15:21 UTC
Saildrone is using a specially built camera system and advanced acoustic technology combined with machine learning to identify activity at sea © Saildrone
At sea, a human of average height can see about five kilometers (three miles) on a clear day. The distance across the Pacific Ocean from San Francisco to Tokyo is about 8,260 kilometers (5,133 miles)—a seemingly impossible distance for any law enforcement organization to monitor.
Maritime domain awareness (MDA) is the effective understanding of anything associated with the safety and security of the global maritime domain, including illegal fishing, drug enforcement, and limiting intrusion into protected marine sanctuaries.
Congress has tasked the United States Coast Guard (USCG) with examining the feasibility, costs, and benefits of improving maritime domain awareness in the remote Pacific Ocean using a low-cost unmanned surface system. Saildrone unmanned surface vehicles (USVs) have conducted extensive data collection missions around the world, from fisheries missions in the Arctic to bathymetry missions in the Gulf of Mexico, demonstrating their significant potential as a tool for MDA in any area of the ocean. Being completely silent and capable of missions up to 12 months in duration, Sailrone USVs effectively provide eyes and ears at sea, supporting a variety of ISR (intelligence, surveillance, reconnaissance) objectives.
Saildrone has been awarded a $1.1 million contract by the USCG Research and Development Center (RDC) to conduct a 30-day demonstration of ISR/MDA capabilities in the Central Pacific Ocean. The goal of the demonstration is to assess low-cost, commercially available autonomous solutions to improve maritime domain awareness in remote regions. The demonstration will investigate detection ranges, appropriate sensor packages to provide desired outcomes, and the flow of communication between the vehicles and command centers.
Saildrone's MDA solutions consist of three parts. First, the vehicles themselves, designed for long-duration missions at sea. Second, an array of detection sensors including optical cameras, automated identification system (AIS) receivers, and optional radar or infrared cameras for night-time capabilities. Third, the crucial AI/ML software, which fuses the data from all sensors, recognizes targets of interest, and alerts the end-user appropriately.
Saildrone has built an unprecedented proprietary data set of some four million images, representing years of data collection by saildrones at sea. Just as the ImageNet data set was instrumental in the development of ML algorithms for visual object detection on land, this data set is unlocking new capabilities at sea, a challenging environment where all pixels are moving across each frame.
"Machine learning is teaching computers to memorize patterns. In order to memorize a pattern, you need a lot of examples of that pattern. The data set that we have built is one of the crucial ingredients, allowing us to do what we're doing," explained Cory Schillaci, senior machine learning engineer at Saildrone.
Computer vision is based on deep neural networks, also known as artificial neural networks. The patterns within the collected data set are represented by numbers, which in turn are mathematically mapped to define a model that a computer can be trained to recognize. Saildrone achieved this over several years, leveraging the industrial-strength large-scale cloud-based compute infrastructure provided by Amazon Web Services (AWS).
"We are excited to see public sector customers continue to utilize the AWS Cloud to drive innovation and spur solutions that allow for missions to be performed better, faster, and in a more secure manner," said Brett McMillen, general manager of the US Federal Civilian & Ground Station for AWS. "Using AWS, Saildrone developed autonomous maritime domain awareness solutions that leverage machine learning and help support the US Coast Guard's efforts to monitor activity from the surface layer to the deep ocean."
When it comes to machine learning, people often think that the exciting part is in training the model, but in reality, the work is in the data collection and annotations, and efficiently deploying the model on an embedded system—in this case, an autonomous vehicle with a limited supply of solar power.
Typically, neural networks are run using specialized hardware called a graphics processing unit (GPU), which consume hundreds of watts of power. Saildrone's USVs are powered exclusively by solar energy, which is shared between data collection, storage, navigation, and communication operations. "A significant part of the work that we've done is in making our models run in a low-power environment," said Schillaci.
The "eyes" of the Saildrone MDA solution are in a specially built 360 degrees camera system, integrated with a GPU. The cameras capture images on a very high frequency and the ML model scans the images looking for one of the patterns it's been trained to find, such as an illegal fishing vessel off the coast of Hawaii. When a vessel has been identified, the vehicle sends an alert to users in real time. Advanced acoustic technology provides the "ears" for sub-surface maritime domain awareness.
The Saildrone MDA solution integrates machine learning with automatic identification system (AIS) information to not only identify the presence of a vessel but identify that vessel using its Maritime Mobile Service Identity (MMSI) number including registration and country of origin.
This proof-of-concept mission on behalf of the USCG will take place in a 52-square-kilometer (20 sq. mi.) area of the Central Pacific about 30 nautical miles south of Oahu, Hawaii. Three Saildrone SUSVs will operate in a "picket line" formation, essentially creating an invisible fence to detect any passing vessels. The primary objectives of the mission are to demonstrate the operational capability of the Saildrone solution and investigate how a small USV system can be effectively used to improve maritime domain awareness in remote areas of the Pacific Ocean.
Saildrone also offers an enhanced surface MDA solution, using its 22-meter (72-foot) medium unmanned surface vehicle (MUSV). This larger platform offers higher patrol speed and wider detection range, due to significantly higher placement of the sensor array at a height of 15 meters (50 feet). In addition to the optical cameras and AIS receivers, the MUSV also carries radar and infra-red (IR) cameras, offering night-time detection capabilities. It comes packaged with the same high-performance AI/ML onboard detection algorithms and alerting system.
When combined with its advanced acoustics sensing suite, the Saildrone MUSV solution offers persistent eyes and ears above and below the sea surface anywhere in the world, redefining ISR/MDA to combat illegal fishing, secure borders, and protect infrastructure.
Resources: United States Coast Guard, "Coast Guard Awards Contracts for Maritime Domain Awareness Study," press release, February 25, 2020