By Chris Kornelis
April 1, 2019 5:46 p.m. ET
This article is originally from The Wall Street Journal, April 1st, Business/Journal Reports: Technology.
SEATTLE—Sitting behind his desk at the University of Washington’s electrical and computer engineering department, Jenq-Neng Hwang watches a video of a fish flopping as it’s pulled out of the water on a commercial fishing boat.
It’s probably a halibut. He’s not exactly sure. His job isn’t to know the species of every fish caught on such boats. It’s to create machines that know—computers that use machine learning to instantly distinguish between hundreds of species, count and measure the length of each fish and record where it was hauled in—all while the fish are very much not holding still.
“If all this information can be estimated in real time on thousands of fishing boats, then we are doing real-time monitoring of the whole ocean,” says Prof. Hwang, whose work is part of the Electronic Monitoring Innovation Project through the National Oceanic and Atmospheric Administration’s Alaska Fisheries Science Center.
Scientists, engineers and fishermen and women around the world are deploying machine learning and computer vision to better understand the health of global fish stocks and attempt to preserve their future productivity. Cameras, satellite imagery and computers guided by artificial intelligence all have roles not just in identifying and measuring daily catches, but in tracking possible illegal fishing as well.
Peter Mous, director of the Fisheries Conservation Program at the Nature Conservancy in Indonesia, works with local fishermen to try to understand whether, or to what degree, fish stocks in the region are overexploited. For the moment, most of this work is still done manually. Technicians for the conservancy in Indonesia examine digital photographs of fish, mostly snappers, taken by the crews of about 300 boats, or 3% of the roughly 10,000 fishing boats in Indonesia.
Snappers are some of the most sought-after fish in these waters. If most of the snappers in the photos are fully grown, Mr. Mous says, that suggests the snapper stock is in good condition. If the boats are catching small fish, he says, it generally means the stock is overfished. In the enormous area that Mr. Mous is studying, covering three time zones, he says there’s a mix of stocks. Some are healthy, but others consist primarily of juvenile fish—suggesting overfishing in those areas.
Within five years, Mr. Mous wants to be working with 5% of the fishing fleet. But it will be difficult and not very cost-efficient, he says, to keep examining the increasing load of photos manually. In addition to the volume, even professionals have trouble keeping track of all the different species present in Indonesian waters, which Mr. Mous calls, “in terms of marine life, the global center of biodiversity.”
To be able to step up its monitoring capacity, the conservancy has enlisted a Swedish company, Refind Technologies, to help it develop an automated system, called FishFace, that uses machine learning and computer vision to identify and measure fish and count catches.
The programmers of FishFace, which is currently deployed on one boat, don’t tell it what characteristics to look for to identify a fish. Instead they show it thousands of pictures of fish whose species are identified, then let the program figure out what characteristics it will use to differentiate each species.
The goal, Mr. Mous says, is to be able to present data to Indonesia’s Ministry of Maritime Affairs and Fisheries in support of setting some limits on catch sizes, limiting numbers of boats or perhaps shortening fishing seasons. In Indonesia, he says, aside from a few protected areas, there are no limits on what fishing boats can catch.
In time, Refind co-founder Johanna Reimers says, the data collected by projects like FishFace could be used to certify that fish bought at a supermarket in, say, Montana, was legally caught halfway around the world.
In addition to counting fish, machine learning can help monitor boats. Global Fishing Watch is an international nonprofit that seeks to monitor, expose and prevent illegal commercial fishing and raise awareness of unmanaged overfishing. Offshore from countries like Namibia, for example, with some 900 miles of coastline and nowhere near the monitoring resources of richer countries, it’s easy for unlicensed ships to come in, scoop up fish and disappear.
Using machine-learning programs that analyze data from sources including radar and GPS, the group creates a map that shows all vessels over 49 feet in length that it can find. It also shares information privately with governments in places like Peru, Indonesia and Panama to help them use their limited resources to go after possible illegal fishing.
The nonprofit’s computers can analyze the movements of boats and the equipment they use to determine what they’re fishing for, and whether they’re licensed to do so.
“You can make a pretty good guess about what kind of species they’re targeting based on what species is likely to be there and can be caught with that kind of gear,” says Paul Woods, chief technology officer.
Mr. Woods says that within 10 years, as his group gets more data—including from governments about their countries’ fisheries—Global Fishing Watch hopes to be able to detect “everything that’s extracting anything from the ocean and make that generally transparent and publicly visible.”
Increasing transparency, he says, might entice action by governments that aren’t inclined to admit to a problem with illegal fishing.
Improving the counting and examining of commercially caught fish in real time will be a big help in tracking the health of global fish stocks. That data may also yield insights possibly resulting from the effects of global warming. As the climate changes and fish disappear from one area and reappear in another, for example, data gathered by monitoring could be used to help direct fishermen to areas where they are likely to catch more fish.
Farron Wallace, a fisheries biologist in the North Pacific Observer Program at NOAA’s Alaska Fisheries Science Center, is leading AFSC’s Electronic Monitoring Innovation Project, working with Prof. Hwang at the University of Washington, to use computer vision and machine learning to eventually keep track of stocks in real time.
The planet is getting hotter every year, Mr. Wallace says. “We need to have much better information…to really do a good job of trying to understand how climate change is going to affect those resources out there.”
Mr. Wallace and Mr. Mous agree that global warming isn’t expected to negatively affect populations of all species of fish.
“The whole global warming is very much a wild card,” says Mr. Mous. “You know that things will change in fisheries, but you don’t know exactly how. For that reason, it is important also to keep a finger on the pulse.”
Mr. Kornelis is a writer in Seattle. He can be reached at reports@wsj.com.