Although AI algorithms have achieved quite some progress in plankton image recognition, most of them can only deal with limited number of known classes. In reality, however, the ocean scientists will definitely meet with new (unknown or unseen) plankton species, where the existing algorithms are very likely to fail.

A research team led by Dr. LI Jianping from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences and their collaborators from Xiamen University reported an anomaly detection pipeline and its execution methods trying that may help to better recognize plankton.  

The paper entitled "Anomaly Detection for In Situ Marine Plankton Images" was published at the International Conference on Computer Vision 2021.  

"Basically, the pipeline needs training before it can be used, which includes two phases, the pre-training phase and the post-training phase," said Dr. LI.

In the pre-training phase, the researchers proposed a data augmentation technique to generate auxiliary datasets so that the AI model is able to acquire abundant imaginary knowledge of the unseen classes. Additionally, they put forward a new CKA loss function to better detect the new classes and classify the existing classes simultaneously.

In the post-training phase, a detector is further trained to discriminate an input image as known or unknown class based on its features extracted by the pre-trained model. If the image is determined as the known class, the detector will further classify it into one of the known classes.  

After training, the AI pipeline can better sort a future image into one of the known classes or unknown class. The proposed open-set classification methods are expected to help marine biologists to better identify their observation targets of interest, so that the in situ monitoring of marine plankton can become more convenient and efficient.

The training has AI classified known plankton, unseen plankton and particles and bubbles. (Image by LI Jianping)