Side effects in Neural Net particle detection

Side effects on border regions

Problem description

With the simulation interface and repetitive filter images, I detect the particles using a self-made neural network. If my understanding is correct, the image is split in 1000x1000px squares (and residual images on borders, found in the "tile" directory) and sent this way to the neural network.

The fact that the images are not always identically sized causes side effect for the particle detection and has an impact on Detection Accuracy. On the images bellow, we can observe that the detection changes on the sides of the preview area. In red, we can observe that the same side effect appears when detecting on the whole filter.

Area of 1000 px

Same region, area of 1500 px

Same region, area of 2000 px

This behavior is not always reproductible with a real usage when screenshots are made, in my case with the Nicolet iN10 micro-FTIR, as image resolution will change.

Main concern

Maintaining aspect ratio is important in image detection and changing the image input method in GEPARD might make the development of an efficient neural network easier, if the size is previously known (and fixed) before annotation and training of a new specific neural net. If the preview area is set to a similar size as the images used for training, we can observe that the predictions are more accurate.

Adapted size for preview area

Quick fix

A quick fix can be done by setting a different value for the variable tileSize in the nodes.py file. This fix allows to minimise the tile size effect by setting most tiles to a desired value. However, the tiles will automatically be squares, leading to differently shaped borders as the captured images can be rectangles. Reducing the size of the tiles lead to a significant increase in computing time.

We can first see here that the images sizes will vary when reaching the borders

But by finding a right number, we can minimize this difference:

And obtain reasonably good results:

Of course adapting the particle min and max sizes is still needed depending on the sample. We can also observe here that the black border, also sent to the neural network, cause the detection to be different on the sides.

Possible improvements

A first "simple" solution might be to consider using the tiles captured by GEPARD as input for the neural network. Depending on the instrument, the image size would be fixed and used for training, and the black surrounding areas ignored.

A second more complexe solution would be to use a solution similar to the one in this small article about slicing for inference. This would be more ressource consuming but could help detecting complexe pattern of small objects (but this might be overkill).

Note: To train my neural network, I used 752x480 tiles, the same size as the image used to create the simulated filter image. This seems an ideal size as it is in most cases not too long to annotate and was a size used in the software.