Morphological Operations

Binary images may contain noise (pixels that passed the initial threshold test but are not desired). Morphological image processing attempts to remove the noise of images while accounting for the form and structure of the image, as well as preserving the desired pixels’ integrity. This guide dives into the math behind morphological operations, and then explains how they can be used in FRC.

A Binay Image

Morphological image processing is a collection of non-linear operations related to the shape or morphology of features in an image. Morphological operations rely only on the relative ordering of pixel values and not on their numerical values, therefore making them especially suited to process binary images.

The core idea in binary morphology is to probe an image with a simple, pre-defined shape, and then draw conclusions on how this shape fits or misses the shapes in the image. This simple “probe” is called kernel, and is itself a binary image (i.e., a subset of the space or grid), The kernel dimensions are not limited to 3x3, but OpenCV limits them to be \(N\times N\) where \(N \in \mathbb{Z}_{odd}\), so here we will make the same restrictions.

Here are two common kernels used:

\(\left( \begin{array}{ccc} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{array} \right)\) \(\left( \begin{array}{ccc} 0 & 1 & 0 \\ 1 & 1 & 1 \\ 0 & 1 & 0 \end{array} \right)\)

Mat kernel = Imgproc.getStructuringElement(Imgproc.RECT, new Size(3,3));
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_CROSS, new Size(3,3));
Mat kernel = cv::getStructuringElement(MORPH_RECT, Size(3,3) Point(-1,-1);
Mat kernel = cv::getStructuringElement(MORPH_CROSS, Size(3,3) Point(-1,-1);
kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))
kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))

Defintion: Let \(E\) be a Euclidean space \(\mathbb{R}^d\) or an integer grid \(\mathbb{Z}^d\) for some dimension \(d\), and \(A\) be a a binary image where \(A \in E\).

Erosion

The erosion of the binary image \(A\) by the kernel \(B\) is defined by:

\(A\ominus B=\bigcap _{{b\in B}}A_{{-b}}.\)

Erosion Example
Imgproc.erode(src, dst, kernel);
cv::erode(src, dst, kernel, Point(-1,-1), 1);
erosion = cv2.erode(src, kernel, iterations=1)

Dilation

The dilation of \(A\) by the structuring element \(B\) is defined by:

\(A\oplus B=\bigcup _{{b\in B}}A_{b}.\)

Dilation Example
Imgproc.dilate(src, dst, kernel);
cv::dilate(src, dst, kernel, Point(-1,-1), 1);
dilation = cv2.dilate(src, kernel, iterations=1)

Properties of Morphologpical Operations

  • They are translation invariant.
  • Dilation is associative.
  • Dilation is distributive over set union.
  • Erosion is distributive over set intersection.
  • Dilation is a pseudo-inverse of the erosion, and vice versa.

While understanding the set theory axioms of erosion and dilation is not necessary to understand them, they are still interesting.

Morophological Operations Playground

Open

Open is another name of erosion followed by dilation. It is useful in removing noise.

Open Example
Imgproc.morphologyEx(mFGMask,mFGMask,Imgproc.MORPH_OPEN,kernel);
cv::morphologyEx(src, dst, MORPH_OPEN, kernel, Point(-1,-1), 1);
opening = cv2.morphologyEx(src, cv2.MORPH_OPEN, kernel)

Close

Closing is reverse of Opening, Dilation followed by Erosion. It is useful in closing small holes inside the foreground objects.

Close Example
Imgproc.morphologyEx(mFGMask,mFGMask,Imgproc.MORPH_CLOSE,kernel);
cv::morphologyEx(src, dst, MORPH_CLOSE, kernel, Point(-1,-1), 1);
closing = cv2.morphologyEx(src, cv2.MORPH_CLOSE, kernel)

Morphological Gradient

It is the difference between dilation and erosion of an image. The result will look like the outline of the object.

Gradient Example
Imgproc.morphologyEx(mFGMask,mFGMask,Imgproc.MORPH_GRADIENT,kernel);
cv::morphologyEx(src, dst, MORPH_GRADIENT, kernel, Point(-1,-1), 1);
opening = cv2.morphologyEx(src, cv2.MORPH_GRADIENT, kernel)

Uses in FRC

FRC provides less than ideal environments for computer vision. Often times there is noise in your images that cannot be overcome by reducing the exposure of your camera and thresholding. When this occurs, consider using a morphological operation.

In many cases, however, these standard kernels will not suffice for FRC. Either erosion will remove too much of the target(s) or dilate will combine targets.

The following kernel will not alter the width of a pixel cluster, only the height. This is useful for when your targets are close together horizontally or if the target is not very tall and you must erode.

\(\left( \begin{array}{ccc} 0 & 0 & 0 \\ 1 & 1 & 1 \\ 0 & 0 & 0 \end{array} \right)\)

Gradient Example

What is not pictured is the border on the top and left side of the image. this is a consequence of this kernel. Despite this consequence, this kernel may be very useful in many cases.

Likewise, this kernel will not alter the height of the pixel cluster, only the width, this is useful with targets who are close together vertically or if the target is not vey tall and you must erode.

\(\left( \begin{array}{ccc} 0 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 0 \end{array} \right)\)

Gradient Example