Thresholding

Thresholding is the process of converting an image to a binary image. If a pixel passes the threshold, it turns white (255 for 8 bit images), else, it turns black (0).

A Gayscaled Image

The rest of the page will talk about the various types of thresholding techniques OpenCV provides.

Threshold

The “threshold” algorithm is defined mathematically as

\[\begin{split}dst(x,y) = \begin{cases} maxValue & \text{if } src(x,y) > value \\ 0 & \text{if } src(x,y) <= value \end{cases}\end{split}\]

In layman’s terms, if \(low < pixel value\) , then the pixel passes the test and is turned white, else it is turned black.

Erosion Example
Imgproc.threshold(src, dst, value, maxValue, Imgproc.THRESH_OTSU);
cv::threshold(src, dst, value, maxValue, THRESH_BINARY);
ret,dst = cv2.threshold(src, value, maxValue, cv2.THRESH_BINARY)

While other methods besides THRESH_BINARY exist in OpenCV, there is not a good application to use any of them in FRC.

Note that this is a high pass filter, and nothing more.

inRange

OpenCV’s “inRange” function checks if \(low < pixel value < high\), and if it is, then the pixel passes the test and is turned white, else it is turned black.

Erosion Example

Note that this is identical to threshold’s output because the parameters used made inRange behave the same. InRange is useful when thresholding for certain colors, as it is more than a simply high pass filter.

Core.inRange(src, low, high, dst);
cv::inRange(src, low, high, dst);
ret, dst = cv2.inRange(src, low, high)

Otsu

Otsu thresholding is an old algorithm that is an adaptive thresholding technique. The algorithm assumes that the image contains two classes of pixels following a bi-modal histogram (foreground pixels and background pixels), it then calculates the optimum threshold separating the two classes so that their combined spread is minimal, or equivalently so that their inter-class variance is maximal.

Otsu’s method exhaustively searches for the threshold that minimizes the intra-class variance, defined as a weighted sum of variances of the two classes:

\(\sigma _{w}^{2}(t)=\omega _{0}(t)\sigma _{0}^{2}(t)+\omega _{1}(t)\sigma _{1}^{2}(t)\)

Weights \(\omega _{0}\) and \(\omega _{1}\) are the probabilities of the two classes separated by a threshold \(t\) and \({\displaystyle \sigma _{0}^{2}}\) and \({\displaystyle \sigma _{1}^{2}}\) are variances of these two classes.

Otsu Demo
Imgproc.threshold(src, dst, 0, 255, Imgproc.THRESH_OTSU);
cv::threshold(src, dst, 0, 255, CV_THRESH_BINARY | CV_THRESH_OTSU);
ret2, dst = cv2.threshold(src ,0 , 255, cv2.THRESH_OTSU)

Otsu thresholding optimizes the upper and lower bounds, so 0 and 255 are simply placeholders as OpenCV doesn’t use a separate function for Otsu thresholding.

Otsu Example

In a typical FRC game, your environment is not drastically changing, so it is best practice to use inRange with hand tuned values instead of Otsu for speed purposes.

Thresholding with Color Images

Up until now, the examples have been with grayscale images. Color images are different in the fact that they have 3 channels instead of one, meaning that threshold values must be provided for each channel (color). This is a very slow and tedious process. To make it easier, use this program: https://github.com/rr1706/Multi-Thresh. This utilizes sliders that dynamically changes the threshold values for each color, and also allows the user to tune HSV images as well. Always use inRange when thresholding RGB images

The syntax for each language changes slightly, as observed:

Core.inRange(src, new Scalar(low1, low2, low3), new Scalar(high1, high2, high3), dst);
cv::inRange(src, Scalar(low1, low2, low3), Scalar(high1, high2, high3), dst);
dst = cv2.inRange(src, np.array([low1, low2, low3]), np.array([high1, high2, high3]))

Using HSV Thresholding

HSV thresholding uses hue, saturation, and value to threshold images. Unlike RGB, HSV separates the image intensity, from the color information. This is very useful in computer applications such as vision tracking. In FRC, HSV is a great tool to detect the reflective vision tape if using a LEDs to illuminate the tape. It is also possible to use an IR camera with IR LEDs which would output a grayscale image. However, there are other threshold options that seperate image intensity with color but HSV is often used simply because the code for converting between RGB and HSV is widely available and easily implemented. Before using HSV to threshold the image, you must convert the image retrieved from the camera to the HSV colorspace (see code below).

Imgproc.cvtColor(src, dst, Imgproc.COLOR_BGR2HSV);
cv::cvtColor(src, dst, CV_BGR2HSV);
dst = cv2.cvtColor(src, cv2.COLOR_BGR2HSV)

Using Trackbars/Sliders for Real Time Tuning

As said above, sliders allow you to dynamically change HSV values, allowing you to fine tune the correct threshold values in real time. Here’s how to create them. Note: Java does not support OpenCV to handle GUI so trackbars must be done with Swing and jsliders. More info on that here.

cv::namedWindow("Title of Window");
cv::createTrackbar("Title of slider", "Title of Window", &variable, highest number);
cv2.namedWindow('Title of Window')
cv2.createTrackbar('Title of Slider’, ‘Title of Window”, 0, 255, nothing)
var = cv2.getTrackbarPos(‘title of slider’, ‘title of window’);

Let’s tackle an example. This is a pretty standard image that one might have if using green LEDs for the 2017 game.

Boiler Raw

The goal is to make the boiler tape white (255), and everything else black (0). By using the Multi-Thresh program, the RGB min and max values were found to be (0, 90, 0), (46, 255, 255), and they produce the following image:

Boiler Thresholded

If you find that you have noise, which is stray pixels, or if you thresholded away part of the inside of your target, please check out the morphological operations page.