The post Feature Points Extraction using Harris Corner Detector appeared first on FIVEKO.

]]>Some common applications for extracting feature points from images through Harris corner detector are:

- Image classification
- Objects recognition
- Objects tracking in video sequence
- Image alignment and stitching for panoramic image creation

Corners are points located between two dominant image edges with different direction. Vertices of image contours are of great importance for defining objects shape. Hence we frequently call them *key points* or *feature points*.

Feature points extraction by Harris detector is based on image derivatives. Because corners are located around derivatives variations, we look for such significant gradient changes. The principle behind Harris corner detector is the search for significant changes in the gradient intensity for local area (window). For that reason when the window is over a linear gradient segment directional changes will decrease maximum gradient value. Otherwise when window is over a vertex, movements in any direction will increase the intensity change.

A movable image patch at position (*x, y*) shifts with defined offset (*dx,dy*):

The Harris corner score is used to determine if the current position contains a corner or not:

Below are code snippets of Harris Corner Detection, which are part of FivekoGFX computer vision library. The proposed algorithm uses OpenGL shaders to provide parallel execution and good performance.

First of all we calculate image gradients in horizontal and vertical direction. As a result our algorithm stores the derivatives as in below snippet.

```
float dx = length((GET_PIXEL(-1, -1)*u_kernel[0] +
GET_PIXEL(-1, 0)*u_kernel[1] +
GET_PIXEL(-1, +1)*u_kernel[2]) -
(GET_PIXEL(+1, -1)*u_kernel[0] +
GET_PIXEL(+1, 0)*u_kernel[1] +
GET_PIXEL(+1, +1)*u_kernel[2]));
float dy = length((GET_PIXEL(-1, -1)*u_kernel[0] +
GET_PIXEL(0, -1)*u_kernel[1] +
GET_PIXEL(+1, -1)*u_kernel[2]) -
(GET_PIXEL(-1, +1)*u_kernel[0] +
GET_PIXEL(0, +1)*u_kernel[1] +
GET_PIXEL(+1, +1)*u_kernel[2]));
gl_FragColor = vec4(dx*dx, dy*dy, dx*dy, 1.0);
```

Next step is to compute Harris corner score at each image pixel. Using accelerated GPU computation, *FivekoGFX* performs local **3×3 average** to get the sums of the derivatives products. The matrix determinant and trace calculates using point parameters from above step.

```
#define DET(_p) ((_p).x*(_p).y - (_p).z*(_p).z)
#define TRACE(_p) ((_p).x + (_p).y)
vec4 p = (GET_PIXEL(0, 0) + GET_PIXEL(-1, -1) +
GET_PIXEL(-1, 0) + GET_PIXEL(-1, 1) +
GET_PIXEL( 0, 1) + GET_PIXEL( 1, 1) +
GET_PIXEL( 1, 0) + GET_PIXEL( 1, -1) +
GET_PIXEL( 0, -1)) / 9.0;
float k = 0.04;
float R = DET(p) - (k * (TRACE(p)*TRACE(p)));
```

Like **Canny edge detector**, this algorithm also implements a NMS (non-maximum suppression) filter. As a result, places that are less likely to be points of interest are set to zero. Feel free to visit FivekoGFX project on GitHub for the detailed non-maximum suppression source code.

Using FivekoGFX library it is possible to extract Harris corners map with a simple API call. Below is an example code snippet for feature points map extraction.

```
var fivekogfx = new FivekoGFX();
// Load the initial image from e.g. a canvas source
fivekogfx.load(canvas);
// Perform Harris Corner Detector
fivekogfx.harrisCorners();
// Extract peaks using non-maximum suppression with specific size
fivekogfx.nms(size);
// Draw the result points map
fivekogfx.draw(canvas);
```

Elapsed:

Visit FivekoGFX on GitHub for more image processing examples and detailed source code!

The post Feature Points Extraction using Harris Corner Detector appeared first on FIVEKO.

]]>The post Color space conversion appeared first on FIVEKO.

]]>A common approach for *RGB* **->** *Gray* color-space conversion is to use an average value from all the three **Red**, **Green**, **Blue** color components. I believe it is good to mention that the human eye has different sensitivity to each color component. For gray-scale conversion *FivekoGFX* uses the Y component from *YCbCr* color space.

The *RGB* **->** *YCbCr* is a widespread color conversion in many computer vision and photography tasks. The **YCbCr** color transformation applies to many use cases such as JPEG compression, skin segmentation and many more. The math used in *FivekoGFX* is as this below:

The input values should be in [0 – 255] (meaning 0.0-1.0 in case of GPU GLSL shaders).

The output values are [16 – 235] for Y and [16 – 240] for Cb and Cr

The *FivekoGFX* implements the following backward **YCbCr** transformation to get back the **RGB** color components:

The result color components are in range [0 – 255].

Our graphics library supports *RGB to HSL* colors pace transformation, which is widely used in many image processing tasks. This conversion is appropriate for many color tracking algorithms since it closely align to human visual system. The HSL acronym stands for the three components: **Hue**, **Saturation** and **Luminance**.

The FivekoGFX implements the RGB to HSL conversion as follows:

For *Luminance*:

For *Hue*:

For *Saturation*:

FivekoGFX implements multiple algorithms for *JavaScript color conversions* with the help of WebGL and **OpenGL GLSL shaders**. As a result, this makes such conversions pretty fast, since all math is done over the **GPU.**

```
/*
* Example code for FivekoGFX color transforms
**/
var fivekogfx = new FivekoGFX();
// Load source image from Canvas object
fivekogfx.load(canvas);
// Convert RGB to GRAYSCALE color space and draw back to canvas
fivekogfx.rgb2gray();
fivekogfx.draw(canvas);
// Convert RGB to YCbCr color space and draw back to canvas
fivekogfx.rgb2ycbcr();
fivekogfx.draw(canvas);
// Convert RGB to HSL color space and draw back to canvas
fivekogfx.rgb2hsl();
fivekogfx.draw(canvas);
```

The post Color space conversion appeared first on FIVEKO.

]]>The post Watershed Image Segmentation appeared first on FIVEKO.

]]>Our algorithm is based on **Meyer’s** *flooding *introduced by F. Meyer in the early 90’s. Originally the algorithm works on a **grayscale image**. A *flooding process* is performed over a **gradient image**, so the basins should emerge along the objects **edges**. The process describes as following:

- Initialize object groups with previouseely selected seeds markers.
- This step extracts the neighboring pixels of each group and moves them into a
**priority queue**. Originally the priority level corresponds to pixel gradient value, but it is possible to use different*weighting measure*. - The lowest
*priority*pixels are extracted from the queue and processed first. If all visited neighbors of extracted pixel have the same label the pixel is marked with their same label. All non-visited pixels are included into the priority queue. - Repeat step 3 till completion

All remaining unlabeled pixel are marked as a boundary i.e. the watershed lines.

As it was mentioned there are different strategies for seed points selection. In our *HTML5* demo application we are using user-controlled markers selection using Left Mouse Click and Right Mouse Click to mark foreground and background areas. Some articles discuss different automatic seeds selection using algorithms like **Binarization**, **Morphological Opening, Distance Transform** and so on. Although the focus of this post is not this part of the image segmentation process, we plan to review it in future articles.

Our **HTML5** realization of Watershed Image Segmentation is based on our custom **JavaScript priority queue** object. During the pixels extraction each neighbor of the currently processed point is inserted into our queue. At this insertion the **push** method makes a scan for proper position by a simple **binary search** thus the list remains sorted for the whole process. The nodes comparator is a custom input method and it allows flexible `PQueue`

usage.

```
var PQueue = function(){
this.nodes = [];
}
PQueue.prototype.push = function(el, comparator){
var nodes = this.nodes;
var offset = 0,
size = nodes.length;
while (offset < size){
const pos = ((offset + size) >>> 1);
const diff = comparator(el, nodes[pos]);
((diff < 0) ? (offset = pos + 1) : (size = pos));
}
nodes.splice(offset, 0, el);
}
```

Normally as a distance measurement between pixels the algorithm use the gradient image. In our demo application we are using different weighting function. The weight is calculated based on the improved **RGB Euclidean distance** [2] between the center point and selected neighbor as on the following equation:

`\sqrt{(2\Delta R^2 + 4\Delta G^2 + 3\Delta B^2)}`

Which the following **JavaScript** code segment represents:

```
function diffRGB(p1, p2){
const rmean = (data[p1 + 0] + data[p2 + 0]) >>> 1;
const r = (data[p1 + 0] - data[p2 + 0]),
g = (data[p1 + 1] - data[p2 + 1]),
b = (data[p1 + 2] - data[p2 + 2]);
return Math.sqrt((((512+rmean)*r*r)>>8) + 4*g*g + (((767-rmean)*b*b)>>8));
}
```

As a preprocessing step we eliminate image noise by a Gaussian blur with small sigma value. After this we initialize the image buffer with appropriate label values corresponding to the input seeds:

```
for (var i = 0, l = seeds.length; seeds[i]; i++){
var p = seeds[i].pixels;
for (var j = 0, c = p.length; j < c; j++){
buff[p[j]] = (i + 1)*80; // For example 80 and 160 grey values
addPoint(p[j]);
}
}
```

As a next step we extract all central pixels from our priority queue until we process the whole image:

```
while (!queue.empty()){
var el = queue.pop();
addPoint(el);
}
```

The neighbor pixels are extracted and placed into the PQueue for further processing:

```
function addPoint(p){
for (var i = 0; i < 4; i++){
const pos = p + dirxy[i];
if (buff[pos] === 0){
buff[pos] = buff[p];
buff[pos + 1] = diffRGB(p, pos);
queue.push(pos, comparator);
}
}
}
```

Use **Left** and **Right** mouse buttons to draw **foreground** and **background** seeds. Press the **Execute** button to trigger segmentation.

Sigma: 3

The post Watershed Image Segmentation appeared first on FIVEKO.

]]>The post Skin Detection and Segmentation in RGB Images appeared first on FIVEKO.

]]>- Face Detection
- Hand Detection and Gesture Recognition
- Image Nudity Measurements

Detecting *skin colored* pixels in image scenes seems like a trivial task, but it has proven quite challenging for many reasons. **Skin detection** process depends highly on image properties like illumination, background color and so on.

The process of **skin colors extraction** is performed over colorful input image. An specific algorithm use *color components* to make proper decision is the current pixel fall down to *skin color space* or not. This results in a binary image called **skin mask**.

The **RGB** color space is one of the most widespread color representation consisting from: R – red, G – green and B- blur color components. These three color channels are highly correlated and contain luminance information along with the chromatic components. Because of this humans perception is not linear to RGB color representation. There are significant studies about RGB to **skin mask** conversions. Below are some of the most widely used transformations for **RGB skin color** *segmentation*.

J. Kovac [2] proposed the following **skin RGB color** thresholds using direct RGB channel components:

- At uniform day light illumination:

(R > 95), (G > 40), (B > 20),

max{R, G, B} − min{R, G, B} > 15,

|R − G| > 15, R>G, R>B

**Skin tone**threshold under*flash light*or*daylight lateral*illumination

(R > 220), (G > 210), (B > 170),

|R − G| ≤ 15, (R>B), (G>B)

Researchers had proposed using **normalized RGB** values in order to minimize discrepancy caused by some color combinations. The *RGB normalization* is achieved using the following equations:

r = R / (R + G + B); g = G / (R + G + B); b = B / (R + G + B);

Gomez and Morales [3] determined that the following **normalized rgb **thresholds leads to best *skin mask* results:

r / g > 1.185, (r.b) / pow((r + g + b), 2) > 0.107, (r.g) / pow((r + g + b), 2) > 0.112

Due to the fact that **RGB** to **YCbCr ***color space conversion* is straight forward it is commonly preferred. The *YCbCr *conversion is not computationally intensive compared to **HSL**/**HSV** transforms. The math used for conversion is the following:

`[[Y], [Cb], [Cr]]=[[16], [128], [128]] + [

[65.481, 128.553, 24.966],

[-37.797, -74.203, 112],

[112, -93.786, -18.214]][[R], [G], [B]]`

Using the math above, the resulting **skin regions** are separated by the following equations:

77≤Cb≤127 and 133≤Cr≤173

Our **FivekoGFX** implementation is using a slightly modified equation as proposed into [4]:

80≤Cb≤120 and 133≤Cr≤173

The *source code* below is an **OpenGL shader** that performs a *skin mask* extraction on a **YCbCr** color image. The *thr* variable holds scaled **threshold values** which the **GSLS** uses to decide if the pixel in question is *skin colored* or not.

```
precision mediump float;
// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;
vec4 thr = vec4(80.0/255.0, 120.0/255.0, 133.0/255.0, 173.0/255.0);
void main() {
vec4 color = texture2D(u_image, gl_FragCoord.xy / u_textureSize);
gl_FragColor = vec4(vec3(( (color[0] > thr[0]) &&
(color[1] >= thr[0]) && (color[1] <= thr[1]) &&
(color[2] >= thr[2]) && (color[2] <=thr[3])) ? 1.0 : 0.0),
color.a);
}
```

In many cases sensitive images providing naked bodies are not desired for web sites addressing general audience. Such cases require some sort of data filtering to prevent inappropriate content uploads. A general naive approach for **nudity detection** could be accomplished using following steps:

- Extract RGB pixel values from image in question
- Transform
**RGB**components to**YCbCr***color space*as described above - Use the proposed
**Skin detection algorithm**in order to*segment*skin regions - Calculate skin to non-skin area ratio
- Validate skin ratio against predefined threshold and reject image if it exceeds

In order to increase algorithm robustness it is possible to include **face detection** step to the proposed sequence.

*Human Computer Interaction Using Hand Gestures*- J. Kovac, P. Peer, and F. Solina, “Human skin colour clustering for face detection,” in in Proc. of International Conference on Computer as a Tool, 2003, pp. 144–148.
- Jones, M.J., Rehg, J.M: “Statistical color models with application to skin detection”, International Journal

of Computer Vision (IJCV),46(1), 81-96(2002). - Explicit Image Detection using YCbCr Space Color Model as Skin Detection

The post Skin Detection and Segmentation in RGB Images appeared first on FIVEKO.

]]>The post Symmetric Nearest Neighbor Filter appeared first on FIVEKO.

]]>Similar to the other image processing filters, the *Symmetric NN filter* works using a sliding window at each image pixel. At each position, we split the pixels under the window into pairs of opposite points. Each of these pairs we compare with the central pixel and we use the closest value as an overall window **sum.** Similar to mean filter, the resulting pixel value is the calculated average.

Symmetric Nearest Neighbor pixel selection

green – central pixel; red – opposite pixels

**FivekoGFX** (*Fiveko Graphics*) library uses a GPU version of *Symmetric Nearest Neighbor filter*. Due to this the *Symmetric NN* can run pretty fast as a tiny Fragment Shader (**OpenGL GLSL**). Since the modern web browsers are capable of running OpenGL Shaders using WebGL it is possible to use this filter in online environment as well**. **Finally you are free to look at the below code fragment of our Symmetric NN filter*.*

```
precision mediump float;
// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;
uniform int u_pixelsCount;
#define KERNEL_SIZE %kernelSize%
#define HALF_SIZE (KERNEL_SIZE / 2)
void main() {
vec2 textCoord = gl_FragCoord.xy / u_textureSize;
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
vec4 meanColor = vec4(0);
vec4 v = texture2D(u_image, textCoord);
int count = 0;
for (int y = 0; y <= HALF_SIZE; y++){
for (int x = -HALF_SIZE; x <= HALF_SIZE; x++){
vec4 v1 = texture2D(u_image, textCoord + vec2(x, y) * onePixel);
vec4 v2 = texture2D(u_image, textCoord + vec2(-x, -y) * onePixel);
vec4 d1 = abs(v - v1);
vec4 d2 = abs(v - v2);
vec4 rv = vec4(((d1[0] < d2[0]) ? v1[0] : v2[0]),
((d1[1] < d2[1]) ? v1[1] : v2[1]),
((d1[2] < d2[2]) ? v1[2] : v2[2]),1);
meanColor += rv;
}
}
gl_FragColor = meanColor / float(u_pixelsCount);
}
```

Elapsed: 1ms

- SubSurfWiki – Symmetric nearest neighbour filter
- Hall, M (2007). Smooth operator: smoothing seismic horizons and attributes.
*The Leading Edge***26**(1), January 2007, p16-20. http://dx.doi.org/10.1190/1.2431821

The post Symmetric Nearest Neighbor Filter appeared first on FIVEKO.

]]>The post Gradient Non-Maximum Suppression appeared first on FIVEKO.

]]>

The process of extraction thin **stable edges** is preceded by gradient magnitude and orientation calculation. Frequently *edge parameters* are resolved using square masks with size *3×3*. The image is processed using these masks in a way that gradients strength and orientation are retrieved as described into Sobel tutorial. The result of this process will lead to **thick edges** with different magnitude as on the following image.

After execution of gradient thinning operator the resulting image will look like the one bellow.

After the edge detection step is done it is time to process gradients and retrieve thin edges ideally one pixel width. One of the most widely used technique is the so called **Non-maximum suppression**. It can be described by the following steps:

* Iterate through each pixel

* For each edge pixel choose its two neighbors based on gradient direction and use them for comparison

*Edge neighbors based on gradient direction*

If the gradient magnitude is lower than one of the both neighbors suppress it by making it zero (black/background). Otherwise keep it unchanged and proceed with the next one

The gradient magnitude and orientation is calculated as described into the Sobel article. Then using the gradient parameters it comes turn to NMS fragment shader execution. Using **degrees** operator the shader convert angle parameter stored into **u_image** from radians to degrees as **theta**. Each orientation is used to find the proper neighbors **ca** and **cb**. Finally compare each two neighbors with the central gradient magnitude **cc** and suppress if needed.

```
precision mediump float;
#define KERNEL_SIZE 3
// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;
#define M_PI 3.1415926535897932384626433832795
void main() {
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
vec2 textCoord = gl_FragCoord.xy / u_textureSize;
vec4 cc = texture2D(u_image, textCoord);
float theta = degrees(cc.y*M_PI*2.0);
int ax = 0, ay = 0;
if ((theta >= 337.5) || (theta < 22.5)) { ax = 1; ay = 0; }
else if ((theta >= 22.5) && (theta < 67.5)) { ax = 1; ay = 1; }
else if ((theta >= 67.5) && (theta < 112.5)) { ax = 0; ay = 1; }
else if ((theta >= 112.5) && (theta < 157.5)) { ax =-1; ay = 1; }
else if ((theta >= 157.5) && (theta < 202.5)) { ax =-1; ay = 0; }
else if ((theta >=202.5) && (theta < 247.5)) { ax =-1; ay =-1; }
else if ((theta >=247.5) && (theta < 292.5)) { ax = 0; ay =-1; }
else if ((theta >= 292.5) && (theta < 337.5)) { ax = 1; ay =-1; }
vec4 ca = texture2D(u_image, textCoord + onePixel*vec2(ax, ay));
vec4 cb = texture2D(u_image, textCoord + onePixel*vec2(-ax, -ay));
gl_FragColor = vec4((((cc.x <= ca.x) || (cc.x < cb.x)) ? vec3(0) : vec3(cc.x)), 1.0);
}
```

Sigma: 3

The post Gradient Non-Maximum Suppression appeared first on FIVEKO.

]]>The post Sobel Filter – Sobel-Feldman Edge Detection and Gradient Extraction appeared first on FIVEKO.

]]>*Sobel operator – 3×3 mask window*

The image blow is an example result of Sobel operator and point out the enhanced objects outlines.

The *Sobel filter* is represented by two 3×3 **convolution** kernels and is computational efficient for both CPU and for GPU parallel computing. Furthermore the Sobel kernels are separable, which is additional option for optimization.

Each image pixel is processed by each kernel in order to produce the final *gradient* value using equation (2). For better performance it is acceptable to use sum of absolute directional values as in equation (3).

After calculating the **vertical** and **horizontal** *gradients*, it is possible to retrieve the edge orientation, as described in (4):

Now as we have ways to calculate **gradient magnitude** and **orientation** it is possible to get **thin** and **strong edges.** This will help for further image analysis and *objects recognition*. You may also check edge thinning by non-maximum suppression for further steps.

First we define two helper functions, which we use for convolution operation. The **conv3x** we use to calculate **Gx** and the second one **conv3y** we use for **Gy**.

```
function conv3x(data, idx, w, p){
return (p[0]*data[idx - w - 4] + p[1]*data[idx - 4] + p[2]*data[idx + w - 4]
-p[0]*data[idx - w + 4] - p[1]*data[idx + 4] - p[2]*data[idx + 4 + 4]);
}
function conv3y(data, idx, w, p){
return (p[0]*data[idx - w - 4] + p[1]*data[idx - w] + p[2]*data[idx - w + 4]
-(p[0]*data[idx + w - 4] + p[1]*data[idx + w] + p[2]*data[idx + w + 4]));
}
```

Next we perform image iteration, so at each pixel we calculate **Gx** and **Gy** as described above.

```
for (j = 0, i = w + 4; i < l; i+=4, j++){
var dx = conv3x(data, i, w, mask);
var dy = conv3y(data, i, w, mask);
//var val = Math.sqrt(dx*dx + dy*dy);
var val = Math.abs(dx) + Math.abs(dy);
buff[j] = val;
}
```

Note that for Sobel operator the values in our mask are: **[1, 2, 1]**.

The source code below represents a way to calculate gradient magnitude and orientation using **Graphics Processing Unit** (GPU) by **WebGL**/**OpenGL** API. The **u_kernel** is an input parameter holding the *Sobel* coefficients. Using the above description we can calculate **dx **and **dy** by math **(1)**. As a next step the **GLSL** shader calculates the gradient orientation **theta** as in **(4)**. Note that we should scale the *orientation angle* from range **[0.0, 2.0*PI]** to **[0.0, 1.0]**. Finally the *gradient magnitude* and *gradient orientation* are set to the first two components of **gl_FragColor** vector. We keep gradients orientation for further processing by Non Maximum Suppression, which produce **thin edges**.

```
precision mediump float;
#define KERNEL_SIZE 3
// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;
uniform float u_kernel[KERNEL_SIZE];
#define M_PI 3.1415926535897932384626433832795
#define GET_PIXEL(_x, _y) (texture2D(u_image, textCoord + onePixel*vec2(_x, _y)))
void main() {
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
vec2 textCoord = gl_FragCoord.xy / u_textureSize;
float dx = (length(GET_PIXEL(-1, -1)*u_kernel[0] +
GET_PIXEL(-1, 0)*u_kernel[1] +
GET_PIXEL(-1, +1)*u_kernel[2]) -
length(GET_PIXEL(+1, -1)*u_kernel[0] +
GET_PIXEL(+1, 0)*u_kernel[1] +
GET_PIXEL(+1, +1)*u_kernel[2]));
float dy = (length(GET_PIXEL(-1, -1)*u_kernel[0] +
GET_PIXEL(0, -1)*u_kernel[1] +
GET_PIXEL(+1, -1)*u_kernel[2]) -
length(GET_PIXEL(-1, +1)*u_kernel[0] +
GET_PIXEL(0, +1)*u_kernel[1] +
GET_PIXEL(+1, +1)*u_kernel[2]));
///gl_FragColor = vec4(vec3(length(vec2(dx, dy))), 1.0);
float theta = (atan(dy, dx) + M_PI) / (2.0*M_PI);
gl_FragColor = vec4(length(vec2(dx, dy)), theta, 0.0, 1.0);
}
```

Sigma: 3

- Irwin Sobel, 2014,
*History and Definition of the Sobel Operator*

The post Sobel Filter – Sobel-Feldman Edge Detection and Gradient Extraction appeared first on FIVEKO.

]]>The post Median Filter – Edge Preserving Filter appeared first on FIVEKO.

]]>The **Median filter** is working using a sliding window iterating through each signal entry one by one. At each point *numerically sort* the list of adjacent neighbors and then replace central value with the middle one from the list. The classical algorithm uses sorting, which is not much efficient and may lead to bigger execution delay. Thus in attempts for optimization there are several different techniques for finding the middle value.

**Array sorting**– this is a classical algorithm where we sort the values below the median kernel and as a result we use the value in the middle**Selection algorithm**– it is acceptable to use partial sorting or other selection method, since the goal is simply to find the middle value.**Histogram medians**– at window movement we can update**histogram**with pixel changes and use it further to resolve the**median value**

The following picture displays an example of *3×3* **sliding window** and how the algorithm process current pixel.

*Replacing current pixel from 3×3 filter window*

The image above show how the sliding mask represents the matrix as 1D array of pixel values. To extract the *middle value* the algorithm sorts the neighbors array . Then the middle value replaces the central pixel.

```
/*
Created by FIVEKO.com 2017
The source code is designed and developed for educational purposes only.
It is designed for no other purpose and neither the authors nor their institutions
accept any liability concerning its use.
*/
"use strict";
function median(pixels, size, colors) {
var d = pixels.data;
var buff = new pixels.data.constructor(new ArrayBuffer(pixels.data.length));
var w = pixels.width;
var h = pixels.height;
const bpr = w*4; // Bytes per row
if (size < 3) return;
const ms = Math.floor(size / 2);
const mc = Math.floor((size*size) / 2);
// Find the middle value by histogram traversal
function getMiddle(h, n){
for (var count = h[0], i = 1; i < 255; i++, count += h[i]){ if (count >= n){
return i;
}
}
return 255;
}
colors = colors || 3;
for (var i, j, c = 0; c < colors; c++){
var srcIt, dstIt, l;
for (i = ms; i < h - ms; i++) {
var hist = new Uint16Array(256);
for (var ii = (i - ms)*bpr; ii <= (i + ms)*bpr; ii+=bpr){
for (var jj = c; jj < (ms*2)*4 + c; jj+=4){
hist[d[ii + jj]]++;
}
}
for (j = ms, dstIt = i*bpr + j*4 + c; j < w - ms; j++, dstIt+=4){
const offset = (i - ms)*bpr + c;
// Add rigtmost values to the histogram
for (srcIt = offset + (j + ms)*4,
l = srcIt + bpr*size; srcIt < l; srcIt+=bpr){
hist[d[srcIt]]++;
}
buff[dstIt] = getMiddle(hist, mc);
// Remove leftmost values to the histogram
for (srcIt = offset + (j - ms)*4,
l = srcIt + bpr*size; srcIt < l; srcIt+=bpr){ hist[d[srcIt]]--; } } } } if (colors > 1){
for (var i = 3; i < l; i+=4){
buff[i] = 255;
}
} else {
for (var i = 3; i < l; i+=4){
buff[i] = 255;
buff[i - 2] = buff[i - 1] = buff[i - 3];
}
}
pixels.data.set(buff);
return pixels;
}
```

Kernel size: 3

The post Median Filter – Edge Preserving Filter appeared first on FIVEKO.

]]>The post Mean Filter for Fast Noise Reduction appeared first on FIVEKO.

]]>Even though the modern camera devices provide pretty good quality, often pictures are **noisy**. In such cases a common technique is to use **low-pass filter** to prevent *high frequency noise* artifacts. Using such filter results in a *blurry image*, so frequently we refer to *Median filter* as “**blurring**” or “**smoothing**” filter. Like the Gaussian blur, an uniform blurring with *mean filter* is done using a convolution kernel like the one below. Keep in mind that this filter is also a separable one, which means it’s usually better to execute it once for rows and once for columns.

In addition to mean filter separation it is possible to apply additional optimization called **“moving averages”** and reduce its complexity further to O(1). The following steps describe the process:

- Iterate through each row/column in a single pass
- For the first pixel use the whole kernel size and calculate the sum and write the
**mean**as output - For each next pixel use the sum from the previous step and subtract values which are no longer under the kernel.
- Add values at the right.
- Calculate the average sum and write it as output

This section provides a **GPU** based approach using **WebGL/OpenGL **shader** **for fast image blurring by iterative *mean filtration*. We base our algorithm on the fact that *Gaussian filter* approximates well by multiple iterations of **box** or so called **mean** **filter**. Equation (1) describes the *standard deviation* of mean filter with size **n**, since it corresponds to uniform probability distribution. In case of multiple iterations **m** the resulting std deviation is result from equation 2.

Thus in order to calculate number of iterations *m* for known kernel size (e.g. *n=3*) and provided sigma, the equation could be represented as following:

The source code below represents **fragment shader (GLSL)** of mean filter. The filter window has a size of 3×3, and for the number of iterations we use the math equation (3) from the above section.

```
precision mediump float;
// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;
void main() {
vec2 textCoord = gl_FragCoord.xy / u_textureSize;
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
gl_FragColor = (
texture2D(u_image, textCoord + onePixel*vec2(-1.0, -1.0)) +
texture2D(u_image, textCoord + onePixel*vec2(0.0, -1.0)) +
texture2D(u_image, textCoord + onePixel*vec2(1.0, -1.0)) +
texture2D(u_image, textCoord + onePixel*vec2(-1.0, 0.0)) +
texture2D(u_image, textCoord + onePixel*vec2(0.0, 0.0)) +
texture2D(u_image, textCoord + onePixel*vec2(1.0, 0.0)) +
texture2D(u_image, textCoord + onePixel*vec2(-1.0, 1.0)) +
texture2D(u_image, textCoord + onePixel*vec2(0.0, 1.0)) +
texture2D(u_image, textCoord + onePixel*vec2(1.0, 1.0))) / 9.0;
}
```

Sigma: 3

The post Mean Filter for Fast Noise Reduction appeared first on FIVEKO.

]]>The post Hough Transform – Basic Shape Detection appeared first on FIVEKO.

]]>- Extract edges using e.g. Sobel edge detector
- Choose desired transformation e.g. line transform or circle transform
- Iterate through the image and populate the accumulator using the transform from previous step
- Search for the local maximum values inside the accumulator space

A line that pass through a point x,y can be represented using the Polar coordinate system by the following equation:

The equation above is used in a way that every significant edge pixel (e.g. *x0*,*y0*) is processed and plot all possible family of lines. Meaning that for each such point the prepared Polar accumulator (r, theta) is filled with calculated radius values at all possible angles. After the whole image is processed and the accumulator is filled, the last step is to find local maximums and retrieve the resulting lines. It should be noted that the returned lines do not contain any information about length and further processing is required to match the exact parts of the image with corresponding lines.

The math behind the **Hough circle transform** is similar to this used for lines detection described above. The voting accumulator cells are determined by the following equation:

In order to optimize the algorithm execution and reduce needed time we use a pre-calculated **lookup table** (**LUT**) with *x* and *y* values respectively. The JavaScript code snippet below shows the lookup table creation:

```
// Compute the circle kernel (LUT)
var kernel = [{x: ~~(r*Math.cos(0)), y: ~~(r*Math.sin(0))}];
for (var i = 1; i < 380; i++){
var phi = i*Math.PI/180;
var x = ~~(r*Math.cos(phi));
var y = ~~(r*Math.sin(phi));
var last_element = kernel[kernel.length - 1];
if (last_element.x != x || last_element.y != y)
kernel.push({x: x, y: y});
}
```

Next step is to loop through image pixels and execute the voting procedures. At every pixel the overall sum is calculated using the lookup table. See the code fragment below:

```
// Create buffer using pixels.data ctor
var buff = new pixels.data.constructor(new ArrayBuffer(pixels.data.length));
var data = pixels.data;
var w = pixels.width - r;
var h = pixels.height - r;
var i, j, bpr = pixels.width*4;
for (i = r; i < h; i++){
for (j = r; j < w; j++){
var sum = 0, l = kernel.length;
for (var k = 0; k < l; k++){
var x = j + kernel[k].x;
var y = i + kernel[k].y;
sum += data[bpr*y + x*4]; // Assume it's grayscale
}
const pos = bpr*i + j*4;
buff[pos + 0] = buff[pos + 1] = buff[pos + 2] = ~~(sum / l);
buff[pos + 3] = 255;
}
}
pixels.data.set(buff);
}
```

In case of **GPU** based algorithm, the *Hough Circle Transform* can be obtained using the following **OpenGL fragment shader**:

```
precision mediump float;
// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;
uniform float u_r;
#define M_PI 3.1415926535897932384626433832795
#define PHI_STEP M_PI/180.0
void main() {
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
vec2 textCoord = gl_FragCoord.xy / u_textureSize;
vec4 sum = vec4(0.0);
float phi = 0.0;
for (int i = 0; i < 360; i++)
{
phi += PHI_STEP;
sum += texture2D(u_image, textCoord + onePixel*vec2(u_r*cos(phi), u_r*sin(phi)));
}
gl_FragColor = vec4(vec3(sum / 360.0), 1.0);
}
```

Sigma: 3

The post Hough Transform – Basic Shape Detection appeared first on FIVEKO.

]]>