Always Move Fast

I really wanted to finish this article last week, but I just didn’t get around to it. This post is going to be on the things you can do with the Python Imaging Library when you implement your own kernels. Remember that depending on the kernel arguments you supply, you may get radically different results. The results may not even be a smoothed image; they could be embossed images, edge maps, etc. With that out of the way, let’s get started.

There are a couple different ways you can implement your own image filters using PIL. The first way is easy for one-off filters that you probably won’t use throughout your code. I mean, you could if you wanted to keep writing the same line of code over and over, but that’s up to you. The way this is done is by making use of PIL’s ImageFilter.Kernel class. You can create an object from this class that has all the common image filter arguments (size, kernel, scale, and offset).

That’s the quick and dirty way to implement your own image filters. It uses the same filter arguments as ImageFilter.SMOOTH so it yields an identical result.

The second way to implement your own image filters is to do it just like PIL does. For each image filter class that PIL provides, you’ll see that they inherit from the ImageFilter.BuiltinFilter class which in turn inherits from the ImageFilter.Kernel class. The stock implementation for ImageFilter.SMOOTH is as follows:

Where filterargs is the size of the kernel (3, 3), the scale factor (13), the offset (0), and the kernel itself. Now since the stock PIL filters are boring and I don’t want this post to be about how the stock filters are implemented, I implemented my own image filter.

The result of running this filter on my picture is this:

Pretty cool, eh? I highly encourage you to implement your own filters and dig into the stock PIL image filters to see how they achieve certain effects. With your own image filters and the stock PIL image filters at your command, you may never have to write your own image processing algorithms at all.

Smoothing an image is helpful for a few different reasons. The big one that I’ve been using image smoothing for is to remove noise from an image. Laplacian edge detection is highly susceptible to noise due to the Laplacian operator being a second derivative operator. The Python Imaging Library (PIL) provides a few different ways to produce smooth images. The most obvious would be to use the ImageFilter.SMOOTH class.

The image I’ll be working on throughout this post is this greyscale picture of me:

Standard smoothing in PIL is super easy. For example:

Using the image of me above, here’s the result of this smoothing operation:

That’s a pretty good result for just using stock PIL image filters. The Python Imaging Library also offers a SMOOTH_MORE filter. Replacing the ImageFilter.SMOOTH above with ImageFilter.SMOOTH_MORE, we get:

Digging into the PIL source gives a really good indication of how it produces these results. For example, the BuiltinFilter classes (such as ImageFilter.SMOOTH) use filter arguments to produce different results. These filter arguments are a size tuple, which is the width and height of the kernel, the convolution kernel itself as a sequence containing weighted values, the scale which is used to divide the result of each pixel, and finally the offset which is added to the result after it has been divided by the scale factor.

For ImageFilter.SMOOTH, these filter arguments are:

I’m going to present a way to implement image smoothing so you can have a better idea of what’s really going on when you smooth an image. One small note: the scale part of the process defaults to the sum of the weights in the kernel. So if it isn’t present, you can calculate the default scale by doing this:

With that out of the way, here’s an implementation of image smoothing that is equivalent to calling ImageFilter.SMOOTH like the code above:

Notice how it loops through 1 <= y < height-1 and 1 <= x < width? That's because doing this processing through the entire image produces weird borders. The code above compensates for this and eliminates the dark borders by copying the original border pixels to the borders of the output image.

So what happens when I run the code above on the original image? Check it out:

Being able to code this kind of stuff is really cool, but for these really basic examples of standard smoothing, using PIL’s built-in filters is definitely the way to go. You’re also not limited to using PIL’s built-in filters. If you require different kernels and even different scaling and offset attributes, PIL provides ways for you to do that.

I was hoping to cover the ways you can implement your own filters in PIL, but it’s getting late so I will try covering them tomorrow if I have time.

Detecting edges in images is being actively researched for many different applications. The most notable of these applications is computer vision. The reason I began studying edge detection algorithms, aside from them being really cool, is that I’ve been noticing that I can use edge detection as part of my toolbox for optical character recognition.

So, how do we define edges in any given image? Edges are really just areas where the pixels intensities contrast. Basically where you have a bunch of light pixels touching a bunch of dark pixels. There are a couple different methods used for detecting edges: gradient and Laplacian. I’m going to be covering a basic gradient edge detection technique and will cover Laplacian techniques in future posts.

Gradient edge detection approximates the first derivative of the image, looking for minimum and maximum intensities in the magnitude of the gradient. Locating edge pixels can be done by setting a threshold of some value and testing if the gradient is greater than that threshold.

The gradient of the image function I is given by the vector:

ߜ I = [∂I / ∂x, ∂I / ∂y]

To approximate the first derivative of the image, we use convolution masks. The method I’m going to present is the Prewitt method. It uses two masks to approximate ∂I / ∂x and ∂I / ∂y, giving us a gradient of the image’s pixels. ∂I / ∂x and ∂I / ∂y detect vertical and horizontal edges, respectively. The masks that define ∂I / ∂x and ∂I / ∂y for the Prewitt operator are:

∂I / ∂x:
[-1, 0, 1]
[-1, 0, 1]
[-1, 0, 1]

∂I / ∂y:
[1, 1, 1]
[0, 0, 0]
[-1, -1, -1]

The resulting outputs of convolving the image with these masks are then added to get the magnitude of the gradient. The magnitude of the gradient is given by:

|G| = sqrt(Gx² + Gy²)

To approximate the magnitude of the gradient, we use:

|G| = |Gx| + |Gy|

After getting the magnitude of the gradient, we want to check if it’s larger than our threshold. All the methods I’ve seen use a threshold of 255. What this means is that when the magnitude of the gradient is larger than 255, we’ve found an edge. We cap the magnitude to 255 if it’s larger than 255 and mark the pixel in the output image as a 0, which is black. This is done implicitly by setting the pixel value to 255 - magnitude, meaning if the magnitude is 255, the pixel value is black. Magnitudes of 0 will set the pixel to 255 - 0, which is white. The magnitudes can be any value between 0 and 255, inclusive.

The Prewitt masks in Python are given by the function get_prewitt_masks():

Now on to the meat of the entire operation. The prewitt() function takes a 1-d array of pixels and the width and height of the input image. It returns a greyscale edge map image.

You can store this code all in one file so when you run it, you can pass the program arguments for the input and output image filenames on the command line. To do so, add this code to the Python file with the edge detection code from earlier:

I called my file prewitt.py, so with all that code in the same file, you can call it from the command line:

$ python prewitt.py input_image.gif output_image.gif

Note that it will work for pretty much any image type you give it. Here are some results of me running the code above:

I suppose that concludes this article on basic edge detection using the Prewitt method. Hope you enjoyed reading it as much as I enjoyed writing it!