Gabor Filter – Image processing for scientists and engineers, Part 6

In the last two sections of this tutorial, we discussed Gaussian and Fourier kernels. The Gaussian kernel is good at weighting neighbors by their proximity to the pixel of interest, and the Fourier kernel gives us information about patterns in the image. So, uh, let’s… multiply… them?

Look at that. New filter. When kernel-ized, it looks like:

Its functionality can be summarized by taking the descriptions of the Gaussian and Fourier kernels and mashing them together. Something like “finding local patterns”. As most images don’t have universal patterns (like, say, a sound wave does), this is a very useful kernel. Also, every single paper that ever uses the Gabor kernel cites it as the best approximation of how humans process visual data. Seeing how I don’t even know how to explain color to a blind man, I have no idea how someone verified this claim. But, whatever. It’s cool, right?

Algorithms involving the Gabor kernel most frequently pop up in scenarios where one needs to test images for similarity; namely, they’re at the core of reverse image searching and fingerprint detection. In my own work, I’ve managed to use them to reconstitute experimental image sequences that got jumbled together by lazy file naming schemes. Less exciting than fingerprint detection, but it did save us from having to redo a lot of annoying lab work.

There is one obnoxious issue with the raw Gabor filter… it has a lot of parameters! It adopts the arguments from both the Gaussian and Fourier kernels, so image characterization by raw Gabor can be rather time intensive. To circumvent this, you commonly see some heuristics applied. In the Gaussian section, I talked about using the natural decay of the function to convert kernel size into a dependent parameter. Additionally, we can relate the Gaussian decay to the wavelength of the Fourier function, removing the parameter entirely. Coding this all up…

In conclusion

While there is much more to learn in the field of image processing, I’m choosing to end my tutorial here. Why? Primarily, I have found that convolution is the most conceptually complex part of image processing. Once this concept “clicks” with a new student, much of the remaining techniques in the field fall into place with little effort. I hope that by focusing on this sub-topic, I have helped soften the image-processing learning curve and have enabled cool new scientific projects.

If you’re looking to continue increasing your knowledge in image processing, I recommend going straight to the literature. I know this sounds crazy – you’ve only started in the field – but, if you keep in mind the mathematical simplifications I discussed throughout the tutorial, papers in this field read very easily.

A great place to start would be Canny edge detection, which detects edges in an image by applying a Gaussian blur, then a Sobel gradient, and then some fancy terms (non-maximum suppression and recursive hysteresis) that encode very basic operations (make edges one pixel wide by keeping the strongest gradients, and have threshold values to control what is considered an “edge”). You could build this starting from the gradient lesson (which, on its own, is a decent edge detector), then add in noise-reduction pre-processing with a blurring operation, and then tackle the complex sounding stuff, if needed.

Another one is combining the Gabor filter with Earth Mover’s Distance metrics to create a reverse-image-search program. This is not nearly as difficult as it sounds. The earth mover’s distance is easy to visualize as a tractor moving piles of dirt into holes (therefore, the name), and the Gabor filter shown above is nothing more than a tool to emphasize localized textures in an image.

Finally, you can easily use these tools to make your (or your friends’) experimental work easier! For example, let’s say you are tracking a cell with a microscope that has a computer interface. The cell keeps moving, so someone is forced to sit in the lab for hours on end moving the lens to keep the cell approximately centered. That’s annoying and repetitive. Fix that. In this case, you can use a threshold filter (or edge/blob detection, if you’re feeling fancy) to find the cell in each recorded tracked frame. Find the centroid by averaging all of the pixels constituting the cell, and then find a way to get the script to move the microscope to recenter the cell (good microscopy software offers an API and documentation to do this; with bad software, you could run a mouse-clicking macro to achieve the same effect).

Once again, I hope these lessons helped. If you’ve been doing anything interesting with image processing in the lab, leave a comment below!

5 Thoughts on “Gabor Filter – Image processing for scientists and engineers, Part 6”

1. Xocoatzin on 19 April 2013 at 8:57 am said:

No part 7

• I know I need to finish this tutorial.

2. Nishant on 14 August 2013 at 1:19 pm said:

Dear Patrick,

Your tutorials are very helpful. This is shown from the fact that this is the first time that I am posting comment after reading any tutorial. You have explained tough maths in easy to understand terms and your 3D-izer is very cool. I am trying to run your script on windows machine, but no success yet. A more detailed step version will be really helpful.

Best regards,

Nishant

• Hi Nishant,

Thanks for the compliment!

I’ve only tested the image 3d-izer on mac/linux, but it should be doable on windows. Let’s see if I can help.

The code works in two steps: image -> data file, then data file -> blender. The first step (done in image_to_json.py) used Python Imaging Library, which can be installed from here: http://www.pythonware.com/products/pil/. If you’re comfortable with the command line, you can also do pip install PIL. The second step (done in json_image_to_blender.py) only requires Blender. If you switch the view in Blender from “3D view” to “text editor” (option is on the bottom left), you can copy and paste the code from json_image_to_blender.py into Blender itself.

Good luck!

3. Emre YAZICI on 1 October 2013 at 5:57 am said:

Wov…

Thank you so much for this document.