This is an extension of the example Use image filters to perform edge detection, smoothing, embossing, and more in C# that adds new features that warp images in arbitrary ways.

The idea is to use two functions F(x, y) and G(x, y) to map the pixel locations (x, y) in an input image to new positions (F(x, y), G(x, y)) in the output image. Unfortunately you can’t warp images by simply mapping the input image’s pixels to the output image. If you do, some (most) of the resulting positions (F(x, y), G(x, y)) will not be at integer locations. You could try to solve that problem by rounding off the position to the nearest pixel, but that would probably not produce a smooth result. It would also mean that multiple pixels might be mapped to the same location and some locations might not be mapped by any pixels in the original image.

The solution is to map pixels in the output image *back* to the positions they should have come from in the input image. For output position (x1, y1), you get an input position (x0, y0) where x0 and y0 are not necessarily integers. You can then use a weighted average of the pixels surrounding (x1, y1) in the input image to determine the color of the output pixel.

This example adds the following warp type enumeration to the `Bitmap32` class. (For information on that class, see the earlier example.)

// Warping types. public enum WarpOperations { Identity, FishEye, Twist, Wave, SmallTop, Wiggles, DoubleWave, }

The `Bitmap32` class also provides the following `Warp` method to use the warp types.

// Warp an image and return a new Bitmap32 holding the result. public Bitmap32 Warp(WarpOperations warp_op, bool lock_result) { // Make a copy of this Bitmap32. Bitmap32 result = this.Clone(); // Lock both bitmaps. bool was_locked = this.IsLocked; this.LockBitmap(); result.LockBitmap(); // Warp the image. WarpImage(this, result, warp_op); // Unlock the bitmaps. if (!lock_result) result.UnlockBitmap(); if (!was_locked) this.UnlockBitmap(); // Return the result. return result; }

To warp images, this code makes a copy of the input image to hold the resulting warped image and locks both images. It then calls the `WarpImage` method to do most of the work. It finishes by unlocking the images if appropriate and returning the result.

The following code shows the `WarpImage` method.

// Transform the image. private static void WarpImage(Bitmap32 bm_src, Bitmap32 bm_dest, WarpOperations warp_op) { // Calculate some image information. double xmid = bm_dest.Width / 2.0; double ymid = bm_dest.Height / 2.0; double rmax = bm_dest.Width * 0.75; int ix_max = bm_src.Width - 2; int iy_max = bm_src.Height - 2; // Generate a result for each output pixel. double x0, y0; for (int y1 = 0; y1 < bm_dest.Height; y1++) { for (int x1 = 0; x1 < bm_dest.Width; x1++) { // Map back to the source image. MapPixel(warp_op, xmid, ymid, rmax, x1, y1, out x0, out y0); // Interpolate to get the result pixel's value. // Find the next smaller integral position. int ix0 = (int)x0; int iy0 = (int)y0; // See if this is out of bounds. if ((ix0 < 0) || (ix0 > ix_max) || (iy0 < 0) || (iy0 > iy_max)) { // The point is outside the image. Use white. bm_dest.SetPixel(x1, y1, 255, 255, 255, 255); } else { // The point lies within the image. // Calculate its value. double dx0 = x0 - ix0; double dy0 = y0 - iy0; double dx1 = 1 - dx0; double dy1 = 1 - dy0; // Get the colors of the surrounding pixels. byte r00, g00, b00, a00, r01, g01, b01, a01, r10, g10, b10, a10, r11, g11, b11, a11; bm_src.GetPixel(ix0, iy0, out r00, out g00, out b00, out a00); bm_src.GetPixel(ix0, iy0 + 1, out r01, out g01, out b01, out a01); bm_src.GetPixel(ix0 + 1, iy0, out r10, out g10, out b10, out a10); bm_src.GetPixel(ix0 + 1, iy0 + 1, out r11, out g11, out b11, out a11); // Compute the weighted average. int r = (int)( r00 * dx1 * dy1 + r01 * dx1 * dy0 + r10 * dx0 * dy1 + r11 * dx0 * dy0); int g = (int)( g00 * dx1 * dy1 + g01 * dx1 * dy0 + g10 * dx0 * dy1 + g11 * dx0 * dy0); int b = (int)( b00 * dx1 * dy1 + b01 * dx1 * dy0 + b10 * dx0 * dy1 + b11 * dx0 * dy0); int a = (int)( a00 * dx1 * dy1 + a01 * dx1 * dy0 + a10 * dx0 * dy1 + a11 * dx0 * dy0); bm_dest.SetPixel(x1, y1, (byte)r, (byte)g, (byte)b, (byte)a); } } } }

This method first calculates some values for the warping functions to use. It then makes the variables `x1` and `y1` loop over the pixels in the output image. For each output pixel `(x1, y1)`, the code calls the `MapPixel` method to map that pixel back to an input position `(x0, y0)` where `x0` and `y0` are not necessarily integers. As you’ll see shortly, `MapPixel` returns different pixels depending on which warp type is passed to it.

Next the code uses bilinear interpolation to pick a color for the output pixel. To do that, it calculates the distances `dx0`, `dy0`, `dx1`, and `dy1` between the input position `(x0, y0)` and the integral pixel values nearest to `x0` and `y0`. It then multiplies the color components of those nearest pixels by the distances to get a weighted average.

To see how this works, consider the picture on the right. The point `(x0, y0)` is the point in the input image that the output point mapped back to. The other points are the nearest pixels.

Now suppose that `x0` is exactly halfway between `ix0` and `ix0 + 1`. In that case, `dx0` and `dx1` are both `0.5`. To calculate the color of the upper dashed point shown in the picture, you take the weighted average of the two upper points. In this case that would be `[color of upper left pixel] * 0.5 + [color of upper right pixel] * 0.5`. In this example, the upper left pixel is red and the upper right pixel is white so the result is pink.

The program calculates the weights as `weight1 = 1 - dx0 = dx1` and `weight2 = 1 - dx1 = dx0`. In the picture, the point `(x0, y0)` is actually a bit closer to the right pixel, so the correct weights should be something more like `0.3` and `0.5`, giving a brighter pink.

Similarly you can calculate that the color of the the bottom dashed point should be a light blue.

Finally you can interpolate between the two dashed points to determine that the actual point `(x0, y0)` should be a sort of purplish color. That is the color that the program assigns to the output pixel `(x1, y1)` in the result picture.

The only piece remaining for this example is the `MapPixel` method that maps an output pixel back to an input pixel. Because this post as gone rather long, I’ll describe that method in my next post.

Outstanding. I’ve acutally been searching for a simple walkthrough like this, and this is perfect.

Pingback: Warp images arbitrarily in C#, Part 2 - C# HelperC# Helper

Pingback: Use DrawImage to warp images in C# - C# HelperC# Helper

Hi Rod – Just wanted to say thanks from a long time fan. I have had your VB Graphics Programming on my book shelf as a reference for years. I just read your explanation of bilinear interpolation above and it is by far the best I have found. The graphic you supplied with it is worth 1,000 words. Do you have something similar covering bicubic interpolation? Would you use this same approach to combine reducing the size of an image and converting it to grayscale in one pass?

Thanks again!

Thanks for the kind words! It’s always nice to hear that someone is getting something from my books. Post a review when you have a chance!

I haven’t implemented bicubic interpolation. (Sorry I don’t have an example for you.) It uses the same basic idea as bilinear interpolation, but instead of using the four points surrounding the target point, it uses 16 points. The points farther away model the rate of change (slope) of the color values, so the method tends to smooth out variations in color. Three’s a good overview on Wikipedia at:

Bicubic interpolation

The picture on that post’s upper right is a bicubic version of the picture in this post. Basically the method makes a two-dimensional spline in color-space to approximate the missing pixels’ colors.

I probably wouldn’t combine resizing and color conversion into a single operation. It would be more complicated, and bicubic interpolation is already complicated enough. It would also reduce your flexibility. For example, you might want to change the way you convert to grayscale. You might want to use averaging, weighted averages of color components, sepia tone, or some other method. See this post: Use an ImageAttributes object to apply general color tones to an image in C#

On the other side, you might want to try bilinear interpolation to see which produces a better result. And you might want to try other filters such as edge detectors and low pass and high pass filters.

It’s easier to keep all of those separate rather than merging them so you can use different combinations.