Title: Warp images arbitrarily in C#, Part 1

This is an extension of the example Use image filters to perform edge detection, smoothing, embossing, and more in C# that adds new features to warp images in arbitrary ways.

The idea is to use two functions F(x, y) and G(x, y) to map the pixel locations (x, y) in an input image to new positions (F(x, y), G(x, y)) in the output image. Unfortunately you can't warp images by simply mapping the input image's pixels to the output image. If you do, some (most) of the resulting positions (F(x, y), G(x, y)) will not be at integer locations. You could try to solve that problem by rounding off the position to the nearest pixel, but that would probably not produce a smooth result. It would also mean that multiple pixels might be mapped to the same location and some locations might not be mapped by any pixels in the original image.

The solution is to map pixels in the output image back to the positions they should have come from in the input image. For output position (x1, y1), you get an input position (x0, y0) where x0 and y0 are not necessarily integers. You can then use a weighted average of the pixels surrounding (x1, y1) in the input image to determine the color of the output pixel.

This example adds the following warp type enumeration to the Bitmap32 class. (For information on that class, see the earlier example.)

// Warping types. public enum WarpOperations { Identity, FishEye, Twist, Wave, SmallTop, Wiggles, DoubleWave, }

The Bitmap32 class also provides the following Warp method to use the warp types.

// Warp an image and return a new Bitmap32 holding the result. public Bitmap32 Warp(WarpOperations warp_op, bool lock_result) { // Make a copy of this Bitmap32. Bitmap32 result = this.Clone(); // Lock both bitmaps. bool was_locked = this.IsLocked; this.LockBitmap(); result.LockBitmap(); // Warp the image. WarpImage(this, result, warp_op); // Unlock the bitmaps. if (!lock_result) result.UnlockBitmap(); if (!was_locked) this.UnlockBitmap(); // Return the result. return result; }

To warp images, this code makes a copy of the input image to hold the resulting warped image and locks both images. It then calls the WarpImage method to do most of the work. It finishes by unlocking the images if appropriate and returning the result.

The following code shows the WarpImage method.

// Transform the image. private static void WarpImage(Bitmap32 bm_src, Bitmap32 bm_dest, WarpOperations warp_op) { // Calculate some image information. double xmid = bm_dest.Width / 2.0; double ymid = bm_dest.Height / 2.0; double rmax = bm_dest.Width * 0.75; int ix_max = bm_src.Width - 2; int iy_max = bm_src.Height - 2; // Generate a result for each output pixel. double x0, y0; for (int y1 = 0; y1 < bm_dest.Height; y1++) { for (int x1 = 0; x1 < bm_dest.Width; x1++) { // Map back to the source image. MapPixel(warp_op, xmid, ymid, rmax, x1, y1, out x0, out y0); // Interpolate to get the result pixel's value. // Find the next smaller integral position. int ix0 = (int)x0; int iy0 = (int)y0; // See if this is out of bounds. if ((ix0 < 0) || (ix0 > ix_max) || (iy0 < 0) || (iy0 > iy_max)) { // The point is outside the image. Use white. bm_dest.SetPixel(x1, y1, 255, 255, 255, 255); } else { // The point lies within the image. // Calculate its value. double dx0 = x0 - ix0; double dy0 = y0 - iy0; double dx1 = 1 - dx0; double dy1 = 1 - dy0; // Get the colors of the surrounding pixels. byte r00, g00, b00, a00, r01, g01, b01, a01, r10, g10, b10, a10, r11, g11, b11, a11; bm_src.GetPixel(ix0, iy0, out r00, out g00, out b00, out a00); bm_src.GetPixel(ix0, iy0 + 1, out r01, out g01, out b01, out a01); bm_src.GetPixel(ix0 + 1, iy0, out r10, out g10, out b10, out a10); bm_src.GetPixel(ix0 + 1, iy0 + 1, out r11, out g11, out b11, out a11); // Compute the weighted average. int r = (int)( r00 * dx1 * dy1 + r01 * dx1 * dy0 + r10 * dx0 * dy1 + r11 * dx0 * dy0); int g = (int)( g00 * dx1 * dy1 + g01 * dx1 * dy0 + g10 * dx0 * dy1 + g11 * dx0 * dy0); int b = (int)( b00 * dx1 * dy1 + b01 * dx1 * dy0 + b10 * dx0 * dy1 + b11 * dx0 * dy0); int a = (int)( a00 * dx1 * dy1 + a01 * dx1 * dy0 + a10 * dx0 * dy1 + a11 * dx0 * dy0); bm_dest.SetPixel(x1, y1, (byte)r, (byte)g, (byte)b, (byte)a); } } } }

This method first calculates some values for the warping functions to use. It then makes the variables x1 and y1 loop over the pixels in the output image. For each output pixel (x1, y1), the code calls the MapPixel method to map that pixel back to an input position (x0, y0) where x0 and y0 are not necessarily integers. As you'll see shortly, MapPixel returns different pixels depending on which warp type is passed to it.

Next the code uses bilinear interpolation to pick a color for the output pixel. To do that, it calculates the distances dx0, dy0, dx1, and dy1 between the input position (x0, y0) and the integral pixel values nearest to x0 and y0. It then multiplies the color components of those nearest pixels by the distances to get a weighted average.

To see how this works, consider the picture on the right. The point (x0, y0) is the point in the input image that the output point mapped back to. The other points are the nearest pixels.

Now suppose that x0 is exactly halfway between ix0 and ix0 + 1. In that case, dx0 and dx1 are both 0.5. To calculate the color of the upper dashed point shown in the picture, you take the weighted average of the two upper points. In this case that would be [color of upper left pixel] * 0.5 + [color of upper right pixel] * 0.5. In this example, the upper left pixel is red and the upper right pixel is white so the result is pink.

The program calculates the weights as weight1 = 1 - dx0 = dx1 and weight2 = 1 - dx1 = dx0. In the picture, the point (x0, y0) is actually a bit closer to the right pixel, so the correct weights should be something more like 0.3 and 0.5, giving a brighter pink.

Similarly you can calculate that the color of the the bottom dashed point should be a light blue.

Finally you can interpolate between the two dashed points to determine that the actual point (x0, y0) should be a sort of purplish color. That is the color that the program assigns to the output pixel (x1, y1) in the result picture.

The only piece remaining for this example is the MapPixel method that maps an output pixel back to an input pixel. Because this post as gone rather long, I'll describe that method in my next post.

Download the example to experiment with it and to see additional details.