Warp images arbitrarily in C#, Part 1

[warp images]

This is an extension of the example Use image filters to perform edge detection, smoothing, embossing, and more in C# that adds new features that warp images in arbitrary ways.

The idea is to use two functions F(x, y) and G(x, y) to map the pixel locations (x, y) in an input image to new positions (F(x, y), G(x, y)) in the output image. Unfortunately you can’t warp images by simply mapping the input image’s pixels to the output image. If you do, some (most) of the resulting positions (F(x, y), G(x, y)) will not be at integer locations. You could try to solve that problem by rounding off the position to the nearest pixel, but that would probably not produce a smooth result. It would also mean that multiple pixels might be mapped to the same location and some locations might not be mapped by any pixels in the original image.

The solution is to map pixels in the output image back to the positions they should have come from in the input image. For output position (x1, y1), you get an input position (x0, y0) where x0 and y0 are not necessarily integers. You can then use a weighted average of the pixels surrounding (x1, y1) in the input image to determine the color of the output pixel.

This example adds the following warp type enumeration to the Bitmap32 class. (For information on that class, see the earlier example.)

// Warping types.
public enum WarpOperations

The Bitmap32 class also provides the following Warp method to use the warp types.

// Warp an image and return a new Bitmap32 holding the result.
public Bitmap32 Warp(WarpOperations warp_op, bool lock_result)
    // Make a copy of this Bitmap32.
    Bitmap32 result = this.Clone();

    // Lock both bitmaps.
    bool was_locked = this.IsLocked;

    // Warp the image.
    WarpImage(this, result, warp_op);

    // Unlock the bitmaps.
    if (!lock_result) result.UnlockBitmap();
    if (!was_locked) this.UnlockBitmap();

    // Return the result.
    return result;

To warp images, this code makes a copy of the input image to hold the resulting warped image and locks both images. It then calls the WarpImage method to do most of the work. It finishes by unlocking the images if appropriate and returning the result.

The following code shows the WarpImage method.

// Transform the image.
private static void WarpImage(Bitmap32 bm_src, Bitmap32 bm_dest,
    WarpOperations warp_op)
    // Calculate some image information.
    double xmid = bm_dest.Width / 2.0;
    double ymid = bm_dest.Height / 2.0;
    double rmax = bm_dest.Width * 0.75;

    int ix_max = bm_src.Width - 2;
    int iy_max = bm_src.Height - 2;

    // Generate a result for each output pixel.
    double x0, y0;
    for (int y1 = 0; y1 < bm_dest.Height; y1++)
        for (int x1 = 0; x1 < bm_dest.Width; x1++)
            // Map back to the source image.
            MapPixel(warp_op, xmid, ymid, rmax, x1, y1,
                out x0, out y0);

            // Interpolate to get the result pixel's value.
            // Find the next smaller integral position.
            int ix0 = (int)x0;
            int iy0 = (int)y0;

            // See if this is out of bounds.
            if ((ix0 < 0) || (ix0 > ix_max) ||
                (iy0 < 0) || (iy0 > iy_max))
                // The point is outside the image. Use white.
                bm_dest.SetPixel(x1, y1, 255, 255, 255, 255);
                // The point lies within the image.
                // Calculate its value.
                double dx0 = x0 - ix0;
                double dy0 = y0 - iy0;
                double dx1 = 1 - dx0;
                double dy1 = 1 - dy0;

                // Get the colors of the surrounding pixels.
                byte r00, g00, b00, a00, r01, g01, b01, a01,
                     r10, g10, b10, a10, r11, g11, b11, a11;
                bm_src.GetPixel(ix0, iy0,
                    out r00, out g00, out b00, out a00);
                bm_src.GetPixel(ix0, iy0 + 1,
                    out r01, out g01, out b01, out a01);
                bm_src.GetPixel(ix0 + 1, iy0,
                    out r10, out g10, out b10, out a10);
                bm_src.GetPixel(ix0 + 1, iy0 + 1,
                    out r11, out g11, out b11, out a11);

                // Compute the weighted average.
                int r = (int)(
                    r00 * dx1 * dy1 + r01 * dx1 * dy0 +
                    r10 * dx0 * dy1 + r11 * dx0 * dy0);
                int g = (int)(
                    g00 * dx1 * dy1 + g01 * dx1 * dy0 +
                    g10 * dx0 * dy1 + g11 * dx0 * dy0);
                int b = (int)(
                    b00 * dx1 * dy1 + b01 * dx1 * dy0 +
                    b10 * dx0 * dy1 + b11 * dx0 * dy0);
                int a = (int)(
                    a00 * dx1 * dy1 + a01 * dx1 * dy0 +
                    a10 * dx0 * dy1 + a11 * dx0 * dy0);
                bm_dest.SetPixel(x1, y1,
                    (byte)r, (byte)g, (byte)b, (byte)a);

This method first calculates some values for the warping functions to use. It then makes the variables x1 and y1 loop over the pixels in the output image. For each output pixel (x1, y1), the code calls the MapPixel method to map that pixel back to an input position (x0, y0) where x0 and y0 are not necessarily integers. As you’ll see shortly, MapPixel returns different pixels depending on which warp type is passed to it.

Next the code uses bilinear interpolation to pick a color for the output pixel. To do that, it calculates the distances dx0, dy0, dx1, and dy1 between the input position (x0, y0) and the integral pixel values nearest to x0 and y0. It then multiplies the color components of those nearest pixels by the distances to get a weighted average.

[warp images]

To see how this works, consider the picture on the right. The point (x0, y0) is the point in the input image that the output point mapped back to. The other points are the nearest pixels.

Now suppose that x0 is exactly halfway between ix0 and ix0 + 1. In that case, dx0 and dx1 are both 0.5. To calculate the color of the upper dashed point shown in the picture, you take the weighted average of the two upper points. In this case that would be [color of upper left pixel] * 0.5 + [color of upper right pixel] * 0.5. In this example, the upper left pixel is red and the upper right pixel is white so the result is pink.

The program calculates the weights as weight1 = 1 - dx0 = dx1 and weight2 = 1 - dx1 = dx0. In the picture, the point (x0, y0) is actually a bit closer to the right pixel, so the correct weights should be something more like 0.3 and 0.5, giving a brighter pink.

Similarly you can calculate that the color of the the bottom dashed point should be a light blue.

Finally you can interpolate between the two dashed points to determine that the actual point (x0, y0) should be a sort of purplish color. That is the color that the program assigns to the output pixel (x1, y1) in the result picture.

The only piece remaining for this example is the MapPixel method that maps an output pixel back to an input pixel. Because this post as gone rather long, I’ll describe that method in my next post.

Download Example   Follow me on Twitter   RSS feed   Donate

This entry was posted in algorithms, graphics, image processing, mathematics and tagged , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

5 Responses to Warp images arbitrarily in C#, Part 1

  1. Kyle M says:

    Outstanding. I’ve acutally been searching for a simple walkthrough like this, and this is perfect.

  2. Pingback: Warp images arbitrarily in C#, Part 2 - C# HelperC# Helper

  3. Pingback: Use DrawImage to warp images in C# - C# HelperC# Helper

  4. Hi Rod – Just wanted to say thanks from a long time fan. I have had your VB Graphics Programming on my book shelf as a reference for years. I just read your explanation of bilinear interpolation above and it is by far the best I have found. The graphic you supplied with it is worth 1,000 words. Do you have something similar covering bicubic interpolation? Would you use this same approach to combine reducing the size of an image and converting it to grayscale in one pass?

    Thanks again!

    • RodStephens says:

      Thanks for the kind words! It’s always nice to hear that someone is getting something from my books. Post a review when you have a chance!

      I haven’t implemented bicubic interpolation. (Sorry I don’t have an example for you.) It uses the same basic idea as bilinear interpolation, but instead of using the four points surrounding the target point, it uses 16 points. The points farther away model the rate of change (slope) of the color values, so the method tends to smooth out variations in color. Three’s a good overview on Wikipedia at:

      Bicubic interpolation

      The picture on that post’s upper right is a bicubic version of the picture in this post. Basically the method makes a two-dimensional spline in color-space to approximate the missing pixels’ colors.

      I probably wouldn’t combine resizing and color conversion into a single operation. It would be more complicated, and bicubic interpolation is already complicated enough. It would also reduce your flexibility. For example, you might want to change the way you convert to grayscale. You might want to use averaging, weighted averages of color components, sepia tone, or some other method. See this post: Use an ImageAttributes object to apply general color tones to an image in C#

      On the other side, you might want to try bilinear interpolation to see which produces a better result. And you might want to try other filters such as edge detectors and low pass and high pass filters.

      It’s easier to keep all of those separate rather than merging them so you can use different combinations.

Comments are closed.