Image Stitching

Read the full writeup.

Read the writeup for part 1 only.

As a bit of a toy problem, we can rectify portions of an image by selecting the four corners of an item, marking their respective destinations, computing the homography matrix over the four pairs of matching points, and finally applying said matrix to the image as a whole. The example below has been rectified over the rightmost set of doors.

Of course, it doesn’t have to be a rectangular target. We can warp into any shape we please, which means that we can easily replace certain regions of an image with other images’ content. Take this collection of Hogwarts portraits, for instance:

We can replace two of the portrait subjects with a video of Relno’s Fox through the following process:

- Extract frames from the video using OpenCV.

- Split the frames into two parts (one for each portrait).

- Mark the positions of each of the two portraits' corners.

- Warp and mask the target images into the portrait image.

- Combine the resulting frames into a video.

For the main act, we stitch overlapping images into panoramas. This really just comes down to identifying corresponding feature points within pairs of images (either through manual selection or an automatic pipeline), warping images into each others’ coordinate systems, and then blending pairs of images together in succession.

We can do this automatically (i.e. given only a list of filepaths) by selectively warping only the images for which a sufficient number of matching points can be detected. The below panorama is the result of a fully automatic mosaicing process.

To round it off, we’re able to create 360° panoramas through a similar (albeit simpler) cylindrical warping procedure.