Our general motivation is to have an automatic alignment scheme, that is one which needs no user input. To accomplish this, we first need to construct an error function that estimates how well a given alignment of images matches the images in their region of overlap. As in the paper, we associate each image with a rotational mosaic, that is warping matrix that transforms the image into an image on the unit sphere in the current computed direction. Assuming a significant overlap between the two warped images being considered (say at least 25% of the area of an image), the error function is then computed as the sum of squared differences (SSD) between the intensities of overlapping pixels in each warped image.

Unfortunately SSD minimization is a difficult problem because it would be too time-consuming to test every possible transformation (unless you do something clever e.g. [Gleicher97]). The approach we've taken is to first apply a Taylor approximation of the differences in image intensities on the update parameter on the warping matrix. The details of the mathematics are given in the paper, but we end up using Cholesky decomposition to solve a system of linear equations with a special property.

Several improvements to the process are implemented. We speed up the process of computing the coefficients of the linear equations by a path-based alignment process which assumes that the Jacobian is constant over a small pixel region. To improve the stability of the numerical computation, instead of using general warping matrices as past literature has done, the novel approach taken by the authors is to only allow 3-D rotation matrices, which makes sense in the the specific context of image alignment. This requires knowing the focal length, which can be estimated using the general warping technique, or simply be given by the user. Lately, the authors of the paper have designed an algorithm to improve the estimate of the focal length by updating the current focal length as the computation of mosaic proceeds.

The authors have designed numerous additional algorithms to improve the alignment process since the SIGGRAPH'97 paper this project was based on. For instance, a global alignment process is designed to align all of the images in the panoramic sequence at the same time, thus imrpoving the global quality of the resulting mosaic substantially. Also, a local alignment process is designed to avoid artifacts such as "ghosting" in which the images are globally aligned well but certain small features are noticebly misaligned resulting in seeing a ghost image of the feature in the final image mosaic. Essentailly, an automatic feature matching technique on a local scale is introduced that alters the global rotation matrix slightly towards a better match for each pixel region. In conclusion, the authors have made substantial progress towards automatic panoramic image mosaic creation.