Documents

1998_feb_115-125.pdf

Categories
Published
of 11
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Description
Mosaicking of Ort horectified Aerial Images Yehuda Afek and Ariel Brand Abstract Aerial photographs are widely used i n surveying, geographic information systems (GIS), and other applications. Analysis of a large area requires the creation of an image mosaic, which i s composed of several aerial photographs. In an ideal situa- tion, a perfect mosaic can be generated using a series of rigid transformations on the source images. In practice, geo- metric dist
Transcript
  Mosaicking of Ort horectified erial Images Yehuda fek and riel Brand bstract Aerial photographs are widely used in surveying geographic information systems GIS), and other applications. Analysis of a large area requires the creation of an image mosaic which is composed of several aerial photographs. In an ideal situa- tion a perfect mosaic can be generated using a series of rigid transformations on the source images. In practice geo- metric distortions and radiometric differences interfere with the mosaicking process. In this paper a complete algorithm to mosaic images taken at different times and conditions with geometric distor- tions and radiometric differences is presented. The algo- rithm which works without any human intervention inte- grates global feature matching algorithms into the process of selecting a seam line. The algorithm may be applied to mo- saic any set of images for which an appropriate matching al- gorithm exists. The creation of an image mosaic is accomplished using local transformations along a computed seam line and a rigid transformation elsewhere. An automatic stereo match- ing algorithm srcinally developed for surface height mea- surement is used to detect matching pairs of tie points across frame boundaries. These tie points are used to com- pute the seam line for the mosaic and to compute geometric and radiometric correcting transformations around this seam line. Introduction Mosaicking is the combination of several image frames into n image mosaic covering a large area. Such mosaics can be used, among other applications, for map making Moik, 1980). Aerial photographs are a common source for creating photo mosaics. When transformed to any local coordinate system e.g., UTM) and then mosaicked, the result is an or- thophoto map sheet. The transformation of the photograph to a local coordinate system Moik, 1980) is also called ortho- rectification. Such a transformation attempts to remove the geometric distortions that exist in a conventional aerial pho- tograph, caused, for example, by distortions in the optical system and by the perspective projection. The transformation depends on a priori knowledge about the optical system, the camera position and attitude at the time of photography, and the surface elevation at each point visible in the photograph. This a priori knowledge is usually known with limited accuracy. In particular, the surface ele- vation is usually available only at grid points, with interpola- tion being used for points between grid points. These inaccu- racies cause distortions in the final orthorectified aerial photograph. These distortions cause features in the map plane to appear at incorrect locations with respect to their true geographic location. The geometric distortions added during the orthorectifi- cation process vary for different photographs, for example, due to the use of different cameras and due to inaccuracies Computer Science Department, Tel-Aviv University, Israel 69978. PE RS February 1998 in the ground elevation data used to correct the perspective projection. Therefore, the same feature appearing in two dif- ferent photographs might appear in different map plane coor- dinates in the two orthorectified images created from both photographs. This distortion prevents the simple mosaicking of the two orthorectified images by a simple two-dimensional rigid transformation. In addition to the geometric distortions, radiometric dif- ferences between adjacent photographs must also be handled to create a seamless mosaic. These differences are caused by sun-angle-dependent shadows; seasonal changes of fields, forests, and water bodies; different atmospheric conditions; and variations during film development. In mosaicking two images, most systems define a seam line. The seam line goes across the overlap area between tie points, which are points that correspond to the same set of features in the two overlapping images. Along this line the images are tailored together by locally matching them geo- metrically and radiometrically. Common photogrammetric software for performing image mosaicking requires that a hu- man operator identify pairs of tie points along the operator selected seam line, that is, pairs of points in the two images that represent the same feature. The main contribution of this paper is in the integration of global feature-matching algorithms into the process of au- tomatic seam-line selection. In existing digital photogram- metric systems, the process of identifying matching tie points and selecting the seam line is done manually, sometimes with the aid of a local matching algorithm. In this paper see Figure I), the system first performs a global matching algo- rithm to automatically identify many potential pairs of tie points in the overlap area, and then proceeds to select a seam line using a subset of these points. This process has several advantages over manual or semi-automated systems: The seam line is selected here from a large set of matched potential tie points. This enables us to use sophisticated tech- niques to select the line along pairs of points while taking into consideration the amount of geometric and radiometric distortion. The seam line thus selected is both locally appro- priate for tailoring the two images and minimizes the accu- mulated distortion along the line. Because we use a global matching algorithm to find all the potential tie points in one shot, the computational overhead of the process is minimized, compared to the overhead intro- duced by local matching used in semi-automated systems. This is because in semi-automated systems the local match- ing starts from scratch several times on neighboring territo- ries. Several tools from stereo photogrammetrylpattern match- ing and graph theory are used in the above process. For the global matching algorithm we have adapted the algorithm of Brookshire et al. 1990). The algorithm for selecting the seam Photogrammetric Engineering & Remote Sensing, Vol. 64, No. 2, February 1998, pp. 115-125. 0099-1112I98I6402-115 3.00/0 1998 American Society for Photogrammetry and Remote Sensing  1. Global layout Overlap Region and Image 2 _ Coarse Seam L~ne Overlap Region and Tie Po~nts 2. Tie points selection Image 2 . I Image The Seam L~nes 3 Seam line setting Image 2 4. Geometric correction 7 Mean Seam Line - _ ar n one Triangulat~on 5. Radiometric correction 6. Merge corrected images Clipped Image - - L Clipped Image Figure 1 Algorithm overview. ebruary 998 PE RS  line in the network of potential tie points is an iterative ex- tension of Dijkstra s shortest path algorithm (Even, 1979). This paper completely automates the creation of the im- age mosaic, including the definition of a seam line based on tie points extracted using a matching algorithm, and the per- formance of geometric and radiometric corrections, yielding a seamless mosaic. Related Work The subject of creating an image mosaic was srcinally man- aged using photomechanical devices (Mullen, 1980). Creating a mosaic in this manner is labor intensive and can t handle local distortions in the different images. Non-digital mosaick- ing is still in use (Vickers, 1993). The trend toward digital photogrammetry (Boniface, 1992) required the development of digital mosaicking algorithms. Such algorithms were de- veloped both for photographic images (Zobrist et al. 1983) and for synthetic aperture radar images (Wive11 et al. 1993; Schultz et a]. 1989). The photographic image mosaicking algorithms address two different problems: smoothing geometric and radiometric discontinuities in adjacent images, and choosing the best im- age source at each point. The latter problem is caused, e.g., by different cloud coverage in each of the images (Nakayama and Tanaka, 1990). In this paper, we address the problem of smoothing geo- metric and radiometric discontinuities in adjacent images. One type of mosaicking algorithm assumes that no geometric or radiometric corrections are needed. These algorithms merely select a seam line that yields the best results for the rigid transformation mosaicking procedure (Hummer-Miller, 1989; Shiren et a]. 1989). Zobrist et a1. (1983) performed geometric and radiometric corrections along seam lines se- lected by the operator. Automatic correlation enhanced the detection of tie points along the seam line. This algorithm was used for orthophoto production (Hood et al. 1989). An- other algorithm (Albertz t al. 1992) added contrast correc- tion to the traditional intensity correction. In this paper, tie points are first extracted in the images overlap region, and then the seam line is selected by an au- tomatic procedure. This avoids the need for manually select- ing the seam line, and enables the use of automatic matching algorithms that are not restricted to work on pre-defined search windows. Such matching algorithms were developed for use in various applications (Perlant and McKeown, 1990). This enhancement yields a seam line that contains many known tie points and, thus, improves the final quality of the resulting mosaic. The Model Aerial photographs give a two-dimensional representation of the three-dimensional world. This representation is based on a perspective projection. In a perspective projection the im- age scale, that is, the ratio between size in the two-dimen- sional image and the three-dimensional world, is not constant. This prevents the correct measurement of real distances and angles using the aerial photograph. Cartographic maps give a two-dimensional representa- tion of the three-dimensional world that have a constant scale. It is thus possible to measure distances and angles on the cartographic map. Orthorectification is the process that transforms an aerial photograph from a perspective projec- tion to a cartographic map projection. The result of this pro- cess is an orthorectified image. Every digital orthorectified image is a geocoded raster image file. The geocoding information is a transformation from the image pixels to the cartographic map-plane coordi- nates. In this paper, only gray-level images are used. Real image data versus empty regions of the image file are explic- itly expressed using a bounding polygon, with vertices given in the map-plane coordinates. The geocoding transformation includes translation and scaling. Resampling must be used for images of different scales. lgorithm Overview The mosaic algorithm is based on six major stages, as de- picted in Figure 1: 1) Global Layout. The first stage handles the global elements. It identifies the overlapping region of the two input images. An intersection of the images bounding polygons yields this overlap, and the geocoding information (assignment of map- plane coordinates to pixels) enables the transformation of polygonal lines to the image pixels. In addition, the first stage also sketches a rough line in whose neighborhood the final seam line will be located, and, for each image, the im- age side o the seam line to be kept in the mosaic is identi- fied. 2) Tie Point Selection. The second stage extracts the tie points. This is done using a matching algorithm. For this paper, a stereo matching algorithm, srcinally designed for the mea- surement of surface height based on image parallax, is used (Brookshire et al., 1990). That algorithm was srcinally de- signed to manage aerial photographs. The selection of tie points need not create a regular grid, nor be of some spe- cific density. 3) Seam Line Setting. The third stage performs the exact selec- tion of the seam line. This line is actually a pair of polylines, each representing a polyline in one of the input images. These polylines should run along corresponding objects and features in the two images. The selection is done using iter- ations of a Dijkstra algorithm for finding the shortest path between the images tie points in a weighted graph. The weights of edges connecting tie points factor in several parameters that effect the quality of the mosaic. These param- eters include the distance between tie points, the distance to the coarse seam line, and the relative deformation be- tween successive pairs of points. (4) Geometric Correction. The fourth stage performs the geomet- ric correction. The final coordinates of each seam line point are computed as the average of its two counterpart tie points coordinates. This results in three seam lines, one in each image and the one along which the two images are pasted together. In the correction process some area along the seam line is stretched (or shrunk) to bring its seam line to the pasting line. To perform this correction, a margin line is first built in each image parallel to the seam line and at a predetermined distance from the seam line. As the spatial quality of the image deteriorates, a larger distance is necessary [in our im- plementation, this distance was set at 100 pixels). The area enclosed between the seam line and the margin line is the margin zone. This zone is next partitioned into triangular regions. Each triangle is given two sets of vertices, one set representing the triangle vertices before the correction, and the other after. A raster copy operation, based on the Fei- bush Levoy Cook algorithm (Foley et al., 1990), is then used to copy the imagery from the area covered by the triangle with the first set of vertices to the one with the second set of vertices. This is done by computing the pixels intensities in the output triangle using bilinear interpolation of the pix- els intensities in the input triangle. If the polylines of the seam line are reselected along the same features in the im- ages after the geometric correction, the two new polylines will have the exact same map-plane coordinates for all cor- responding pairs of points. This ensures the perfect match- ing of the two images when mosaicked along the pasting line. (5) Radiometric Correction. The fifth stage handles the radio- metric corrections necessary to create a seamless mosaic. The correction is again based on triangulating an interest area along the seam line as in the previous stage. For each triangle, radiometric correction parameters are computed PE RS February 1998  based on image gray values at tie point neighborhoods. ces connected by the edge. The graph and the selected coarse These parameters aim to bring both the images average gray seam line are demonstrated in Figure for two examples. level and standard deviation to equal values on both sides ~h, election of a path using the ~ijk~t~~ lgorithm ensures of the seam line. The gives both images a common the selection of the shortest possible coarse seam line. radiometric appearance along the seam line. (6) Merge Corrected Images. To complete the mosaic, all that is necessary is to build a new image file, i.e., he merged im- Tie Point Selection age of the two unchanged egions of the input images and The tie point selection algorithm used in this paper is based the two modified margin zones, This can now be accom- on an article by Brookshire et a]. (1990). The algorithm was plished using four copy operations, taking into considera- developed to solve the stereophotogrammetry problem by tion the geocoding information. Data from only one image measuring the image parallax of two stereo images and using will be taken for each side of the seam line, and the geo- this measure to compute the ground elevation. However, the metric correction stage ensures that there is no overlap algorithm is more general and solves the problem of match- along the seam line. ing two images with relative distortions in arbitrary direc- tions. Other matching algorithms, such as that by Medioni lobal Layout and Nevatia (1985) or by Weng et al. (1992), may also be Each input image contains the actual raster data, the geocod- used with no modifications required at other stages of the ing information, and a bounding polygon (given in map- mosaicking algorithm. plane coordinates). This last polygon specifies the separation The algorithm refines the disparity map by matching of the image pixels between those that have a null value, or corresponding topological features at successively finer reso- otherwise contain irrelevant data, and those that actually lutions as shown in Figure 3 First, the srcinal images are contain photographic data. reduced to the desired pseudo-hex resolution. Then an edge- To enable the mosaicking of two input images, the mu- vector operator (Bowker, 1974) is applied to produce a full tual location of the two images on the map projection plane edge-vector field. Next, the threshold and association operators must be determined. This information is easily extracted thin the vector field to its most prominent edges. The node from the two bounding polygons. The images mosaic will operator is then applied to both associated edge-vector fields. cover the area bounded by the union of the two images For each node in image 1 he matching algorithm searches a bounding polygons. The actual union can be computed by small hexagonal window in image for candidate matching the Weiler polygon clipping algorithm (Foley et al. 1990). nodes and correlates the local edge-vector fields to find a Apart from the general region in which the mosaic will match. The disparity network from the matches is then filled take place, a general path for the seam line must also be in by a spatial interpolation, expanded to the next resolution computed. Later on, this coarse selection will be refined in size, and then filled again by interpolation. These steps are the final selection of the seam line. Usually, the task of se- repeated at pseudo-hex resolutions 54, 18, 6, and 2. At the lecting a coarse path for the seam line is better done by a hu- srcinal gray-scale level a modified, normalized cross-correla- man operator. This is due to the need to inspect the images tion (Duda and Hart, 1973) on small windows provides the in many aspects, such as cloud coverage, blur versus sharp final disparity network. In principal, the matching points image sections, dense areas versus open spaces, and so on. found can be further refined to sub-pixel precision using an All these are hard to account for in an automatic procedure, algorithm such as that by Lyvers et al. (1989). and were not handled in this paper. The actual selection of a coarse path for the seam line is Seam Line Setting done by selecting a line stretching between the intersection The tie points identified during the matching stage are scat- points of the input images bounding polygons. Such a line tered all over the images. The next task is to build a seam can be computed by defining a graph representing the global line for the mosaic based on these tie points. The seam line layout of the images and applying the Dijkstra algorithm end points were found during the global layout stage, but the (Even, 1979). The graph vertices include the intersection specific path must yet be selected. points of the images bounding polygons, as well as all verti- Selecting a seam line is actually the selection of a set of ces of the bounding polygons. The edges of this graph will tie points. The polyline running between these points will include all line segments between any two vertices that are serve as a seam line. Geographically the lines selected for entirely within the images overlap region. both images are equal, but in the map projection plane they The overlap region can again be computed using the differ due to the geometric distortions. Our aim is to select Weiler polygon clipping algorithm. The weight of each edge such a set of tie points that will enable good application of is the Euclidean length of the line segment between the verti- the geometric correction. The Coarse Seam Line \ image lmage Example 9 he Coarse Seam L~ne vertex of th graph @ bounding polygons intersection points Figure 2. The selection of the coarse seam line. February 1998 PE RS
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks