The Role of Seams in High Resolution Image Mosaics

When discussing true orthophoto generation I made reference to the image mosaicking process. I thought I would touch on that more today, with an emphasis on high resolution imagery. One of the main challenges in mosaic production is ensuring the mosaic is seamless. That is, one cannot easily discern where the edges of the input images are. This can be challenging for a number of reasons. One of the most difficult aspects involves the input image geometry. Because the input images have different perspective centers, the geometry of surface objects will vary between images. For example, a tall building in the center of a frame image may not exhibit any building lean, but the same building in the next image will show noticeable lean. So the big challenge in the mosaic process is ensuring the seams between input images conceal any mismatches. While seams between the images can be automatically generated, a quality control check must be performed to ensure there are no issues. Seams usually need to be manually edited if there are any problems.

Here's an example: the image below shows a multi-story building with a seam cutting right through it. Because the angle of building lean is different in each images, it looks like a mess.

In this example, the seam has been edited so that it shifts north of the building. This is ok, but it also reveals that there is a lot of building lean in the image portions chosen for inclusion in the mosaic.

This example shows the same building with the seam edited the other way. Instead of diverting north of the building, I've moved it to the other side. You can see that this provides a better top-down perspective on the building, which will generally make for a better output mosaic.

Another typical problem area: bridge decks. The image below shows a seam cutting through a freeway overpass. It is easy to see that the edges are misaligned.

Below, the seam is edited to run parallel with the bridge instead of cutting through it.

Note that accuracy issues in the terrain model used to create the input orthophotos, such as not having accurate enough terrain or the existence of errors in the terrain, can also introduce "mismatches" between images. In the bridge example above I used a DEM where I modelled the terrain with reasonable accuracy (performing terrain editing in stereo), but I stopped short of modeling the bridge decks. This means that the image area of the ground is accurate, but there is some offset for the bridge. While the seam editing technique hides the error and creates a visually-appealing result, the fact remains that the ortho may be considered flawed because the seams in the mosaic simply mask the error - they do not eliminate it.

Upcoming ERDAS Webinar: Mosaicking

I'll be doing a webinar next week on mosaicking, which was just announced on the ERDAS web-site yesterday. The main focus will be planning considerations for frame photograph mosaicking, processing techniques (radiometry, image manipulation, seams, output considerations), and final mosaic product generation. This may be of interest to both LPS and ERDAS IMAGINE users (along with anyone generally interested in mosaicking), since I'll be using MosaicPro for the processing: an add-on module to both LPS and IMAGINE.

The registration page is here. We'll be hosting the webinar at both 3AM and 11AM EST, so please feel free to join us for either session!

An Overview of Leica Photogrammetry Suite 9.2

I mentioned on Friday that LPS 9.2 is now shipping, so I thought I would follow up with some of the features and benefits. I'm not going to go over every single item, since most of them are located here already.

The main release theme in 9.2 is a focus on robust performance. What does this mean? Well, there aren't tonnes of new features or modules, but we put major effort into improving existing components: in particular terrain processing, orthophoto generation, and mosaicking workflows.

The terrain improvements largely focused on the ATE (Automatic Terrain Extraction) and TE (Terrain Editor) modules. In ATE there is actually quite a bit of new functionality. In addition to rolling in "Adaptive ATE" capabilities and other enhancements in the LPS 9.1 service packs, we added "object filtering" capabilities. Object filtering is basically designed to remove surface features such as trees and buildings. Since ATE results can vary greatly depending on factors such as the sensor type, contrast, light conditions, scanning quality, and so forth, we also added several user-definable settings for object filtering. Depending on the project area, users can define the target object width/height for removal, as well as several other parameters such as slope threshold, smoothing, topographic type, and more. On the whole we've seen pretty decent results with this new option. Of course the results can vary greatly between datasets (e.g. removing a group of detached single family homes in relatively flat or gently rolling terrain is different from trying to remove a group of row-houses with pitched roofs in hilly terrain...), but on average filtering can reduce editing times by 30% or so when producing bare-earth terrain products.

The main new feature in the Terrain Editor (TE) is a new "Terrain Following Cursor" option, which is a great way to QC your terrain datasets. When enabled, the cursor follows the terrain surface, so you can easily see if you are off the ground and hence need to do some editing. Also in 9.2 there are some significant performance improvements when working with large (e.g. over a couple of million posts) LTF (Leica Terrain Format) TIN terrain datasets. Here's a screenshot of where to find the new icon for the Terrain Following Cursor:
This version of the Terrain Editor also features performance improvements when working with large (e.g. several million points or more) LTF files.

Leica MosaicPro improvements largely consisted of under-the-hood memory consumption cleanups that allow the tool to handle significantly larger projects. In addition, the dodging interface has been cleaned up, the feathering algorithm updated, and a long-requested "undo" button for seam editing has been added (hint: the "backspace" keyboard button will under vertex-by-vertex edits if you want to go back a couple but haven't "accepted" the overall seam edit). Another major improvement for MosaicPro is the ability to batch script the entire process and kick it off from a command line prompt. This enables a nice workflow for people that use the same radiometric settings across multiple projects - all you need to do is open up the application to edit your seams. After that you can save them as a file and then script the whole operation.

Something that isn't in the What's New document but is a major improvement concerns orthorectification speeds. The team spent some time on algorithm optimization and the result was a significant speedup in processing times when using LTF TINs as a terrain source. The exact improvement will vary with the size of the LTF, imagery, and so forth, in general we're seeing speedups of 25-30% or more. This can significantly reduce overall processing time for large projects.

That's all for now. There are a lot more enhancements and features (new sensor models, RPC triangulation improvements, the Terrain Prep Tool, and more...), but these are the main themes of the release! In a future post I will write about our improved support for medium-format digital camera workflows, along with the updated releases of PRO600 and ORIMA.