DigitalGlobe on NBC: A Closer Look at 3D Olympics

I've seen a couple of posts regarding DigitalGlobe's Bejing Oympic Games coverage. In my opinion this is a great consumer-grade marriage of geospatial technology and mainstream media. On the DG homepage "DigitalGlobe on NBC" is prominently displayed, which takes you to http://www.digitalglobe-aegistg.com/, their site with EAgis Technologies (a joint effort between DG, EAgis, and NBC).

The posts above comment on the technological underpinnings on the 3D scenes offered on the new site. One common theme is how a combination of different technologies (photogrammetry, 3D modeling, satellite imagery, 3D visualization) can be used together to provide a powerful and immersive viewing experience.

The http://www.digitalglobe-aegistg.com/ site allows visitors to download 3D PDFs, a KMZ and perspective views of various Olympic sites. 3D PDFs have been around for awhile, but this is the first time I've had a chance to examine one: very cool, although not the same experience as a KML file in a virtual world. Compare for yourself below:

KML in Google Earth:

3D PDF:
Aside from the availability of 3D example data, the site also provides some insight into the creation of the dataset. The "How Can This Be Possible?" heading expands to provide a high-level introduction to the technology and workflow. The workflow is divided up into four parts with a graphic associated with each one: the "3D wireframe" generation, imagery capture, feature extraction and extrusion, and fully textured 3D model generation.

"3D Wireframe" Generation

The site mentions that the wireframe represents the earth's terrain, and that it was derived from two DG satellite images. Sounds like classical photogrammetry! Two images associated with sensor model can be viewed in stereo to extract (measure) 3D positions (e.g. points with an accurate XYZ location). For high-accuracy applications relying on the satellite sensor model may not be enough, and there would be a need to collect and measure ground control points and then run through the triangulation process. However, once that is done there are numerous applications that may be used for terrain extraction.

The screen capture of the wireframe is actually a Triangulated Irregular Network (TIN), which when compared to raster DEMs is a more efficient means of modeling terrain. These can be automatically correlated using point matching algorithms or manually compiled by hand - which can be a very time-consuming process.

Why is the TIN important? The terrain represents a fundamental part of an immersive 3D scene. If it isn't accurate then the scene will not look realistic... In inaccurate terrain model could also cause problems in the image processing (orthorectification) part of the workflow.

It is also important to note that terrain can come from a number of sources: manual compilation, automatic correlation, LIDAR, IFSAR, and other sources.

Imagery Capture

This screen capture shows imagery draped over the terrain. The imagery would have come from QuickBird or WorldView-1 satellites. For a good-looking scene high-resolution satellite imagery or aerial photography is important. Sometimes satellite imagery is useful, but often aerial photography is the best solution. Why? If imagery is captured from a sensor mounted on a plane, the data acquisition organization has full control over the scale/resolution of the photography. Flying low equals higher resolution...

Another important note is that the terrain model discussed in the previous step would likely be used to orthorectify the image. This will result in a geometrically accurate orthophoto with real-world coordinates. The accuracy of the terrain is important: if there are large errors the 3D features discussed in the next step may not appear in the correct position if they were extracted in a stereo feature extraction system (building footprints digitized off the orthos and then extruded would be ok though).

Feature Extraction and Extrusion

The text for this segment talks about "special tools" being used to determine "footprints" of buildings and then extruding them. This might work for some rectangular buildings with flat roofs, but it is clear that all the download-able content on the site was not derived from automatic extrusion. There are a couple of ways to generate 3D buildings. The quick an dirty way (extrusion) involves digitizing the building footprints in 2D from a digital orthophoto. Then you need to tag the building polygon with an attribute to represent height. This is a fairly straightforward procedure if a digital surface model (DSM) of the area is available. The drawback of extrusion is that, although quick, it may not be accurate. Extrusion assigns one elevation value for the entire building (roof) area, so buildings with pitched or complex roof structures will not be modeled accurately.

Photogrammetric feature extraction can model buildings with greater detail, since specific building detail can be modeled in stereo by viewing and measuring buildings in 3D. However, photogrammetric feature extraction is performed from a "top-down" perspective, so features like balconies may be difficult to model. This is where CAD or CAD-like 3D modeling packages and ground-based photography can help. One workflow for 3D city construction is to photogrammetrically extract the buildings and then import them into a CAD package to add more detail to the models. Ground photos can also provide photo-realistic image texture, as can aerial photography, but capturing all four sides of a building can be difficult without planning the acquisition flight with a very high degree of overlap - which can add to the project cost (more fuel, more data to process). In addition, aerial photography may not be able to capture street-level image texture or areas with dense skyscrapers.

At any rate, there are many ways to go about generating the 3D buildings - it all depends on the level of detail required and the project budget...

Textured 3D Model Generation

As I mentioned above, texture can be applied to buildings from both ground and aerial photography. There's a number of tools that can be used to texture the buildings, here is an example video of how this can be done in SketchUp. There's a number of 3D modeling applications out there to do this sort of work. Again, production costs rise an accordance with the level of detail applied to a building. A "perfect" building cannot be easily automated and can be laborious to produce in sophisticated packages such as Autodesk's 3ds Max.

Looking at the Beijing Institute of Technology model it is clear that a lot of effort went into building it. Not only does the texture look great, but there is a lot of 3D modeling that has been done in a professional 3D modeling system. The rounded rooftop would be very difficult to model in a photogrammetric feature extraction system, and the model contains detail of the roof overhangs - which would likely come from the use of ground photography.

At any rate it is nice to see this technology getting some mainstream media coverage. Photogrammetry and 3D mapping have been around for a long time, but the mass-market popularity of visualization packages such as Google Earth is exposing this technology to a much broader audience.

LPS 9.3 Preview: KML Export

We're getting late in the release cycle for the upcoming LPS 9.3 release (the beta testing phase has now started!) so I thought I'd start previewing some of the new functionality we're releasing.

An increasing number of geospatial applications are supporting KML (although the word "support" can mean a lot of things), so mentioning that we'll be able to export the LPS Block File as a KML file isn't earth-shattering news. However, KML in the context of photogrammetric applications is relatively new and there are some interesting implications.

First I'll show how the exporter works and then get into what some of the uses are. Here's a screen capture of a small photogrammetric project in the LPS Project Manager, in area of Waldkirch, Switzerland.As you can see it is a relatively "complete" project. There are triangulated images, GCPs and Tie Points, some DTMs, and orthophotos. From the Project Manager, we have a new drop down entry in the "Tools" section called "Export to KML". Click on this and the following dialog appears.The dialog allows you to choose which elements of the photogrammetric project (Block File) you would like to export. Check the various boxes and then you can hit the "Export" button to generate the KML file.

For this dataset I've uploaded the output KML file here. Feel free to download it and check it out. Note that the various photogrammetric data elements (e.g. Ground Control Points) can be turned on and off. Here is a screen capture of the file in Google Earth.
So this brings us to the question, why is this relevant? The first thing that comes to mind is project tracking and status reporting. Photogrammetric/mapping projects are increasingly completed in disparate geographic areas. This can make project tracking a challenge. While there's a mixed-bag of current approaches to project tracking, a KML file can provide a relatively compact (especially if you leave out the tie points) and visual representation of what parts of the project are complete. For example, an organization with an office in the USA that is working with a partner in another part of the world could request daily updates of status for a large digital ortho project. By looking at the "orthos" layer, the project coordinator could not only see how many are complete (like they may currently do with MS Excel or other spreadsheet apps) but also see a visual of the completed project areas. Thus, they could see if the "challenging" parts of the project had been tackled yet (e.g. rugged terrain or urban areas) and manage accordingly.

I'll talk about this a bit more in future posts, as well as hightlight some of the other solutions we've been working on this year. We're certainly looking forward to getting the new release out!

Upcoming ERDAS Webinar: Mosaicking

I'll be doing a webinar next week on mosaicking, which was just announced on the ERDAS web-site yesterday. The main focus will be planning considerations for frame photograph mosaicking, processing techniques (radiometry, image manipulation, seams, output considerations), and final mosaic product generation. This may be of interest to both LPS and ERDAS IMAGINE users (along with anyone generally interested in mosaicking), since I'll be using MosaicPro for the processing: an add-on module to both LPS and IMAGINE.

The registration page is here. We'll be hosting the webinar at both 3AM and 11AM EST, so please feel free to join us for either session!

Photogrammetry at the Acropolis

After a few weeks offline I'm now back and writing from Liege, Belgium. During my time off I had the opportunity to visit the Acropolis in Athens, Greece. While walking up to the Parthenon I noticed there was a terrestrial laser scanner set-up and operational - although unfortunately I didn't get any photos. But that was enough to get me wondering what the project was about. At the top of the Acropolis I found a sign with a short description of the project (photos below). Since it is difficult to read I have reproduced the text below:

DATA ACQUISITION FOR THE PHOTOGRAMMETRIC RECORDING OF THE ACROPOLIS

The Acropolis Restoration Service carries out the project of geometric documentation of the Acropolis hill, the circuit Wall and the Erechtheion, using photogrammetric methods together with 3-dimensional scanning.

All the information to emerge is to be entered in a Geographic Information System (G.I.S) that will be available through the Acropolis Restoration Service's web site (ysma.culture.gr).

Photogrammetry at the Acropolis was also a subject of discussion at the recent ISPRS Conference in Beijing. One of the technical sessions (TS-SS19) was "Recording and Documenting the Acropolis of Athens - From Classical Ancient Greece to Modern Olympics". While I wasn't at the conference, a colleague sent me the paper for "Recording, Modeling, Visualisation and GIS Applications Development for the Acropolis of Athens", by Tsingas et al. The paper discusses the various techniques employed by the project outlined above, which include geodetic field measurements, terrestrial scanning, and photogrammetric data capture and processing. Of the many data products to come out of the project, an interesting one is a top-view orthomosaic with a 10mm resolution. A 22MP camera was used on a balloon system, as motorized vehicles such as helicopters are not permitted to fly above the Acropolis. Also of interest (and news to me) is that Leica Geosystems is a partner in the project. One of the terrestrial scanners is a Leica HD3000, while ERDAS LPS is used for parts of the photogrammetric processing. This included camera calibration, bundle adjustment, and terrain processing.

The paper describes the methodology in detail, and I will see if it is available online anywhere - it provides an excellent discussion of various techniques used in concert to fully capture a highly detailed digital version of the monument. A few other good papers on photogrammetry/mapping at the Acropolis are here and here.

Short Hiatus

I'll be traveling (and generally away from computers) for the next five weeks so the lights will be out at The Fiducial Mark for the next little while. I'll be back in action by late July and will be reporting from a different location: see below for details!

In the meantime, check out e-planet for the latest posts from some other folks at ERDAS.

Leica Geosystems Chinese Earthquake Press Release

This press release was sent out nearly a week ago. I wanted to highlight it since I haven't seen much about it anywhere else and I think it is a great example of how rapid response applications can save lives (and hey, there's photogrammetry involved too). I won't rehash the story here since you can read about it in the press release, but I would like to highlight a few key points.

We hear a lot about how aerial photography is used for rapid response mapping, but an important consideration to the type of airborne sensor employed for such applications. On reflection, the ADS40 is a great camera for this sort of thing. Why? The fact that it is a pushbroom sensor offers a great advantage. This means it collects a "pixel carpet" instead of the typical 4Kx4K or 9Kx9K frame camera. This is critical for rapid mapping applications because the image analyst (for example, the person who saw the SOS in the press release image) doesn't have to waste time loading frames or performing a large mosaicking job with a lot of seams. They can either review the strip directly or run a quick and dirty mosaicking job that doesn't require much in terms of seam edits.

Make sure you check out the image associated with the press release (link below). It is fairly high-res when you zoom in and the earthquake damage is noticeable. You can also see from the bottom of the image that the screen capture was taken from ERDAS IMAGINE.


At any rate, I will write a Sensor Spotlight on the ADS40 in the future and highlight some of the other applications it is suited for..

Cell Arrays in LPS

In LPS we use a spreadsheet like tool called a “Cell Array” for managing and manipulating data in several parts of the software package. The first place you see a Cell Array is when you load a project in the LPS Project Manager: it is the list at the bottom with one row per image. Generally Cell Arrays are used for different purposes in the various places they are embedded, so I will highlight a few of them here.

In the LPS Manager, the Cell Array lists all the images and also provides an image-by-image status. There are status columns (either red for incomplete or green for complete) for Pyramid Layer, Interior Orientation, Exterior Orientation, if there is a DTM associated with the image, an Ortho associated with the image, and whether the image is “online”. That is, does the image reside where the blockfile has referenced it. There is also an “Active” column that flags each particular image for inclusion in various processing operations (e.g. running pyramid generation or APM on just the active images). From the Cell Array you have the ability to select images, perform a criteria selection, and delete the selection (remove images from the project).

One of the powerful capabilities is to use the criteria selection plus the “formula” option to activate or deactivate images. A trick here is to use the criteria selection to select a group (e.g. Image ID > 30) and then activate or deactivate your selection group using the “formula” Cell Array option. Once you have a selection set, right click on the “Active” column heading and choose the “Formula” option. Type in “1” and hit Apply to activate all the images. Likewise you can type in “0” and hit apply to deactivate all the selected images (see the screenshot below).

Another place with a Cell Array is the Frame Editor. When you are in “Edit All Images” mode, the image, sensor name, orientation parameters and other information are all made available in the cell array. This is a useful place for setting the image orientation status. For example, if you want to import orientation parameters from GPS/IMU data you can cut and paste the six parameters in, set the status to “fixed” and immediately view the images in stereo (assuming there aren’t any issues with the orientation parameters). It is also possible to Import and Export data (among other things) by right clicking on the column heading. For example, orientation parameters can be exported by selecting the images, highlighting the headings for the orientation parameters, then right-clicking on the column heading and choosing "export". See the second screenshot below for an example.

A Cell Array is included in the LPS Stereo Point Measurement tool as well. This Cell Array can be useful for managing tie and control points. By saying “managing”, I mean performing tasks such as:

• Importing GCP points
• Exporting tie points
• Defining the point status
• Activating and Deactivating points
• Deleting points

For example, if you ran Automatic Point Measurement and then proceeded through triangulation, you can export your tie points as XYZ data and then use that as the basis for a “quick and dirty” surface model. Automatic Terrain Extraction is a better choice (where you can use tie and control points as seed data), but at least this allows you some flexibility.