Thoughts on Google Building Maker and 3D Building Extraction

Earlier this week Google introduced the Google Building Maker, which it bills as a simple tool for creating buildings for Google Earth. I gave it a whirl earlier today and while it is a relatively simple toolset, the direction is impressive and has some broad-reaching implications for the future.

The Concept

The general concept is fairly straightforward: using nothing but a browser, you can create a digital 3D representation of building structures using a very basic set of tools. Target buildings are located from a top-down view, and then buildings can be digitized from a series of oblique photos. Several oblique photos can be used to ensure your measurements are accurate from various angles. When you're finished digitizing your target building, you can save it to the Google 3D Warehouse. Upon saving, textures from the various photos are automatically applied to the model. Once the model is uploaded to the 3D Warehouse, you may download it as a KMZ or Collada file.

The Workflow

1) The first step is to launch the Google Building Maker.

2) Once the Building Maker is launched, you have to select a city to extract a model from. At present there are a few dozen cities in the USA, Canada, Mexico, Europe and Japan available. Why can't you extract buildings from your hometown? Most likely because the entire system relies on oblique aerial (not satellite) photography for making 3D measurements. The reason more cities aren't available is because oblique aerial photography is not cheap to acquire: so it makes sense that the currently available cities are clustered in the more prosperous regions of the world. Think hyperlocal advertising...

3) Once you've chosen a city, you are provided with the ability to zoom into a location until you can see a placemark. Once you're at the right zoom level, you can select/drag/drop the placemark to choose the building you wish to model. One interesting aspect of this process is that, in the initial view, you are provided with a color coded graphic overlay of the particular city you want to create a model in. The blue area, typically in the urban core, shows where buildings already exist. The white area shows the areas where Google doesn't have 3D buildings but does have all the ingredients (read: oblique photography) to create them.

Legend: Blue = Buildings Already Exist, White: No Buildings


Selecting an individual building

4) After you have dropped a placemark on a building, hit the "Start Building" button on the left. This will switch your perspective from a top-down imagery view to an oblique perspective view. You'll also notice a series of modeling tools on the top right. These include add box, add gable, and add vertical freeform block. There is also a tool for toggling the snapping of points and lines, along with a series of tools for where to place the new block (e.g. above selected, below selected, inline with selected, and unconstrained height). The tools are very rudimentary. For example, it isn't possible to model curves. However, I believe the idea for now is to collect the basic building structure. If it is an easy rectangular building then no further modeling is required, and if it is a complex structure you have the option to "refine your building" in Google SketchUp after hitting the save button. This isn't a particularly slick workflow, but it is fantastic step forward for users without access to photogrammetric tools or other methods for making measurement from photographs.

Perspective View Prior to Building Extraction

Extracted Building

5) Once you've saved your model to the Google 3D Warehouse, you may download it as a KMZ or Collada file. This is a win-win scenario that provides both you and Google with access to your model.

Extracted building in the Google 3D Warehouse

Photo-textured building displayed in Google Earth

The Implications: What Does This Mean???


Personally I think there are a lot of implications for this technology on a number of levels:

• Photogrammetry Software Vendors: can Building Maker replace proprietary COTS software for 3D feature extraction, such as solutions offered by ERDAS, BAE, and Intergraph (among many others)? In the short-term, no. The current Building Maker tools are simplistic and just scrape the surface in terms of functionality (e.g. no curves, no parallel lines, no attribution, etc). In the long term: Google is certainly setting the stage for a consumer-level system that may one day provide a robust system for 3D urban modeling. In that sense the Building Maker is a very disruptive move by Google.
• Crowdsourcing: An interesting aspect of the system is that Google denotes the areas they already have buildings for during the selection process. The implicit message for users is: collect buildings that we don't already have (although in fairness there are no restrictions on collecting existing buildings - but why would you?). The other item of note here is that the areas that have oblique coverage that do NOT have buildings already tend to be suburban areas outside the core that consist of low-rise buildings. I can't help but think that this is a clever idea by Google to acquire buildings for free in these areas rather than partner with a professional services company to do the job. But to give Google due credit, they provide users with access to their models. This means that, as a user, I now can use Building Maker to create as many 3D models as I want and then keep the output. And furthermore, I can do this without buying stereo imagery and the software required to perform stereo feature extraction - which can be a significant sum even when considering a small area. As an individual consumer, I now have access to a measurement toolset that was previously only available to professional businesses that had made the requisite investments in both the data and software packages...
• Is there a partnership between Google and an oblique vendor? Considering that Pictometry is the leading oblique vendor, I can't help but wonder if there is a partnership afoot. One thing to note here is that the oblique imagery is "Copyright 2009 Google".
• Can this lead to "browser-based photogrammetry"? In recent years I've been a proponent of developing systems that hide the photogrammetry and offer easy-to-use tools to enable 3D geoinformation solutions. I find it quite compelling that this is what Google has come to the plate with: a solution that anyone can use (consumer grade) that makes measurement from aerial photography very easy, and then provides value-added capabilities such as photo-texturing. While 3D feature extraction is only one aspect of photogrammetry, I believe it is a sign of things to come that a giant such as Google has already come to market with a functional solution that many vendors have only been thinking of...
• 3D measurements from mono imagery were commercialized initially (to my knowledge) by Geotango, which was subsequently acquired by Microsoft. This highlights the nascent competition between Google and Microsoft in this space.
• As mentioned in the Google Earth Blog, the big limitation is the fact that you have to choose from a list of available cities. One can only wonder what this will do for the oblique airborne photography market...

At any rate, kudos to Google for once again changing the game!

3D GIS Seminar

Just a quick note to say that the Bentley Seminar on "Bentley 3D GIS" is now available here. You need to register for both their site and the seminar, but it is quick and painless. The main focus on the seminar is (a) the need for 3D GIS for infrastructure and other applications and, (b) Bentley's tools for integrating GIS, CAD, and BIM data for city modeling applications. This focused on Bentley Map as a primary tool for data integration, modeling, and visualization. CityGML export has been added, which may make Bentley Map one of the earliest systems from a major vendor to support the standard.

It's great to see more people thinking about 3D cities beyond just the physical models. It will be interesting to see who will be the first to market with a system that integrates model creation and collection (e.g. collect buildings in stereo, from LIDAR, or automated techniques) along with urban information management.

Exploring Stereo 3D Feature Extraction (Part 2)

While the previous post introduced the concept of 3D stereo feature extraction, this post will focus more on specific tools for 3D vector capture.

For urban 3D modeling, manual stereo feature extraction typically involves collecting rooftops (and sometimes even more detailed structures such as rooftop mechanical features) as 3D polygon features. To create solid 3D models, an extra step is taken to extrude the feature down to the ground. There are typically numerous methods for doing this. In PRO600, an ERDAS application that runs in both the Bentley Microstation and PowerMap environments, there are three options for building extrusion via the “Create Building” operation:

• Manually measure the ground height.
• Specify a fixed object height.
• Extend the feature down to the surface of a digital elevation model.

If you have a DEM available, clearly this is the least time-consuming option - because you can extrude a large number of buildings all at the same time. There are also a number of options to consider when performing the extrusion: see below for Object Creation options (click on the image to enlarge):

Create Building and Object Creation Preferences in PRO600

To demonstrate extrusion and object creation, here is a screen capture of a set of extracted building features. These have all been extruded with the aforementioned “Create Building” operation.

Extracted 3D Building Vectors Displayed in Bentley PowerMap

Here is the same feature dataset with an orthophoto backdrop. Notice how the building polygons fall around the base of the building (the true location) and not the tops of the buildings. This is one of the advantages over heads-up digitizing directly from orthophotos. If the building has any lean at all, then tracing rooftop vectors off the ortho will produce building polygons in the wrong location.

Building Vectors and ECW Orthophoto Displayed in Bentley PowerMap

Switching to an isometric view and altering the display mode, you can see the buildings rendered from a perspective view.

Perspective View of 3D Building Vectors in Bentley PowerMap

While extruding features is one thing, the process of capturing them is another. PRO600 features an extensible library catalog containing feature definitions. While a default catalog comes with the software, it can be completely modified or replaced if necessary. Double-clicking on a feature in the catalog shows the feature attributes (color, description, style, etc.) which can then be edited and saved. Collected features can also be selected an modified on an individual basis. For example, you may have a selection set of buildings that you would like to highlight with a different line thickness and color to make them stand out from the rest of the building vectors in the dataset.

PRO600 Library Catalog and Shape Definitions

While the "Building" feature above triggers off the "Collect Squared" Microstation/PowerMap command, it is also possible to switch into arc-collection mode mid-feature to model any curving or circular lines. An increasing number of building structures feature curving corners instead of sharp edges, making tools like this a necessity.

Modeling complex feature can take time, so snapping is also important. PRO600 has a few different snapping options, including 2D, 3D, and a 2D/3D hybrid - where the tool switches between 2D and 3D based on a user-defined tolerance. Here is a look at the snapping preferences, which also show optional modifications such as the ability to cut a vertex or add a new vertex (to the feature being snapped to).

PRO600 Snapping Preferences

Aside from building polygon features, there are also specific tools available for mapping linear features such as roads. For example, a parallel line tool is available to reduce the time required to digitize both sides of a road or freeway feature. Since photogrammetric mapping has been widely used in the engineering community (e.g. State-level Departments of Transportation in the USA) a number of tools have emerged that are particular to high-accuracy, high-throughput mapping applications. Fortunately, these tools also lend themselves to a broad number of other applications, including visualization, 3D city modeling, urban/environmental planning, and much more.

Exploring Stereo 3D Feature Extraction (Part 1)

Manual operator-driven 2D and 3D feature extraction techniques have in use for many years. One of the early methods of getting data into a GIS was from 2D digitization. In addition to tablet digitizing systems, softcopy photogrammetry approaches to vector data have also been available for many years.

Remember this? Click the image for details

The photogrammetric approach, also known as stereoscopic feature extraction, is one step closer to the source of the data than GIS-based digitizing. Instead of compiling feature data from an orthophoto or another source of information, stereo extraction uses the original images along with the triangulation metadata. The triangulation metadata consists of exterior orientation parameters that are required for stereo pair generation. Exterior orientation data could also come from airborne GPS/IMU data – which may be necessary for mapping remote areas where collecting ground control points is not an option.

Stereo feature extraction is inherently 3D. The vector data are measured in X, Y, and Z by manipulating a floating cursor while viewing a stereo pair of left and right images. As a result, the point, line, and polygon feature data all have Z values associated with each vertex. Because GIS developed in a 2D paradigm, many early stereo feature extraction packages were coupled with CAD environments as a platform. Basically a stereo viewer would be added for stereo image viewing and 3D measurement, and the feature data would render in the application's native “viewer”. In more recent times applications have been developed for GIS packages in addition to CAD environments, as well as a number of native applications that do not require platform technology.

Stereo Feature Extraction in PRO600

So why is this important? Data used for GIS analysis needs to come from somewhere. While there are a number of methods used for generating vector data, the demand for 3D vector data appears to be on the rise. One area of current discussion is the format and environment for generating and persisting 3D vectors. Currently the end application tends to determine the format, but this can be challenging for people trying to create one-size-fits-all data products (e.g. if I want to create a textured 3D city model, what format should I use without being locked into one system?). In this respect the development of CityGML is intriguing, as the geometric properties are but one aspect of urban information modeling and are viewed as an inter-related component of a much broader system. But persistence models aside, tools are still required for the original content generation.

More on specific 3D feature extraction tools tomorrow...

3D PDF Generation

I thought I would write a "how-to" post after reading some recent coverage on 3D PDFs. There are a few different software packages that can generate 3D PDFs, but I will cover the workflow using LPS, PRO600 Fundamentals, and Bentley PowerMap. The actual 3D PDF generation is done with PowerMap (also supported in MicroStation). Aside from photogrammetric mapping it is also possible to build a model from guesstimating building dimensions or designing a model from scratch.

To produce the 3D PDF below, I performed the photogrammetric pre-processing in LPS and then extracted 3D building vectors in PRO600 Fundamentals (which runs on top of Bentley PowerMap) via stereo feature extraction. After that, the steps are:

1) From PowerMap set the View extent to the area to be exported.
2) Choose File > Print. This opens the dialog below.
3) Ensure "Plot to 3D" is checked.
4) Choose Settings > 3D Plotting: you may want to change some settings such as the background color and a number of advanced settings.
5) Choose File > Print (from within the Print menu) when you're ready to save out the 3D PDF.



Click on the image below to download the 3D PDF I generated. I didn't get into any sophisticated modeling, but the tools within PowerMap and MicroStation will allow for creation of highly realistic models.



This post also gives me a chance to try out Scribd: check out the 2D PDF I exported from PowerMap as well, with imagery as a backdrop. Personally I like the zoom and pan functionality as well as full-screen mode: a nice alternative to viewing in Acrobat...
Los Angeles Ortho and Vectors See here for some more 3D PDF examples from the Bentley MicroStation V8i page.

3D Buildings and Terrain in Virtual Worlds

One the the problems I've grappled with in the past is the challenge posed by modeling accurate photogrammetrically extracted 3D buildings in virtual worlds with a less accurate base terrain layer. The folks over at the Bluesky blog have outlined the same challenge in this post. Here's an example of the problem: when I import my KML 3D buildings into Google Earth, they float off the ground due to the accuracy of the base terrain layer. Here's a screen capture depicting the problem:
Even if you modify the terrain exaggeration the buildings still float. There's a few different methods for resolving this. One method suggested in the Bluesky blog post is to model terrain and imagery right into the KML file and then load it up in GE. This certainly works. Another method would be to take a look at a different system for visualizing the data.

The latest update to ERDAS TITAN has a good technique for sorting out this problem: it allows you to specify your own terrain layer. Once you add your data as layers in the viewer, you'll see a similar problem as depicted above - again due to the accuracy of the default terrain layer. The only difference is that instead of seeing floating buildings the buildings are embedded under the terrain layer, so that some of the smaller buildings are not visible. Here's a screen capture that displays the issue:

You can see that some of the buildings in the foreground appear very "flat", and there are others that are not even visible.

The solution is to add a terrain layer. In this case I processed my own digitial orthomosaic along with a terrain layer during the photogrammetric processing part of the project. I added the digital elevation models as layer, right-clicked on it and chose the "use as terrain" option.
The layer gets shifted to the terrain folder and the TITAN viewer updates to reflect the new base terrain. The result is that the buildings sit perfectly on top of the terrain. Here is the result, from the same perspective as the screen capture above:

In summary, take a look at TITAN if you're interested in this kind of workflow. The ERDAS TITAN Client is free, so you can download it from the ERDAS site and give it a whirl with your data.

3D City Modeling

I'll be hosting a webinar on December 9th entitled: "3D City Construction Workflows: From Collection to Presentation". This will be an opportunity for us to explore workflows for planning projects and developing 3D building content data, which can then be fed into various GIS and mapping applications. We'll also take a look at data presentation: methods for visualizing and sharing the data.

3D city modeling is a rather nebulous topic, and there are many ways to create 3D city models. Here are a few different options:

- Automatically extruding 2D building polygons digitized from an ortho, either from a rooftop Z value or guesstimating the height. This is the "quick and dirty" method.
- Manually model each building individually and tag it with an XY location (often guesstimating the spatial dimensions). These models may be based on ground photos, oblique imagery, or a combination.
- Photogrammetrically measure and extrude buildings.
- Photogrammetrically measure and extrue buildings, and then model further detail and add elements such as photo-texture.

The webinar will primarily focus on the 3rd option: using photogrammetric tools to measure buildings. In addition to the actual extraction process, other discussion points such as data capture, data quality, accuracy, and other topics will be outlined as well.

If you're a GIS/photogrammetry/mapping professional, please tune in to one of the scheduled webinars. You can sign up here.

DigitalGlobe on NBC: A Closer Look at 3D Olympics

I've seen a couple of posts regarding DigitalGlobe's Bejing Oympic Games coverage. In my opinion this is a great consumer-grade marriage of geospatial technology and mainstream media. On the DG homepage "DigitalGlobe on NBC" is prominently displayed, which takes you to http://www.digitalglobe-aegistg.com/, their site with EAgis Technologies (a joint effort between DG, EAgis, and NBC).

The posts above comment on the technological underpinnings on the 3D scenes offered on the new site. One common theme is how a combination of different technologies (photogrammetry, 3D modeling, satellite imagery, 3D visualization) can be used together to provide a powerful and immersive viewing experience.

The http://www.digitalglobe-aegistg.com/ site allows visitors to download 3D PDFs, a KMZ and perspective views of various Olympic sites. 3D PDFs have been around for awhile, but this is the first time I've had a chance to examine one: very cool, although not the same experience as a KML file in a virtual world. Compare for yourself below:

KML in Google Earth:

3D PDF:
Aside from the availability of 3D example data, the site also provides some insight into the creation of the dataset. The "How Can This Be Possible?" heading expands to provide a high-level introduction to the technology and workflow. The workflow is divided up into four parts with a graphic associated with each one: the "3D wireframe" generation, imagery capture, feature extraction and extrusion, and fully textured 3D model generation.

"3D Wireframe" Generation

The site mentions that the wireframe represents the earth's terrain, and that it was derived from two DG satellite images. Sounds like classical photogrammetry! Two images associated with sensor model can be viewed in stereo to extract (measure) 3D positions (e.g. points with an accurate XYZ location). For high-accuracy applications relying on the satellite sensor model may not be enough, and there would be a need to collect and measure ground control points and then run through the triangulation process. However, once that is done there are numerous applications that may be used for terrain extraction.

The screen capture of the wireframe is actually a Triangulated Irregular Network (TIN), which when compared to raster DEMs is a more efficient means of modeling terrain. These can be automatically correlated using point matching algorithms or manually compiled by hand - which can be a very time-consuming process.

Why is the TIN important? The terrain represents a fundamental part of an immersive 3D scene. If it isn't accurate then the scene will not look realistic... In inaccurate terrain model could also cause problems in the image processing (orthorectification) part of the workflow.

It is also important to note that terrain can come from a number of sources: manual compilation, automatic correlation, LIDAR, IFSAR, and other sources.

Imagery Capture

This screen capture shows imagery draped over the terrain. The imagery would have come from QuickBird or WorldView-1 satellites. For a good-looking scene high-resolution satellite imagery or aerial photography is important. Sometimes satellite imagery is useful, but often aerial photography is the best solution. Why? If imagery is captured from a sensor mounted on a plane, the data acquisition organization has full control over the scale/resolution of the photography. Flying low equals higher resolution...

Another important note is that the terrain model discussed in the previous step would likely be used to orthorectify the image. This will result in a geometrically accurate orthophoto with real-world coordinates. The accuracy of the terrain is important: if there are large errors the 3D features discussed in the next step may not appear in the correct position if they were extracted in a stereo feature extraction system (building footprints digitized off the orthos and then extruded would be ok though).

Feature Extraction and Extrusion

The text for this segment talks about "special tools" being used to determine "footprints" of buildings and then extruding them. This might work for some rectangular buildings with flat roofs, but it is clear that all the download-able content on the site was not derived from automatic extrusion. There are a couple of ways to generate 3D buildings. The quick an dirty way (extrusion) involves digitizing the building footprints in 2D from a digital orthophoto. Then you need to tag the building polygon with an attribute to represent height. This is a fairly straightforward procedure if a digital surface model (DSM) of the area is available. The drawback of extrusion is that, although quick, it may not be accurate. Extrusion assigns one elevation value for the entire building (roof) area, so buildings with pitched or complex roof structures will not be modeled accurately.

Photogrammetric feature extraction can model buildings with greater detail, since specific building detail can be modeled in stereo by viewing and measuring buildings in 3D. However, photogrammetric feature extraction is performed from a "top-down" perspective, so features like balconies may be difficult to model. This is where CAD or CAD-like 3D modeling packages and ground-based photography can help. One workflow for 3D city construction is to photogrammetrically extract the buildings and then import them into a CAD package to add more detail to the models. Ground photos can also provide photo-realistic image texture, as can aerial photography, but capturing all four sides of a building can be difficult without planning the acquisition flight with a very high degree of overlap - which can add to the project cost (more fuel, more data to process). In addition, aerial photography may not be able to capture street-level image texture or areas with dense skyscrapers.

At any rate, there are many ways to go about generating the 3D buildings - it all depends on the level of detail required and the project budget...

Textured 3D Model Generation

As I mentioned above, texture can be applied to buildings from both ground and aerial photography. There's a number of tools that can be used to texture the buildings, here is an example video of how this can be done in SketchUp. There's a number of 3D modeling applications out there to do this sort of work. Again, production costs rise an accordance with the level of detail applied to a building. A "perfect" building cannot be easily automated and can be laborious to produce in sophisticated packages such as Autodesk's 3ds Max.

Looking at the Beijing Institute of Technology model it is clear that a lot of effort went into building it. Not only does the texture look great, but there is a lot of 3D modeling that has been done in a professional 3D modeling system. The rounded rooftop would be very difficult to model in a photogrammetric feature extraction system, and the model contains detail of the roof overhangs - which would likely come from the use of ground photography.

At any rate it is nice to see this technology getting some mainstream media coverage. Photogrammetry and 3D mapping have been around for a long time, but the mass-market popularity of visualization packages such as Google Earth is exposing this technology to a much broader audience.

3D City Building Case Study

While high-fidelity 3D city models have been around for 10+ years, they have been getting a lot of attention in recent years with the advent of Google Earth and Virtual Earth. I've touched a little bit on 3D building extraction and texturing in a previous post, but recently came across an excellent case study from Magnasoft entitled "Build Virtual Cities to Plan Real Cities".

The case study provides an overview of start to finish textured city construction. The main input data consists of stereo pairs, and the software involved includes LPS, MicroStation, and ArcGIS. The final products go well beyond just buildings: they also include roads, tree canopies, water bodies, electric poles, walls, and fences. You can see from the stereo feature compilation screen shot (on part 5) that complex building structures can be modeled in high detail.

Just to explain the process in a bit more detail: MicroStation is important because LPS' main 3D feature extraction application, PRO600, is an MDL application that hooks in the LPS stereo viewer and allows users to extract 3D objects in stereo within the MicroStation environment. After the building intelligence is added in ArcGIS, the models are brought into Stereo Analyst for IMAGINE (another 3D extraction tool, which is an add-on to ERDAS IMAGINE), where a feature called the Texel Mapper can be used to automatically texture the buildings from the "block" of aerial photographs. I'll expand more on this process in a future post...