Typical data sources for reconstructing 3-D building models in- clude optical images (airborne or spaceborne), airborne LiDAR point clouds, terrestrial LiDAR point clouds and close range im- ages. In addition to them, recent advances in very high resolution synthetic aperture radar imaging together with its key attributes such as self-illumination and all-weather capability, have also at- tracted attention of many remote sensing analysts in character- izing urban objects such as buildings. However, SAR projects a 3-D scene onto two native coordinates i.e., “range” and “az- imuth”. In order to fully localize a point in 3-D, advanced in- terferometric SAR (InSAR) techniques are required that process stack(s) of complex-valued SAR images to retrieve the lost third dimension (i.e., the “elevation” coordinate). Among other InSAR methods, SAR tomography (TomoSAR) is the ultimate way of 3- D SAR imaging. By exploiting stack(s) of SAR images taken from slightly different positions, it builds up a synthetic aperture in elevation that enables the retrieval of precise 3-D position of dominant scatterers within one azimuth-range SAR image pixel. TomoSAR processing of very high resolution data of urban ar- eas provided by modern satellites (e.g., TerraSAR-X, TanDEM-X
LiDAR points of buildings are extracted from the scene using previously available 2D building bound- ary polygons. Nearby pointsfrom terrain and vegetation are removed using filtering procedures. For roof segmentation, a robust region growing technique is developed. A unique feature of the segmentation method is the growing of triangles of a Triangulated Irregular Network (TIN) instead of LiDAR points. This minimizes the gaps between segments, because LiDAR points at segment intersections can be assigned multiple segment labels. Additionally, robust adaptive thresholds are introduced as region growing criteria. These enable the region growing procedure to stop at weak edges, while also segmenting non-planar roof segments. Results show that the proposed segmentation outperforms other methods concerning undersegmentation, and that it recognizes even weak edges. Evaluation and an extensive analysis of the input parameters’ effects on the results have shown that the segmentation is very robust against LiDAR point cloud characteristics and segment shape. Segment boundaries are cretated by collapsing the convex hull of segment points. Point density variations in across-track and along-track directions are considered in the collapsing procedure. For building modeling, the 2.5D dual contouring approach of Zhou and Neumann  is adapted to model complex roofs. After overlying a 2D grid to the segmented point cloud, vertices of the 3D buildingmodel are estimated for each grid cell by minimizing a Quadratic Error Function (QEF). Each QEF minimization results in a hyperpoint, which consists of one or more vertices of the buildingmodel at the same x-y-coordinates. This 2.5D-characteristic enables the connection of building ver- tices at step edges with vertical walls. In contrast to Zhou and Neumann , the proposed method
220.127.116.11.4 Edge-Based Technique Edge-detection technique is a technique to detect edge pix- els using Gradient, Laplacian or Canny filtering and then link those pixels to form contours at the end. Linking of edges, in a predefined neighborhood, depends on two criteria. The first one is the magnitude of the gradient and the second is the direction of the gradient vector. Since edges are important features in an image to separate regions, a large variety of edge detection algo- rithms have been developed for image segmentation in computer vision area (Shapiro u. Stockman ). (Heath u. a. ) demonstrate a proposed experimental strategy by comparing four well- known edge detectors: Canny, Nalwa–Binford, Sarkar–Boyer, and Sobel. (Jiang u. Bunke ) presented a novel edge detection algorithm for range images based on a scan line approximation technique. LiDAR data are converted into range image, e.g. DSM (Digital Surface Model) to make it suitable to image edge-detection methods. The performance of segmentation is largely dependent on the edge detector. However, the operation of converting 3D point clouds to 2.5D range images inevitably causes information loss. For airborne LiDAR data, the overlapping surface such as multi- layer building roofs, bridges, and tree branches on top of roofs cause buildings and bridges either under segmented or wrongly classified. The point clouds obtained by terrestrial LiDAR are usually combined from the scans in several different positions, converting such kind of true 3D data into 2.5D would cause great loss of information (Wang u. Shan ).
Recent advances in very high resolution synthetic aperture radar (SAR) imagery and its key attributes—self-illumination and all-weather capability—have attracted the attention of many remote sensing analysts in the characterization of urban environments. Various techniques have been developed that make use of SAR imagery for building detection and recon- struction. Complex building shapes surrounded by roads and other structures make building detection a challenging problem. One possible solution is to discriminate buildings from other objects using the building height and width measurements extracted from SAR imagery . The key issue is then the building height retrieval. For this purpose, various methods have been developed, including using sound electromagnetic models , layover  or shadow analysis  and simulation- based methods . In , an approach particularly suited for the detection and extraction of large buildings based on information acquired from interferometric SAR (InSAR) data is proposed. Stochastic model-based and low level feature-based approaches for extracting and reconstructing buildings from a single SAR intensity image are presented in  and , respectively. Wang et al.  presented an approach for build- ing extraction from high-resolution single-aspect polarimetric SAR data. Since, in urban areas, the structures are densely packed, the appearance of one particular building is dependent on the viewing angle of the sensor. Using a single-view SAR image, it is difficult to detect buildings that have no orientation component in the sensor’s azimuth direction . To overcome this limit, multiview SAR acquisitions are required. In , an approach for estimating building dimensions using multi- view SAR images is presented. Bolter and Leberl  and Thiele et al.  proposed methods for buildingreconstruction based on multiview InSAR data. Buildingreconstruction in context to stereoscopic SAR radargrammetric and multiview polarimetric SAR acquisitions has also been used in  and , respectively.
A novel method for creating detailed building models with com- plex roof shapes from LiDAR point clouds is proposed in this paper. The 2.5D Dual Contouring method of Zhou and Neumann (2010) is used and adapted in a way that step edges and inter- section edges can be created between roof segments. A main contribution of this work is the modification and weighting of the Quadratic Error Function (QEF) for modeling step edges and intersection edges. The modeling depends on the step edge prob- abilities of local height layers. A prerequisite for adaptive 2.5D Dual Contouring is a roof segmentation technique which stops at smooth edges. The applied robust TIN-based region growing reli- ably stops at smooth edges. Consequently, undersegmentation is significantly reduced. The resulting building models show a very high fit to the input LiDAR points. Each roof segment is repre- sented by a triangulation, thus also non-planar roof shapes can be modelled. Subsequent model regularization is recommended, because buildings are represented by a large number of vertices. Errors in reconstruction result mostly from wrong or missing con- nections of the vertices. Thus, the way the connections of the ver- tices to the buildingmodel should be more robust. Wrong con- nections could be avoided by checking for the consistency of the model with the building footprint. Under assumption that build- ing edges are mostly orthogonal or parallel to the main build-
In this paper, we propose an approach for façade detection and reconstructionfrom such point clouds. Firstly, the façade region is extracted by thresholding the point density on the ground plane. The extracted façades points are then clustered into segments corresponding to individual façades by means of slope analysis. Surface (flat or curved) model parameters of the segmented building façades are further estimated. Finally, the elevation estimates of each raw TomoSAR point is refined by using its more accurate azimuth and range coordinates, and the corresponding reconstructed surface model of the façade. The proposed approach is illustrated and validated by examples using TomoSAR point clouds generated from a stack of 25 TerraSAR-X high spotlight images
With data provided by modern meter-resolution SAR sensors and advanced multi-pass interferometric techniques such as tomographic SAR inversion (TomoSAR), it is now possible to generate 4-D (space-time) point clouds of the illuminated area with point density of approx. 1 million points/km 2 . However, due to side looking geometry, these point clouds exhibit much higher density of points on building façades in contrast to nadir looking LiDAR geometry (typically used for object reconstruction). Moreover, temporally incoherent objects such as trees cannot be reconstructed from multi-pass spaceborne SAR image stacks and provide moderate 3-D positioning accuracy in the order of 1m as compared to airborne LiDAR systems (around 0.1m). Despite of these special considerations, object reconstructionfrom these high quality point clouds can greatly support the reconstruction of dynamic city models that could be potentially used to monitor and visualize the dynamics of urban infrastructure in very high level of details. Motivated by these chances, earlier approaches have been proposed to reconstruct building façades from this class of data. E.g., experimental results provided in (Zhu, 2014) and (Shahzad, 2014) over smaller and larger areas demonstrate that façade reconstruction is an appropriate first step to detect and reconstruct building shape when dense points on the façade are available. In particular, when data from multiple views e.g., from both ascending and descending orbits, are available, the full shape of buildings can be reconstructed using extracted façade points. However, there are cases when no or only few
There are different challenges relevant to 3D buildingmodel generation as well as an accuracy evaluation of the produced model which should be discussed. Firstly, the heights located in an outline of a building should be consistent. Some incon- sistencies come from noise produced during the SAR-optical dense matching process. Moreover, regarding shapes of roofs particularly inclined types, the location of points differs from the topmost toward the lowest part. Then, making a decision, which cluster of points are suitable for 3D reconstruction in the LOD 1 level is a challenge. In this research, we simply used median as a robust estimator against the large heights changes of reconstructed points located in a footprint of building.
errors caused by the incompleteness of data arising, for instance, from missed faces due to insufficient points, occlusion or vegetation clutter. It is said that having increased the point density of modern ALS, the data-driven approach allows to have a more accurate and robust result than that through the model-driven approach (Oude Elberink, 2008). A number of methods for doing the reconstruction, based on either model-driven or data-driven approaches using ALS data have been presented in the literature. When reviewing relevant literature, it is observed that the major problem is the efficient manipulation of the topology and roof primitives. From recent literature, it is seen that roof topology graphs (RTG) are widely used in both data-driven and model-driven approaches, especially for the efficient manipulation of topology and roof primitives (e.g. Verma et al., 2006; Milde et al., 2008; Oude Elberink and Vosselman, 2009), and in many cases, accurate results have been obtained. However, many unsolved problems need to be addressed within the processing chain of buildingreconstruction. This will be discussed in detail under Section 1.2.
In this Chapter a new method is proposed for generating 3D building models on different levels of detail (LOD). The proposed work flow is presented in Figure 6.1. The 3D models on differ- ent LOD follow the standard definition of the City Geography Markup Language (CityGML) described in (Kolbe et al., 2005). CityGML defines five LOD for multi-scale modeling: LOD0 – Regional model consisting of the 2.5D Digital Terrain Model (DTM), LOD1 – Building block model without roof structures, LOD2 – Buildingmodel including roof structures, LOD3 – Buildingmodel including architectural details, LOD4 – Buildingmodel including the interior. Algorithms for producing the first three levels of the LOD are explained in this chapter. Accord- ing to the above categorization, the first LOD corresponds to the digital terrain model (DTM). The non-ground regions are filtered using geodesic reconstruction to produce the DTM from LIDAR DSM (Arefi and Hahn, 2005; Arefi et al., 2007b). The LOD1 consists of a 3D representation of buildings using prismatic models, i.e., the building roof is approximated by a horizontal plane. Two techniques are implemented for the approximation of the detected building outline: hierarchical fitting of Minimum Bounding Rectangles and RANSAC-based straight line fitting and merging (Arefi et al., 2007a). For the third level of detail (LOD2), a projection-based approach is proposed resulting in a buildingmodel with roof structures. The algorithm is fast, because 2D data are analyzed instead of 3D data, i.e., lines are extracted rather than planes. The algorithm begins with extracting the building ridge lines thought to represent building parts. According to the location and orientation of each ridge line one para- metric model is generated. The models of the building parts are merged to form the overall buildingmodel.
Presegmentation typically classifies the point cloud into buildingpoints and other points, mainly terrain and vegetation in LiDAR point clouds. If 2D building footprints are avail- able beforehand, building point clouds can be directly extracted [Rau and Lin, 2011]. A popular way is ground filtering method, in which a Digital Terrain Model (DTM) is pro- duced by morphological filter operations [Morgan and Tempfli, 2000] [Zhang et al., 2003] [Pingel et al., 2013], then a height threshold is set on the DTM. Another approach is to fit planes to pointsclouds, and clustering points. The largest cluster is assumed to be ground [Verma et al., 2006]. [Lafarge and Mallet, 2012] defineed expectation values for buildings, vegetation, ground and clutter by combining different covariance-based measures and height information by energy optimization. [Dorninger and Pfeifer, 2008] extracted all planar regions of the scene using region growing method in feature space and group the extracted points to buildings with a mean-shift algorithm.
world coordinates enable the generation of high quality TomoSAR point clouds, containing not only the 3D positions of the scatterer location but also estimates of seasonal/temporal deformation, that are very attractive for generating 4-D city models from space. However there are some special considerations associated to these point clouds that are worth to mention : 1) TomoSAR point clouds deliver moderate 3D positioning accuracy on the order of 1 m; 2) few number of images and limited orbit spread render the location error of TomoSARpoints highly anisotropic, with an elevation error typically one or two orders of magnitude higher than in range and azimuth ; 3) Due to the coherent imaging nature, temporally incoherent objects such as trees cannot be reconstructed from multipass spaceborne SAR image stacks; and 4) TomoSAR point clouds possess much higher density of points on the building façades due to side looking SAR geometry enabling systematic reconstruction of buildings footprint via façade points analysis. As depicted over smaller and larger areas in  and , façade reconstruction turns out to be an appropriate first step to detect and reconstruct building shape from these point clouds when dense points on the façade are available. Especially, when data from multiple views e.g., from both ascending and descending orbits, are available, the
In this paper we propose a method which aims at simplifying the 3D reconstruction of the building blocks by decomposing the overall model into several smaller ones corresponding to each building part. A similar method has been already reported by the author (Arefi, 2009) for reconstruction of high resolution LI- DAR data. In this paper due to a lower quality of the DEM pro- duced by stereo matching of satellite data (Worldview–2) com- paring to the LIDAR data, an additional data source is employed. Accordingly, the Worldview orthorectified image is employed for a better extraction of the ridge lines. According to each ridge line a projection-based algorithm is employed to transfer the 3D points into 2D space by projecting the corresponding pixels of each building part onto a 2D plane that is defined based on the orientation of the ridge line. According to the type of the roof, a predefined 2D model is fitted to the data and in the next step, the 2D model in extended to 3D by analyzing the third dimen- sion of the points. A final model regarding the parametric roof structures of the building block is defined by merging all the indi- vidual models and employing some post processing refinements regarding the coinciding nodes and corners to shape the appro- priate model. Additionally prismatic models with flat roof are provided regarding to the remaining areas that are not contain- ing ridge lines. Finally, all parametric and prismatic models are merged to form a final 3D model of the building.
Roof model selection is the process of fitting models into point clouds and selecting the most appropriate modelfrom a library which minimize a cost function. Mass and Vosselman [ 46 ] proposed a new method for gable roof reconstruction by computing their parameters using the analysis of invariant moments of the 3-D point clouds of LiDAR data. The information on the roof type and shape parameters is computed by using the heights of the point clouds as weight functions in moment equations. Haala et al. [ 47 ] estimated roof plane parameters by segmenting the DSM of aerial images and by analyzing the surface normals and ground plane orientations of the segments. A similar approach was presented in Kada and McKinley [ 40 ] using LiDAR points of each cell. They determined the roof types according to the number of segments. Zheng et al. [ 23 ] also used a similar approach for roof type identification, in which the root mean square error (RMSE) between the DSM and corresponding pointsfrom the candidate roof model determines the quality of reconstruction. Poullis and You [ 48 ] computed the roof model parameters using a nonlinear bound-constraint minimization. During this optimization, a Gaussian mixture model (GMM) was used to detect and exclude outliers from the fitting plane, where the parameters of GMM were estimated using an expectation-maximization (EM) algorithm. Lafarge et al. [ 2 ] proposed a stochastic method for reconstructing 3-D building models from the DSM of satellite imagery (PLEIADES satellite data simulations with resolutions of 0.7 m). They used a Bayesian algorithm based on RJMCMC to decide the buildingmodel which best fitted the DSM data. Huang et al. [ 49 ] utilized generative statistical models to reconstruct 3-D building models from LiDAR data. The method finds the optimal combination of parameters by a stochastic search. Henn et al. [ 41 ] proposed a strategy for 3-D buildingreconstructionfrom a small number of LiDAR data points. This method estimates roof parameters by fitting the roof models and by estimating their parameters by M-estimator sample consensus (MSAC). It determines the most probable roof model by a support vector machines (SVM). Zheng and Weng [ 42 ] proposed a method based on LiDAR data and building footprints. They computed some morphological and physical parameters from a decomposed footprint. They then applied a decision tree-based classifier to these statistical features to classify the building footprints into seven roof types. According to the roof type, they calculated the roof model parameters based on the statistical moments of the points within the cells.
L-shape is the most common building outline appearing in TomoSAR point clouds. For each building segment, one or zero L-shape is detected. The detection is achieved by catching two interconnected line segment using gray-scale Hough transform . Each pixel in the Hough transform matrix represents a line. In our algorithm, the pixel with the highest amplitude is firstly extracted, representing the first line of the L-shape. The second one is the pixel with highest amplitude away from the first one larger than a certain angle. To reject irregular L-shapes, constraints have to be put on this angle, the minimum pixel amplitude, and the minimum length of the line segments. For example, Figure 5 is the L-shape detection result of the test area near the Berlin central station. The detected L- shapes are overlaid on the binary façade image. The end points of L-shapes are marked with red and yellow dots.
Suveg and Vosselman (2004) developed a method for 3D reconstruction of buildings by integrating 3D information from stereo images and a large scale GIS map. First a building on ground plan is subdivided into primitives. Optimum schemes of partitioning are determined by minimum description length criteria and a search tree. Then the building primitives are verified by a least square fitting algorithm. The approximated values of the fitting method are obtained from map and 3D information from the stereo images. Kada and McKinley (2009) proposed a method for decomposition of the footprint by cell decomposition. In this method, the buildings are partitioned into non-intersecting cells along its façade polygons using vertical planes. The roof shapes are determined by directions generated from LiDAR points. Then models of buildings blocks are reconstructed using a library of parameterized standard shapes of model. Arefi and Reinartz (2013) proposed a model-driven approach based on the analysis of the 3D points of DSM from satellite images in a 2D projection plane. In this method parametric models are generated through single ridgeline reconstruction and subsequent merging of all ridgelines for a building. The edge information is extracted from the orthorectified image.
Automatic generation of 3D building models is an essential pre- requisite in a wide variety of applications such as tourism, urban planning and automatic navigation. Although over the last decades, many approaches of building detection and reconstructionfrom 3D point clouds and high resolution aerial images have been reported. The fully 3D buildingreconstruction is still a challenging issue due to the complexity of urban scenes. There are basically two strategies for building roof reconstruction: bottom-up/data-driven and top- down/model-driven methods. The bottom-up methods (e.g. region growing (Rottensteiner and Briese, 2003), Hough transform (Vosselman and Dijkman, 2001), RANSAC (Tarsha- Kurdi et al., 2008)) extract roof planes and other geometrical information from the point clouds. For roof reconstruction, the corresponding planes are assembled and vertices, ridges and eaves are determined (Sohn and Huang, 2008). Sampath and Shan (2010) used a bottom-up approach to segment the LiDAR points to planar and non-planar planes using eigenvalues of the covariance matrix in a small neighborhood. Then, the normal vectors of planar points are clustered by fuzzy k-means clustering. Afterwards, an adjacency matrix is considered to obtain the breaklines and roof vertices of corresponding planes. This method is used for reconstruction of moderately complex buildings. Rottensteiner et al. (2005), presents an algorithm to delineate building roof boundaries from LIDAR data with high level of detail. In this method, roof planes are initially extracted
In order to validate our approach, we run the algorithm over TomoSAR point clouds generated from TerraSAR-X high spotlight images. Fig. 1(b) shows the result of applying SD estimation procedure. The two parameters r (radius of the neighborhood cylinder) and d are empirically set to 5m and 2m respectively according to the point density of the data set. One can observe that TH value influences the number of extracted façade points. Lower TH value results in higher completeness but lower correctness. To extract lower façades and to automate the procedure, the threshold TH is set to the maximum of SD histogram value. This includes not only the façade points but additionally also some nonfaçade points with relative high SD, e.g., roof points. To reject these pointsfrom the set of extracted points after SD thresholding, surface normals information is utilized. Fig. 1(c) shows the extracted façade points by retaining only those points having normals between ±15 degrees from the horizontal axis.
With improved sensor resolution and advanced multi-pass interferometric techniques such as SAR tomographic inversion (TomoSAR), it is now possible to reconstruct both shape and motion of urban infrastructures. These sophisticated techniques not only opens up new possibilities to monitor and visualize the dynamics of urban infrastructure in very high level of details but also allows us to take a step further towards generation of 4D (space-time) or even higher dimensional dynamic city models that can potentially incorporate temporal (motion) behaviour along with the 3D information. Motivated by these chances, this paper presents a post processing approach that systematically allows automatic reconstruction of building façades from 4D point cloud generated from tomographic SAR processing and put the particular focus on robust reconstruction of large areas. The approach is modular and consists of extracting facade points via point density estimation procedure based on directional window approach. Segmentation of facades into individual segments is then carried out using an unsupervised clustering procedure combining both the density-based clustering and the mean-shift algorithm. Subsequently, points of individual facade segments are identified as belonging to flat or curved surface and general 1st and 2nd order polynomials are used to model the facade geometry. Finally, intersection points of the adjacent façades describing the vertex points are determined to complete the reconstruction process. The proposed approach is illustrated and validated by examples using TomoSAR point clouds over the city of Las Vegas generated from a stack of TerraSAR- X high resolution spotlight images.
shaped footprint could be reconstructed. Also, the approach utilizes roof points in determining the complete shape of the buildings and therefore resolves problems, as mentioned in (Shahzad and Zhu, 2015a), related to the visibility of the façades mainly pointing towards the azimuth direction of the SAR sensor. However, few points still need to be addressed. For instance, the reconstruction accuracy is restricted due to less number of available points and data gaps in the TomoSAR point cloud. This could be improved by incorporating data from other viewing angles and/or adding more constraints such as parallelism or using a model based approaches based on a library of low level feature sets. Also, we have compared our results to the OSM data which is regularly updated but not yet fully complete. Therefore, a more accurate ground truth would be needed for assessing the exact performance of the approach. Nevertheless, this paper presents the first demonstration of automatic reconstruction of 2-D/3-D building footprints from this class of data. Moreover, the developed methods are not strictly applicable to TomoSAR point clouds only but are also applicable to work on unstructured 3-D point clouds generated from a different sensor with similar configuration (i.e., oblique geometry) with both low and high point densities. In the future, we will explore the potential of extending the algorithm towards generation of automatically reconstructed complete watertight prismatic (or polyhedral) 3-D/4-D building models from space.