Detecting Buildings from Satellite Imagery with Keras
By Philip Harman
The purpose of this project was to utilize machine learning tools to build a model capable of detecting buildings in satellite imagery.
For the notebooks and datasets used, check out my GitHub.
Shown: Austin, Texas (RGB and Labelled Set)
​
Use Cases
Building recognition from satellite imagery has a variety of applications, including the following real-life examples:
-
Investment tools: Tracking urban growth to project the demand for construction materials.
-
Census data: Combining geospatial analysis with less costly & more accurate “micro-censuses” to create disaggregated population estimates in Afghanistan.
-
Public health resource allocation: Mapping rural settlements in Nigeria to aid in vaccine distribution.
​
​
Data Sources and Feature Engineering
The satellite imagery and labeled data set that I used were originally obtained from the Landsat Program and Open Street Maps (respectively). The dataset and labels were packaged using Quilt for a similar project by Jared Yamaoka, which I was able to download using the documentation provided here.
​
Landsat-8 provides nine bands in total. In addition to these, I calculated the NDVI and Moisture index of the imagery (calculations shown to the right).
​
The last step was to normalize, resize, and sharpen the bands/rasters above to match the resolution of the labelled dataset (from 30m to 15m). I did this using the Panchromatic band, with an approach proposed in J. Yamaoka’s work.
​
​
After stacking up these engineered features, I began reviewing each band individually and comparing the results to the labelled data set and the original RGB image. My goal was to see which features made which pictured objects stand out the most (i.e. buildings, roads, vegetation, bare land, and bodies of water). Some key findings were:
-
NDVI: Nicely distinguished vegetation and buildings, but was biased towards water and bare land.
-
IR: Improved bare land recognition, but was still biased towards water. Poor contrast between vegetation and buildings.
-
Moisture: Excellent for recognizing grasslands, and reduced bias towards water. Poor recognition of buildings.
​
​
I initially removed the Red, Green, Blue, and Coastal features due to concerns about collinearity between these bands (as shown below). I later experimented by adding these features back into the model, but I didn’t find that they improved performance. For this reason, I chose to move forward with the IR, NDVI, and Moisture Index features for the final model.
​
​
SMOTE, Model Selection and Results
First drafts of the model had a bias towards labelling points as "not building", as a consequence of an unbalanced dataset. To resolve this, I applied SMOTE oversampling to train on a slightly more balanced dataset. I found the best results with a balance of 33% "building" / 67% "not building" (versus the original 81%).
​
I chose a Keras Sequential model after some experimentation. I used the architecture below based on this example.
With this approach, the model was able to identify buildings with a train/test accuracy of 79.7 / 78.6%.
​
​
Areas of Improvement for this Specific Model
-
Labeling consistency (ex. Fig. 3 Northwest). There are some cases where buildings are clearly visible in the RGB image, but are not are not labelled accordingly. This hinders training and evaluation, but could be resolved with a more thoroughly labelled set.
-
Categorizing Roads (Fig. 2, Fig. 4) Recognizing roads could be useful, but it wasn’t the purpose of this model.
-
Shorelines (Fig. 2 Southeast) and Grasslands (Fig. 1 West). Further investigation is needed to resolve this.
​
​
Factors to Consider in the Future
-
Vegetation changes: Variations in plant life would impact the NDVI raster and the overall model. This should be considered when using this model in different climates or seasons.
-
Variations in lighting: The presence of shadows will vary due to time of day, time of year, and latitudinal position of the city in question. At large scale, this could affect model performance.
-
City density: The model used in Austin would likely not perform well in a significantly more or less dense city, like San Francisco or rural settlements.
​
​
Links and Helpful Sources​