Information about the built environment at scale provides actionable insight that can transform a broad spectrum of industries, by enabling companies to deliver better, more targeted services to customers, telecommunications operators to perform accurate infrastructure planning, and government agencies to conduct regional planning and management. This is the vision behind Geoscape®, a location-intelligence product conceived by PSMA Australia and developed in partnership with DigitalGlobe, a Maxar company, that delivers detailed information about buildings and their attributes, trees and land cover for every address in Australia.
The data for each building in Geoscape includes the presence or absence of solar panels. This information is important for the insurance industry because solar panels may pose a fire hazard and thus affect insurance premiums. For firefighters, knowing if a structure has solar panels can also help them stay safe. In addition, quantifying solar panel adoption can help solar panel manufacturers market to customers, enable utilities to understand and predict distributed energy generation, and drive research on factors that affect renewable energy choices.
How did DigitalGlobe identify solar panels on 15 million buildings spanning 7.6 million km2 of the Australian continent? We first collected a clean data set of hundreds of thousands of rooftops by running a Tomnod crowdsourcing campaign where we asked the crowd to identify rooftops as having or not having solar panels. Our team curated the crowd labels. The curated data set was used to train a convolutional neural network to classify each rooftop as having or not having solar panels. The resulting model was containerized on GBDX and deployed over the entire building data set.
A sample of the results for five different cities is shown in the following map. The presence and absence of a solar panel is indicated by green and red, respectively. (The building footprints were provided by our partner, Ecopia.)
While maps are useful for visual inspection, numbers are necessary to assess the model accuracy. Our team examined a large sample of the model’s classifications and recorded the classification errors. For comparison purposes, we had the Tomnod crowd label the same buildings, and recorded the crowd’s classification errors as well. We then computed confidence intervals for both the model’s and the crowd’s precision and recall. (Precision is the percentage of buildings that actually have solar panels as a percentage of the buildings identified as having a solar panel. Recall is the percentage of buildings with solar panels that were positively identified.) The results are shown in the following plot for two sets of buildings in different parts of Australia, with each set numbering in the tens of thousands.
[caption id="attachment_6628" align="aligncenter" width="2156"]
Precision and recall confidence intervals for the model and the Tomnod crowd on two sets of buildings.[/caption]
These numbers are astonishing for multiple reasons. First, the model outperformed the crowd. Not only is it orders of magnitude quicker to deploy the model compared to running a crowdsourcing campaign, it is also more accurate. In addition, achieving precision and recall that exceed 90 percent given the complexity of this task is remarkable. Even at 30 cm/pixel resolution solar panels are not easy to distinguish and tell apart from similar features such as solar water heaters and skylights. Here are some representative examples where the model made the correct decision and the crowd did not.
[caption id="attachment_6629" align="aligncenter" width="550"]
Faintly discernible solar panels; model said yes, crowd said no.[/caption]
[caption id="attachment_6630" align="aligncenter" width="738"]
Skylights on the left, a pergola and a water heater on the right; model said no, crowd said yes.[/caption]
While the model outperformed the crowd, it too delivered some false positives and false negatives shown in the samples below.
[caption id="attachment_6631" align="aligncenter" width="1390"]
Model false positives[/caption]
[caption id="attachment_6632" align="aligncenter" width="1448"]
Model false negatives[/caption]
Why did the model behave this way? It’s not clear, which is part of the fascination of deep learning! At DigitalGlobe Research and Development, we are committed to looking for these answers, and to harnessing the amazing progress of this technology to build accurate and competitive information products that can provide answers to our customers’ questions at global scale. Stay tuned for updates!
Geoscape® is a registered trademark of PSMA Australia.