Explaining Residential Electricity Consumption with Satellite Imagery and GANs

This post describes a recent research project in which we use satellite imagery and generative adversarial nets (GANs) to visualize local landcover differences that correspond to multiple levels of residential electricity consumption. Results from our work indicate that larger, more visually distinct buildings and roads are the features most correlated with high electricity consumption; to a lesser extent, sharper edges in fields — possibly indicating higher levels of landscape management — are also linked to higher consumption. A full report on our methodology and findings is available here.

Authorship note: Simone Fobi and Terry Conlon jointly contributed to the following work under the supervision of Professors Carl Vondrick, Vijay Modi, and Jay Taneja.

Figure 1: Transforming a low electricity consumption image to a high consumption one using a CycleGAN variant.

Introduction

In recent years, governments, entrepreneurs, and engineers have explored new pathways for electrification, such as solar home systems and mini-grids, and redoubled investments in traditional grid extensions, all in an effort to improve livelihoods and build sustainable mechanisms for delivering electricity services. To design the most robust, cost-effective systems possible, system planners need accurate estimates of electricity consumption in a region; however, because of widespread data accessibility issues, reliable consumption information is rarely forthcoming.

Over the past decade, there has also been tremendous growth in the availability of global satellite imagery. Researchers have used satellite imagery for many purposes, including for estimating economic development; given the link between development and electricity consumption, we can extend this logic to evaluate the relationship between imagery and electricity use. By pairing satellite imagery with more sparse electricity consumption averages, we can understand consumption patterns in regions where reported electricity demand data does not exist. Here, technology that identifies the drivers of electricity consumption offers myriad benefits to planners and researchers looking to understand the best, region-specific methods for expanding electricity access.

Independently, GANs have become increasingly popular for exploring the exact nature of a decision boundary in classification problems. Traditional GANs alternate training a generator and a discriminator in order to generate images indistinguishable from those in a target set. Here, we use a GAN variant — a CycleGAN — to implement unpaired image-to-image translation between classes of images that correspond to various levels of monthly electricity consumption. Generative networks allow us to create visual features that are important in differentiating between images in multiple electricity consumption classes and general across landscapes.

In this project, we help answer the question of what developmental features are present in high and low electricity consumption images. Presented visually, this understanding increases domain knowledge about what types of development correspond to higher electricity consumption. Furthermore, we show the application of generated imagery to the field of energy system planning. Computer vision researchers have made significant progress in the image generation field in recent years; however, these advances have mostly found applications in related, academic contexts. By employing these advanced techniques to another another technical domain, our work provides justification for further knowledge transfer across disciplines.

Data and Methods

Average postpaid monthly electricity consumption data for this report comes from Kenya Power and Lighting Company (KPLC). We use reported consumption quantities from 2013 to pair with available satellite imagery, collected by DigitalGlobe, taken during the same year. The DigitalGlobe imagery has 3 bands (RGB), a resolution of approximately 0.48 meters per pixel, and is split into patches measuring 450m by 450m; only image patches that contain residential electricity consumers with measured monthly data — as determined by the coordinates associated with each the consumers — are considered as model inputs. It is important to note that not all households in the image have an electricity connection. In addition, as we only have a sample of consumption for the country, there are electricity customers in the image for whom we don’t have consumption data; within each patch, we assume that the customers for which we have data are representative of all customers.

We pare down the image set by considering only image patches with consumption that falls in set electricity consumption classes. Here, in determining the most appropriate class boundaries, we first evaluated the relative distribution of average monthly consumption quantities for all image patches with reported monthly consumption. We constrain the low-consumption class to contain 2915 images with consumption between 3 kWh/month and 30 kWh/month; the medium-consumption class consists of 2874 images that contain residential consumers that average between 40 kWh/month and 70 kWh/month; the high-consumption class contains 2914 images with consumption between 90 kWh/month and 1000 kWh/month. Here, the three consumption classes correspond to consumption values likely to be found in rural, peri-urban, and urban settings. The high consumption and low consumption classes are used in the binary (two class) CycleGAN implementation, the primary focus of this post, while images from all three classes are used in the conditional CycleGAN implementation, discussed briefly in Extensions.

In building our binary CycleGAN model, we adapt the structure of the original CycleGAN implementation presented in JY Zhu et al. (2017). Our model architecture is shown in Figure 2.

Figure 2: Binary CycleGAN architecture. Input images of class X and class Y are given as inputs; the model alternately trains the generators G and F and the discriminators Dx and Dy in order to generate better fake images y_fake and x_fake.

Here, images of class X (low consumption) and Y (high consumption) are given as inputs to generators G and F, respectively. The generated images (y_fake, x_fake) are both sent to their respective discriminators (Dx and Dy), where they are compared to real images in class X and Y to quantify the adversarial loss. The generated images are also sent back through the opposing generator (F and G, respectively) to recreate images of the original class. The recreated images (y_rec, x_rec) are compared to the original x and y images to attain the cycle-consistency loss. During training, these losses — the adversarial loss and the cycle consistency loss — are used to update the weights in the discriminator and generator.

In training and validation, all images are randomly cropped to a size of (472, 472, 3), as the generators and discriminators have difficulty processing images at the original size of (930, 930, 3). We also implement a number of custom losses not present in the original CycleGAN baseline. Along with the adversarial loss (generator and discriminator loss combined) and the cycle consistency loss, both explained in JY Zhu et al. (2017), our network enforces a class maximizing loss and an illumination loss. The class maximizing loss ensures that when transforming an image across the decision boundary, generated images are perceptibly different from their original inputs; in practice, this loss works as a helpful counterbalance to the cycle consistency loss (similar to how the two parts of the generator and discriminator losses act adversarially). The final loss implemented, the illumination loss, ensures that any global change in the pixel values is kept minimal. Because electricity consumption is closely intertwined with development, overall image hue can mirror electricity consumption: images that contain low consumption are often more rural and therefore more green, while images with higher consumption are more urbanized and less green. Therefore, the illumination loss prevents the CycleGAN from merely increasing or decreasing the greenness to generate alternate-class images, instead incentivizing local, distinct image changes.

Results

In transitioning from a low-consumption image to a high-consumption one (Figure 3), the binary CycleGAN implements two primary changes: road and building footprints are both enlarged and brightened. These changes make roads and buildings stand in sharper contrast to background features. We believe these results make sense, as the lighter roads in the transformed images look to have a higher quality than roads in the original one, indicating more development in the generated image. Similarly, tin roofs are seen as higher-status home improvements, and making the upgrade from a thatched roof to a reflective one likely parallels an increase in electricity consumption for a particular household. A secondary change visible in transforming from low-consumption to high-consumption is additional contrast in green areas surrounding houses, roads, and various other human developments. This increase in contrast has the effect of making cropland delineations in a larger pastoral area more clear, giving the impression of higher level of agricultural cultivation. Again, we find it realistic that more visible signs of development coincide with higher levels of electricity consumption.

Figure 3: Transforming low consumption images to high consumption ones; the mask shows pixels that have changed in the transformation.

Transforming from the high consumption class to the low consumption class (Figure 4) largely makes makes the inverse changes to the input imagery: road and buildings footprints are dimmed and blended into their surroundings. These generated images on average look more rural than their high-consumption counterparts.

Figure 4: Transforming high consumption images to low consumption ones; the mask shows pixels that have changed in the transformation.

Extensions

Building upon our binary CycleGAN, we developed a conditional CycleGAN, which generates images in 3 classes of electricity consumption. Along with an image to be transformed, our conditional GAN takes in a label that represents the desired class of the output image: an image in class X would be paired with label n_y or n_z, indicating that the image should be transformed into either class Y or Z. Figure 5 shows an output from our conditional CycleGAN, whereby an image with a medium level of electricity consumption (blue border) is turned into both low consuming and high consuming analogues. These generated images exhibit similar image transformation to those generated by our binary CycleGAN: fake high-consumption images display more development and features more consistent with those in urban settings, while fake low-consumption images present the opposite. Unfortunately, our results for same-direction transformation (e.g. transforming low consumption to medium consumption and subsequently to high consumption) are not as differentiable.

Figure 5: Transforming a medium consumption landscape into low and high consumption images using a conditional CycleGAN.

We also investigated using generated images to improve a binary image consumption classifier. Here, we designed a UNet that takes an image and recreates the mask that transforms it from one class to another (see Figure 6). Using the learned recreation of these masks, we trained a simple convolutional neural net (CNN) for the classification task of differentiating between low and high consumption images. Over 120 training epochs, we attained a maximum classification accuracy using the mask recreation network of 0.74; we were not able to outperform a classification baseline that achieved an accuracy of 0.77 using the satellite imagery as is and an identical CNN and training strategy. As with the results from our conditional CycleGAN implementation, determining whether GAN-generated masks can aid in electricity consumption classification efforts requires additional exploration.

Figure 6: UNet and simple CNN architecture.

This post presents multiple methods of using satellite imagery and computer vision to visualize drivers for residential electricity consumption in Kenya; as best we can tell, it is the first application of GANs to the topic of understanding and predicting electricity consumption levels. Results from our binary CycleGAN implementation reveal that image transformations focus on altering building and road footprints: in generated high consumption images, roads and buildings are brightened and enlarged; these same features are dimmed when generating low consumption images. Intuitively, these results indicate that more visible forms of development coincide with increased levels of electricity consumption. Going forward, open questions remain about how well GANs can generate differentiable images in multiple adjacent electricity consumption classes and whether generated images can improve electricity consumption classification networks; we hope our initial results lead to additional research addressing these issues and others at the nexus of computer vision, deep-learning, and energy system planning.