Evaluating super-resolution reconstruction of satellite images

Super-resolution reconstruction (SRR) consists in enhancing image spatial resolution given a single image or a bunch of images presenting the same scene. Potential benefits of SRR are evident, when images of high resolution are required, but are unavailable due to technological limitations or economic reasons. Obviously, this is inherent to satellite imagery—medium or high resolution data (e.g., Sentinel-2 images) could be processed to obtain images of higher resolution, whose accessibility is much lower due to their price, lower coverage and longer revisit times. Although SRR is a well-studied problem of computer vision, to the best of our knowledge, no satisfactory solution exists that would allow for enhancing satellite images in practice. One of the pivotal obstacles we identified, is concerned with the evaluation methodology commonly adopted while developing the SRR techniques—an image of high resolution ^(ℎ) is downscaled using different offsets and degradation operators to obtain a set of low-resolution images { ⁽⁾}. The goal is to reconstruct the original image from {⁽⁾} and the similarity between the outcome ′^(ℎ) and^(ℎ), commonly measured with peak signal-to-noise ratio (PSNR), is used to evaluate the quality of SRR. While such procedure is sufficient to verify some aspects of the algorithms, the assumptions imposed on the degradation model may actually not hold. Unfortunately, there are no benchmarks encompassing the real images acquired at different resolutions, which could correspond to the real-world conditions. In this paper, we intend to fill the aforementioned gap with our validation framework, encompassing (i) synthetic images, (ii) satellite images degraded using known operators, and (iii) images covering roughly the same area acquired using sensors of different resolution ({⁽⁾} is composed of either multiple images of the same area, or different spectra of a hyper-spectral image). Furthermore, we extend the measures used to compare ′^(ℎ) with ^(ℎ)—not only do we rely on various similarity measures (such as PSNR), but also we explore how to exploit landmark detectors to assess whether and how SRR reduces the ground sampling distance (GSD). Importantly, we utilize the elaborated evaluation metrics while defining the objective function, whose optimization allows for reconstructing the super-resolution image. In the work reported here, we optimize that function using evolutionary computation and we present the initial, yet very encouraging results, obtained for real-world data, including Sentinel-2 images. This allows us to outline the future research pathways which will lead to developing effective SRR methods suitable for practical applications.