Change Detection and Image Time Series Analysis 2

In this chapter, the problem of the generation of a classification map from an input time series composed of multisensor multiresolution remote sensing images has been discussed. First, the related literature in the area of remote sensing data fusion has been reviewed. Then, an advanced approach based on multiple quad-trees in cascade has been described. It derives from the multisensor generalization of a previous technique focused on the time series of single-sensor data, and addresses the challenging problem of multisensor, multifrequency and multiresolution fusion for classification purposes.

In the framework of this approach, two algorithms have been developed for two different multimodal fusion objectives. In the first one, the general task of jointly classifying a time series of multisensor multiresolution imagery is considered. In the second one, the focus is on the special case of the fusion of multimission data acquired by COSMO-SkyMed, RADARSAT-2 and Pléiades. In the case of both techniques, the fusion task is formalized in the framework of hierarchical probabilistic graphical models – most remarkably hierarchical MRF on cascaded quad-trees. Inference and parameter estimation are addressed through the maximum posterior mode criterion and FMM, respectively.

Examples of experimental results provided by the two proposed algorithms have been shown with regard to high- or very-high-resolution time series associated with case studies in Port-au-Prince, Haiti. The results have suggested that the described algorithms successfully benefit from the data sources in the input multisensor time series, improving the classification result, compared to those obtained using single-mission, single-scale or previous methods in terms of classification accuracy, computation time or spatial regularity.

A major property of the proposed hierarchical Markovian framework is its flexibility. The graphical architecture associated with multiple quad-trees in cascade allows the incorporation of input image sources associated with different sensors, acquisition times and spatial resolutions – jointly. Relevant extensions of this framework may involve the combination with spatial–contextual models within each layer of the quad-trees, or with the intrinsically multiscale structure of CNN (Goodfellow et al. 2016).

The authors would like to thank the Italian Space Agency (ASI), the French Space Agency (CNES) and the Canadian Space Agency (CSA) for providing COSMO-SkyMed, Pléiades and RADARSAT-2 data, respectively. The COSMO-SkyMed and RADARSAT-2 images were procured in the context of the SOAR-ASI 5245 project, framed within the joint ASI-CSA announcement of opportunity.

ALEjaily, A., El Rube, I., Mangoud, M. (2008). Fusion of remote sensing images using contourlet transform. Innovations and Advanced Techniques in Systems, Computing Sciences and Software Engineering , pp. 213–218.

Basseville, M., Benveniste, A., Willsky, A. (1992a). Multiscale autoregressive processes I. Schur–Levinson parametrizations. IEEE Transactions on Signal Processing , 40(8), 1915–1934.

Basseville, M., Benveniste, A., Willsky, A. (1992b). Multiscale autoregressive processes II. Lattice structures for whitening and modeling. IEEE Transactions on Signal Processing , 40(8), 1935–1954.

Benedetti, P., Ienco, D., Gaetano, R., Ose, K., Pensa, R.G., Dupuy, S. (2018). M3fusion: A deep learning architecture for multiscale multimodal multitemporal satellite data fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11(12), 4939–4949.

Benz, U. (1999). Supervised fuzzy analysis of single and multichannel SAR data. IEEE Transactions on Geoscience and Remote Sensing , 37(2), 1023–1037.

Berthod, M., Kato, Z., Yu, S., Zerubia, J. (1996). Bayesian image classification using Markov random fields. Image and Vision Computing , 14(4), 285–295.

Bouman, C. (1991). A multiscale image model for Bayesian image segmentation. Thesis, Purdue University, School of Electrical Engineering.

Brunner, D., Lemoine, G., Bruzzone, L. (2010). Earthquake damage assessment of buildings using VHR optical and SAR imagery. IEEE Transactions on Geoscience and Remote Sensing , 48(5), 2403–2420.

Bruzzone, L., Prieto, D.F., Serpico, S. (1999). A neural-statistical approach to multitemporal and multisource remote-sensing image classification. IEEE Transactions on Geoscience and Remote Sensing , 37(3), 1350–1359.

Burt, P. (1984). The Pyramid as a Structure for Efficient Computation . Springer, Berlin, Heidelberg.

Carvalho, M., Le Saux, B., Trouvé-Peloux, P., Champagnat, F., Almansa, A. (2019). Multitask learning of height and semantics from aerial images. IEEE Geoscience and Remote Sensing Letters , pp. 1391–1395.

Celeux, G., Chauveau, D., Diebolt, J. (1996). Stochastic versions of the EM algorithm: An experimental study in the mixture case. Journal of Statistical Computation and Simulation , 55(4), 287–314.

Chardin, A. (2000). Modèles énergétiques hiérarchiques pour la résolution des problèmes inverses en analyse d’images: application à la télédetection. PhD Thesis, University of Rennes 1, France.

Chen, Y., Li, C., Ghamisi, P., Jia, X., Gu, Y. (2017). Deep fusion of remote sensing data for accurate classification. IEEE Geoscience and Remote Sensing Letters , 14(8), 1253–1257.

Cheng, X., Zheng, Y., Zhang, J., Yang, Z. (2020). Multi-task multi-source deep correlation filter for remote sensing data fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13, 3723–3734.

Chibani, Y. and Houacine, A. (2003). Redundant versus orthogonal wavelet decomposition for multisensor image fusion. Pattern Recognition , 36(4), 879–887.

Choi, M., Kim, R., Nam, M.-R., Kim, H. (2005). Fusion of multispectral and panchromatic satellite images using the curvelet transform. IEEE Geoscience and Remote Sensing Letters , 2(2), 136–140.

Demirel, H. and Anbarjafari, G. (2010). Satellite image resolution enhancement using complex wavelet transform. IEEE Geoscience and Remote Sensing Letters , 7(1), 123–126.

Dousset, B. and Gourmelon, F. (2003). Satellite multi-sensor data analysis of urban surface temperatures and landcover. ISPRS Journal of Photogrammetry and Remote Sensing , 58(1), 43–54.

Forster, B., Van De Ville, D., Berent, J., Sage, D., Unser, M. (2004). Complex wavelets for extended depth-of-field: A new method for the fusion of multichannel microscopy images. Microscopy Research and Technique , 65(1–2), 33–42.

Fukunaga, K. (2013). Introduction to Statistical Pattern Recognition . Academic Press, Cambridge, MA.

Gamba, P., Lisini, G., Tomás, L., Almeida, C., Fonseca, L. (2011). Joint VHR-LIDAR classification framework in urban areas using a priori knowledge and post processing shape optimization. IEEE Urban Remote Sensing Event (JURSE) , pp. 93–96.

Ghamisi, P., Höfle, B., Zhu, X.X. (2016). Hyperspectral and LiDAR data fusion using extinction profiles and deep convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 10(6), 3011–3024.

Gómez-Chova, L., Tuia, D., Moser, G., Camps-Valls, G. (2015). Multimodal classification of remote sensing images: A review and future directions. Proceedings of the IEEE , 103(9), 1560–1584.

Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning . MIT Press, Cambridge, MA.

Graffigne, C., Heitz, F., Perez, P., Preteux, F., Sigelle, M., Zerubia, J. (1995). Hierarchical Markov random field models applied to image analysis: A review. International Symposium on Optical Science, Engineering, and Instrumentation (SPIE) , International Society for Optics and Photonics, pp. 2–17.