Change Detection and Image Time-Series Analysis 1

Change detection algorithm design : unlike the supervised and semi-supervised CD methods that rely on the available reference samples, unsupervised CD algorithms focus more on the automation and accuracy. Thus, basically, most of the unsupervised CD approaches are data-driven by analyzing the multitemporal data itself. Within this context, for binary CD, if we consider a given change index generated in the previous step, for example, the magnitude of differencing image, automatic thresholding such as empirical segmentation (Bruzzone and Prieto 2000b), Kittler–Illingworth (KI) (Bazi et al . 2005), Otsu (1979) and Bayesian-based expectation–maximization (EM) (Bruzzone and Prieto 2000a) are all simple but effective algorithms proposed in the literature. However, the successful use of such methods depends on the assumption of a certain data distribution such as Gaussian or Rayleigh–Rice mixture (Bovolo et al . 2012; Zanetti et al . 2015), where a wrong estimation may lead to many detection errors. On the contrary, clustering algorithms such as k-means, fuzzy c-means and Gustafson–Kessel clustering (GKC) have been used to address the binary CD problem (Celik 2009; Ghosh et al . 2011), which are distribution-free but require a specific setting to avoid unstable performance, such as the accuracy decrease due to random initialization.

For a multiclass CD case, the unsupervised task becomes more complex since several sub-problems should be solved simultaneously, including the binary change and no-change separation, the number of multiclass change estimation and the multiclass change discrimination (Liu et al . 2019c). In particular, among many solutions, we recall the classical multiple CD technique – change vector analysis (CVA) (Malila 1980). It was designed to analyze possible multiple changes in pairs of bitemporal image bands. A theoretical definition was given to the original CVA approach in the polar domain to provide a more clear mathematical explanation to CVA (Bovolo and Bruzzone 2007b). However, it still has a limitation, i.e. only a part of all possible changes can be detected since only two selected bands are considered in each implementation. If more spectral channels are considered, it becomes very difficult to simultaneously model and visualize multidimensional changes. To break this constraint, a compressed change vector analysis (C 2VA) approach was proposed, which successfully extended the original CVA to a two-dimensional (2D) representation of the multi-band problem (Bovolo et al . 2012). Other works in the literature developed different variations of CVA. For example, a modified CVA was developed to determine the magnitude threshold and direction by combining single-date image classification results (Chen et al . 2003). An improved thresholding approach on change magnitude was designed to optimize the binary separation on each specific change class (Bovolo and Bruzzone 2011). A hierarchical version of C 2VA with an adaptive and sequential projection of spectral change vectors (SCVs) at each level of the hierarchy was proposed to detect multiple changes in bitemporal hyperspectral images (Liu et al . 2015). In this chapter, we also explore the potential capability of C 2VA and extend it from the spectral–spatial point of view.

Performance evaluation : similar to the supervised CD methods, unsupervised binary and multiclass CD approaches can usually be assessed according to the detection accuracy or error index, such as overall accuracy ( OA ), Kappa coefficient ( Kappa ), omission errors ( OE ) and commission errors ( CE ), receiver operating characteristic ( ROC ) curve and area under the curve ( AUC ) value. In this case, the accuracy evaluation usually relies on the manually interpreted change reference map. Note that such a reference map is only used for accuracy evaluation, which is not considered as training data as in the supervised case. In addition, the computational time cost is also another important indicator that reflects the automation and efficiency of unsupervised methods.

The current development of unsupervised CD techniques for multispectral remote sensing images has had great success in many practical applications. However, there are still open issues and challenges that deserve to be further analyzed, which include but are not limited to the following:

1 1) a high-precision multitemporal pre-processing procedure, for example, co-registration techniques;

2 2) multitemporal data quality improvement due to bad imaging conditions, such as system noise, cloud contamination and seasonal spectral variations;

3 3) advanced techniques for correctly estimating the real number of multiclass changes in image scenarios;

4 4) spectral–spatial modeling of change targets to enhance the original pixel-wise spectral representation;

5 5) robust and efficient CD approach in an unsupervised fashion, especially for a large complex CD scene;

6 6) change feature representation by taking advantage of both machine learning and deep learning techniques.

Despite the success of aforementioned CD methods, especially the CVA-based methods, they mainly focus on the spectral changes in each individual pixel or a local neighborhood (Bovolo 2009; Bovolo et al . 2012; Liu et al . 2015). The geometrical characteristics of change targets are not fully modeled and preserved. This may increase the ambiguity due to abnormal spectral variations in isolated pixels and errors (e.g. co-registration errors), leading to the presence of omission and commission errors, especially when dealing with VHR images. In this case, traditional pixel-based CD methods may lose their effectiveness since they were developed under the assumption that pixels are spatially independent. However, for multispectral images in complex urban scenarios, challenging issues may arise due to the limited spectral representation; thus, the same class of objects may have different spectra, or different objects may have the same or very similar spectra. This may significantly increase the detection difficulty, especially when considering the multiclass CD task.

To address the above problems in pixel-based CD (PBCD) techniques, spectral–spatial joint analysis and object-based CD (OBCD) methods are mainstream techniques proposed in the literature. For the former, morphological filters (i.e. self-dual reconstruction filters and alternating sequential filters) were combined with CVA for binary CD in VHR images (Mura et al . 2008). However, a sliding window (i.e. structuring element (SE)) for filtering should be fixed at a given level; thus, it is not robust for multilevel implementation. Morphological attribute profiles (APs) were applied to extract structure-related geometrical features within the scene from each date of panchromatic images (Falco et al . 2013). It includes a multilevel extraction of connected regions in the scene at different scales. Building change information based on the difference in the multitemporal morphological building index (MBI) at the feature and decision level was considered for detecting building changes in VHR images (Huang et al . 2014). A spectral–spatial band expansion approach was developed to enhance the change representation in multispectral images with limited bands, where additional bands were generated from both spectral and spatial viewpoints (Liu et al . 2019b).

For OBCD methods, four main categories exist: image-object, class-object, multitemporal-object and hybrid CD (Chen et al . 2012; Hussain et al . 2013). An object-based CD approach was designed in (Liu and Du 2010), which analyzes different spectral and texture features extracted during the segmentation process from two independent time images. A superpixel segmentation was applied to stacked bitemporal images, and several derived features were used to describe changes in the difference image according to the supervised classification (Wu et al . 2012). An objected-based method was designed to create objects in each single-time image according to segmentation and then generated different representative features (Wang et al . 2018). A weighted Dempster–Shafer theory (wDST) fusion OBCD method was proposed by combining multiple PBCD results, which can automatically calculate and assign a certainty weight for each object of the PBCD result while considering the stability of an object (Han et al . 2020). However, the selection of the optimal segmentation scale is still an open issue and was mainly implemented based on the empirical analysis in OBCD methods (Kaszta et al . 2016). Moreover, most of the above existing work focused on solving a binary CD problem, and very few were designed for dealing with the more challenging unsupervised multiclass CD case (Liu et al . 2019b).