Dongming Feng - Computer Vision for Structural Dynamics and Health Monitoring

Dynamics‐based SHM techniques can be categorized into frequency‐domain and time‐domain system identification methods. Carden and Fanning [5] presented an extensive literature review of frequency‐domain SHM techniques based on changes in measured modal properties such as natural frequencies, mode shapes and their curvatures, modal flexibility and its derivatives, modal strain energy, frequency response functions, etc. Modal properties are obtained using various modal analysis techniques, e.g. the natural excitation technique, frequency domain decomposition, stochastic subspace identification, the random decrement technique, blind source separation, and the autoregressive‐moving‐average model‐fitting method. All of these methods have achieved satisfactory performance in numerical and experimental studies. For example, Kim and Stubbs [6] proposed a technique to locate and quantify cracks in beam‐type structures based on a single damage indicator by using changes in natural frequencies. Lee et al. [7] presented a neural network–based method for element‐level damage detection using mode shape differences between intact and damaged structures. Pandey et al. [8] proposed for the first time that mode shape curvature, which is the second derivative of the mode shape, is a sensitive indicator of damage. Feng et al. [9] developed the first neural network–based system identification framework for updating baseline structural models of two sensor‐instrumented highway bridges.

Time‐domain SHM techniques, rather than working with modal quantities, directly utilize measured structural response time histories to identify structural parameters. The identification in the time domain is often formulated as an optimization process, wherein the objective function is defined as the discrepancy between the measured and predicted responses. In the majority of existing studies, which are referred to as input–output methods , the known or measured excitation forces are a prerequisite for obtaining the predicted structural responses. However, it is highly difficult to measure excitation forces such as vehicle loads on bridges. Recently, there have been attempts to simultaneously identify both structural parameters and input forces from output‐only identification formulations. For example, Rahneshin and Chierichetti [10] proposed an iterative algorithm – the extended load confluence algorithm – to predict dynamic structural responses in which limited or no information about the applied loads is available. Xu et al. [11] presented a weighted adaptive iterative least‐squares estimation method to identify structural parameters and dynamic input loadings from incomplete measurements. Sun and Betti [12] demonstrated the effectiveness of a hybrid heuristic optimization strategy for simultaneous identification of structural parameters and input loads via three numerical examples. Feng et al. [13] proposed a numerical methodology to simultaneously identify bridge structural parameters and moving vehicle axle load histories from a limited number of acceleration measurements.

On the other hand, various filter‐type algorithms for online system identification have been extensively studied in the literature, using either input–output or output‐only time‐domain data. Examples include the extended Kalman filter, unscented Kalman filter, particle filter, and H ∞filter. For example, Chen and Feng [14] proposed a recursive Bayesian filtering approach to update structural parameters and their uncertainties in a probabilistic structural model. Soyoz and Feng [15] formulated an extended Kalman filter for instantaneous detection of seismic damage of bridges and validated its efficacy through large‐scale seismic shaking‐table tests. Although these online estimation algorithms have proved to be successful in many applications, they also present challenges. For example, the sensitivity of these methods to initial guess values affects the stability and convergence of estimated parameters to exact ones. In addition, parameter/damage identification methods based on heuristic algorithms – e.g. genetic algorithm, particle swarm optimization, artificial neural network, differential evolution, and artificial bee colony – have gained increasing attention due to their global optimization performance. However, validation of these methods is mostly limited to numerical or controlled laboratory examples rather than real‐world structures.

For both frequency‐ and time‐domain methods, vibration‐based SHM strategies have proved effective in evaluating the global health state of structures and performing a rapid risk assessment. However, their wide deployment in realistic engineering structures is limited by the prohibitive requirement of installing dense on‐structure sensor networks (primarily accelerometers) and associated data‐acquisition systems. Contact‐type wired sensors require time‐consuming, labor‐intensive installation and costly maintenance for successful long‐term monitoring, which poses many economic and practical challenges. Although wireless sensor technology has addressed several limitations of wired sensors by eliminating cumbersome wiring, data acquisition remains challenging due to the complexity of data transmission, time synchronization, and power consumption, especially when hundreds of wireless sensors are mounted on a large‐scale structure to measure dynamic responses. Moreover, one main bottleneck is that conventional on‐structure sensors provide sparse, discrete point‐wise measurements and thus low spatial‐sensing resolutions, which limits the effectiveness of SHM on a large‐scale structure. Although such a sensor network with a limited number of sensors may allow for the detection of changes in overall structural dynamics, it is often insufficient for identifying the location or assessing the extent of damage.

To address these practical limitations, the research and engineering practitioner communities have been actively exploring new sensor technologies that can advance the current state of SHM practice. This book introduces the emerging computer vision–based sensor technology.

While most SHM studies are based on the measurement of structural acceleration responses, displacement responses more directly reflect overall structural stiffness and thus offer the potential for improved accuracy in the assessment of structural conditions. As shown in Figure 1.1, sensors currently available for measuring structural displacements can be classified as contact types, such as the linear variable differential transformer (LVDT); and string potentiometer and noncontact types, such as GPS, laser vibrometers, and radar interferometry systems. These displacement sensors suffer from many limitations for field applications. For example, it is costly and highly difficult, if not impossible, to install an LVDT or a string potentiometer, which requires a stationary reference point; noncontact laser vibrometers are generally accurate but are costly and have a short measurement distance because of safety regulations; GPS sensors are easier to install, but the measurement accuracy is limited; and an interferometric radar system allows remote measurements with good resolution but requires reflecting surfaces mounted on the structure, which can be difficult to install and maintain.

Figure 1.1 Common displacement sensors: (a) LVDT; (b) laser vibrometer; (c) GPS.

Rapid advances in cameras and computer vision techniques have made vision‐based sensing a promising alternative to conventional sensors for structural dynamic displacement measurement and health monitoring. As shown in Figure 1.2, a typical computer vision–based sensor system simply consists of one or more digital cameras and a computing unit such as a laptop or a tablet PC with measurement software installed. Video images of features on a structure, such as rivets and edges, are captured by the camera and streamed into the computer. By processing the digital video images using the measurement software, displacement time histories can be obtained at multiple locations simultaneously. The emerging vision‐based sensor offers significant advantages over conventional contact‐type and other noncontact‐type displacement sensors, as summarized next [16]: