Pubudu N. Pathirana - Human Motion Capture and Identification for Assistive Systems Design in Rehabilitation

In a small number of applications, a single IMU is used to monitor specific conditions (usually relating the movement of one joint) in telerehabilitation. For instance, Giansanti et al . [116] utilised an IMU with one three‐axis accelerometer and a gyroscope to detect the risk of falling in telerehabilitation. The other example is Han et al . [141], who integrated a 6 degrees of freedom (6DOF, including three-axis accelerometer and gyroscope) IMU with a customised ankle foot orthosis (AFO) to provide a telerehabilitation diagnostic service for patients with two types of conditions, including those with muscle weakness because of brain injuries and those who were about to receive total knee replacement surgery due to osteoarthritis of knee joints. Experiments in the study confirmed the high sensitivity and specificity of the AFO‐IMU module in measuring the flexion and extension motions of knee joints.

However, it is obvious that a single IMU is insufficient to monitor the movement involving multiple joints, such as upper extremity movements and whole body movements. Therefore, a body area network (BAN) or a body sensor network (BSN) was developed to fill this gap. For instance, Nerino et al . [257] proposed a BSN to provide knee telerehabilitation services to patients with anterior cruciate ligament (ACL). In the proposed system, multiple IMUs with 9 DOF (including a three‐axis accelerometer, gyroscope and magnetometer) were attached to the thigh, calf and foot, thus enabling measurements to be taken of the angle of the knee and ankle. After comparing this to the Vicon system, the average angular errors measured by the BSN on the knee and ankle were 2.4° and 3.1° with a standard deviation of 1.8° and 2.4°, respectively. Horak et al . [145] summarised the role played by body‐worn movement monitor devices in rehabilitation services for balance and gait. Cancela et al . [61] evaluated the wearability of a BAN‐based system, named PERFORM, to monitor the symptoms of patients with PD. This system is composed of four tri‐axial accelerometers locating on two legs and two arms, respectively, and a central sensor with one tri‐axial accelerometer and gyroscope positioned on the waist (refer to Figure 1.9). Analysis was conducted considering comfort, biomechanical and physiological aspects of the system. According to the experiment, it was found that patients were generally satisfied to wear such a system. However, some patients were concerned about their privacy, as well as how others might think about them particularly in a public area, thereby showing a bit of anxiety and unwillingness to use this system. Furthermore, the strap made patients uncomfortable and difficult to wear when by themselves. Last, but not least, feedback was necessary during monitoring so that patients knew the system was working properly.

Figure 1.9 Locations of five sensors worn by a subject. Source: Cancela et al . [61].

Kinematic model‐based state estimation can often be used to estimate parameters of interest while combining different information available typically in real‐time applications. Let the N sensor measurements be and the unknown states of the dynamic system be . The dynamic system can be modelled in the form of a state space model,

(1.2)

(1.3)

where A , B and C denote the system matrices and w is the uncertainty input.

Three filters are commonly used to achieve the goal.

First of all, a Kalman filter is used extensively as an optimal state estimator under Bayesian assumptions and is extended to cover non‐linear dynamic and measurement models with stochastic, normal or Gaussian distributed noise [29, 210, 231]. This filter was introduced with the purpose of addressing the limitations of other filters in solving the Wiener problems [163]. Various versions of the Kalman filter, ranging from optimal (also called standard) to extended and various unscented versions pertaining to both linear and non‐linear systems have covered a large number of application scenarios. The proposed system is linear with position, velocity and acceleration acting as state variables, enabling the implementation of an optimal Kalman filter for data fusion. Detailed information about the Kalman filter can be found in Chapter 6. Although Kalman filtering is pervasive with the underlying assumptions of Gaussian noise or system uncertainty distributions, when the uncertainties deviate from these assumptions, the performance degradation can be significant and a more generic uncertainty assumption is warranted.

Secondly, particle filters, which are a form of sequential Monte Carlo sampling, are widely used for state estimation problems with both linear and non‐linear dynamic systems [172, 383, 385]. They relax the single Gaussian assumption in the state and the measurement uncertainties, allowing the handling of more complex noise situations via sampling from multiple probability densities. When this function is a single Gaussian, the particle filter essentially simplifies to the standard Kalman filter. The application of a particle filter in our linear system is described in Chapter 6.

From the literature, it is clear that a number of sensors have been applied in telerehabilitation. For the evaluation of the approaches introduced in this thesis, we utilised Kinect as the example of affordable OBMCDs to collect data for the following two major reasons.

First of all, Kinect is a non‐contact motion capture device. Patients do not need to attach any extra item to their bodies. As a result, they may be able to perform rehabilitation exercises or ADLs with more natural poses.

Secondly, compared to conventional RGB cameras, Kinect is able to track 3D positions of 20–25 joints throughout a human body. Therefore, without making much effort in processing images, data collected from a Kinect are much more convenient for further analysis. This is critical for automatically assessing the kinematic performance of patients in a telerehabilitation system without the presence of a professional therapist.

However, the majority of affordable OBMCDs still suffer from a number of drawbacks. One is that it may not be accurate enough to track small movements, as well as the positions of joints occluded by other body parts. Secondly, its small viewing range limits its application in lower extremity rehabilitation, which, in many cases, involves a large moving area. A solution for these limitations is discussed in Chapter 2.