Pubudu N. Pathirana - Human Motion Capture and Identification for Assistive Systems Design in Rehabilitation

Though angles of joints, as well as their derivatives, are utilised widely in encoding human motions, trajectories of joints and their derivatives can also be observed in some rehabilitation and telerehabilitation applications.

The first example is that of Chang et al . [67], who developed a programme to use Kinect as the motion capture device for spinal cord injury (SCI) rehabilitation. In this programme, the trajectories of the hand, elbow and shoulder were recorded to represent the external rotation of upper extremities. Similarly, Su [339] also developed a rehabilitation system, named KHRD, to provide home‐based rehabilitation services. To represent human motions, two key features were used, including trajectories of joints, as well as their speed. Additionally, Cordella et al . [78] modified the Kinect into a marker‐based device to measure the positions of joints on a hand (refer to Figure 1.11). Markers with a dimension of 1.2 cm were attached to the joints of fingers, as well as the wrist. After detecting the centre of these markers, a robust tracking scheme was developed to track the position of each marker. Thus the movement of a hand was encoded by the trajectories of each joint on the hand, as well as the trajectory of the wrist.

From the literature, it is found that encoders used in human motion recognition are similar to those in physical telerehabilitation in many studies. For instance, features like trajectories, velocity, acceleration, angle, angular velocity and angular acceleration are most commonly used in both fields. Though patients with movement disorders usually have a limited range of motions, they may be required to do certain tasks so as to evaluate their ability to perform ADLs, which usually are composed of a series of simple movements.

Figure 1.11 Marker‐based hand tacking system. Source: Cordella et al . [78].

As a result, there remain challenges in developing formal descriptions and robust computational procedures for the automatic interpretation and representation of motions of patients. The majority of studies [92, 158] employed a variety of human motion encoders to recognise or decompose general movement, such as reaching, waving hands, jumping, walking and so on. Few of them investigated details in each general movement, for example, the even smaller atomic components included in these general movements that are of importance for syntactic and structural descriptions of human movements in detail, especially in a clinic and rehabilitation environment, where the details of movements of body parts require a form of motion language or, at least, syntax. A novel approach to encode human motion trajectories will be discussed in Chapter 3.

In recent decades, with the advancements in telerehabilitation and associated motion capture technologies, an increasing number of research and development activities are focusing on the development of automated quantitative measures of patient performance in ADLs [136, 262]. Due to the important role played by the upper extremity in ADLs [99], an automated approach for measuring and assessing the ability of upper extremities to perform certain tasks is vital for telerehabilitation systems to deliver their full potential.

In the past few decades, a number of approaches have been proposed for assessing upper extremities, the majority of which are questionnaire‐based. For musculoskeletal movement disorders of the extremities, most scales are generic. For instance, the self‐reported Musculoskeletal Function Assessment (MFA) instrument [226], Short Musculoskeletal Function Assessment (SMFA) questionnaire [344] and self‐administered measure of disabilities of the arm, shoulder and hand [148] were developed, but few of them are condition specific. More examples can be found in Adam et al . [99]. However, for neurological movement disorders, assessments are more disease‐specific and rarely focus on upper extremities. For instance, Chedokee‐McMaster (CM) assessment, the Fugl‐Meyer (FM) assessment and the Wolf Motor Function Test (WMFT) assessment are for stroke. The Fahn‐Marsden rating scale (F‐M) [58], the Global Dystonia Rating Scale (GDRS) [12], the Unified Dystonia Rating Scale (UDRS) [11] and so on [22] were developed for dystonia. The Parkinson's Disease Questionnaire (PDQ‐39) [284] and its shorter version (PDQ‐8) [159], as well as the Parkinson's Disease Quality of Life (PDQL) questionnaire [144], the Webster [375] and the Unified Parkinsonâs Disease Rating Scale (UPDRS) [227] were developed for Parkinson's disease. More relevant to our work, Lane et al . [192] developed the Abnormal Involuntary Movement Scale (AIMS) to assess patients with dyskinesia. In this scale, the amplitude of involuntary movements was taken into consideration. In addition, Goetz et al . [119] proposed to use the Objective Dyskinesia Rating Scale (also known as the Rush Dyskinesia scale) to assess the severity of dyskinesia.

Recently, with the development of sensing technologies, a number of approaches have been proposed to either automate conventional testing scales or develop new methods. Some examples are shown as follows.

Knorr et al . [177] automated two tasks, including reaching to the front and to the side, in the WMFT for people after stroke. Three accelerometers were attached to the hand, the corresponding forearm and the upper arm respectively to capture the characteristics of motion patterns. Eventually, two linear features (root mean square errors of acceleration and jerk) and a non‐linear feature (approximate entropy of acceleration) were evaluated as the measurements of functional limitation and motor impairment. Similarly, Wade et al . [369] tried to automate the widely used WMFT for post‐stroke patients. In their proposal, one IMU was attached to the wrist of the subject to measure the time used by the subject to finish each task in different WMFT tasks. Hester et al . [143] utilised 14 features to predict the clinical measurement scores of patients with stroke. These features were extracted from data collected by four accelerometers, three of which were attached to the affected hand, forearm and upper arm, while the fourth sensor was on the trunk. As for the tasks, different from the two previous studies, some tasks were not included in the WMFT. The process of predicting clinical scores from these 14 features is shown in Figure 1.12. Furthermore, Leibowitz et al . [202] introduced a protocol to quantitatively measure the proprioception deficit. In the protocol, the affected hand was located under a square board so that it could not be seen by the subject. After passively moving the impaired hand to one of four locations, the healthy hand was required to be moved to the same location actively. Eventually, the positions of both hands were measured by a MiniBIRD500 magnetic tracking system and the positional differences between the impaired and non‐impaired hands were computed as an indicator.

Figure 1.12 The process of predicting clinical scores from 14 features.