Simon Haykin - Nonlinear Filters

1 Cover

2 Title Page Nonlinear Filters Theory and Applications Peyman Setoodeh McMaster University Ontario, Canada Saeid Habibi McMaster University Ontario, Canada Simon Haykin McMaster University Ontario, Canada

3 Copyright

4 Dedication

5 List of Figures

6 List of Table

7 Preface

8 Acknowledgments

9 Acronyms

10 1 Introduction1.1 State of a Dynamic System 1.2 State Estimation 1.3 Construals of Computing 1.4 Statistical Modeling 1.5 Vision for the Book

11 2 Observability2.1 Introduction 2.2 State‐Space Model 2.3 The Concept of Observability 2.4 Observability of Linear Time‐Invariant Systems 2.5 Observability of Linear Time‐Varying Systems 2.6 Observability of Nonlinear Systems 2.7 Observability of Stochastic Systems 2.8 Degree of Observability 2.9 Invertibility 2.10 Concluding Remarks

12 3 Observers3.1 Introduction 3.2 Luenberger Observer 3.3 Extended Luenberger‐Type Observer 3.4 Sliding‐Mode Observer 3.5 Unknown‐Input Observer 3.6 Concluding Remarks

13 4 Bayesian Paradigm and Optimal Nonlinear Filtering4.1 Introduction 4.2 Bayes' Rule 4.3 Optimal Nonlinear Filtering 4.4 Fisher Information 4.5 Posterior Cramér–Rao Lower Bound 4.6 Concluding Remarks

14 5 Kalman Filter5.1 Introduction 5.2 Kalman Filter 5.3 Kalman Smoother 5.4 Information Filter 5.5 Extended Kalman Filter 5.6 Extended Information Filter 5.7 Divided‐Difference Filter 5.8 Unscented Kalman Filter 5.9 Cubature Kalman Filter 5.10 Generalized PID Filter 5.11 Gaussian‐Sum Filter 5.12 Applications 5.13 Concluding Remarks

15 6 Particle Filter6.1 Introduction 6.2 Monte Carlo Method 6.3 Importance Sampling 6.4 Sequential Importance Sampling 6.5 Resampling 6.6 Sample Impoverishment 6.7 Choosing the Proposal Distribution 6.8 Generic Particle Filter 6.9 Applications 6.10 Concluding Remarks

16 7 Smooth Variable‐Structure Filter7.1 Introduction 7.2 The Switching Gain 7.3 Stability Analysis 7.4 Smoothing Subspace 7.5 Filter Corrective Term for Linear Systems 7.6 Filter Corrective Term for Nonlinear Systems 7.7 Bias Compensation 7.8 The Secondary Performance Indicator 7.9 Second‐Order Smooth Variable Structure Filter 7.10 Optimal Smoothing Boundary Design 7.11 Combination of SVSF with Other Filters 7.12 Applications 7.13 Concluding Remarks

17 8 Deep Learning8.1 Introduction 8.2 Gradient Descent 8.3 Stochastic Gradient Descent 8.4 Natural Gradient Descent 8.5 Neural Networks 8.6 Backpropagation 8.7 Backpropagation Through Time 8.8 Regularization 8.9 Initialization 8.10 Convolutional Neural Network 8.11 Long Short‐Term Memory 8.12 Hebbian Learning 8.13 Gibbs Sampling 8.14 Boltzmann Machine 8.15 Autoencoder 8.16 Generative Adversarial Network 8.17 Transformer 8.18 Concluding Remarks

18 9 Deep Learning‐Based Filters9.1 Introduction 9.2 Variational Inference 9.3 Amortized Variational Inference 9.4 Deep Kalman Filter 9.5 Backpropagation Kalman Filter 9.6 Differentiable Particle Filter 9.7 Deep Rao–Blackwellized Particle Filter 9.8 Deep Variational Bayes Filter 9.9 Kalman Variational Autoencoder 9.10 Deep Variational Information Bottleneck 9.11 Wasserstein Distributionally Robust Kalman Filter 9.12 Hierarchical Invertible Neural Transport 9.13 Applications 9.14 Concluding Remarks

19 10 Expectation Maximization10.1 Introduction 10.2 Expectation Maximization Algorithm 10.3 Particle Expectation Maximization 10.4 Expectation Maximization for Gaussian Mixture Models 10.5 Neural Expectation Maximization 10.6 Relational Neural Expectation Maximization 10.7 Variational Filtering Expectation Maximization 10.8 Amortized Variational Filtering Expectation Maximization 10.9 Applications 10.10 Concluding Remarks

20 11 Reinforcement Learning‐Based Filter11.1 Introduction 11.2 Reinforcement Learning 11.3 Variational Inference as Reinforcement Learning 11.4 Application 11.5 Concluding Remarks

21 12 Nonparametric Bayesian Models12.1 Introduction 12.2 Parametric vs Nonparametric Models 12.3 Measure‐Theoretic Probability 12.4 Exchangeability 12.5 Kolmogorov Extension Theorem 12.6 Extension of Bayesian Models 12.7 Conjugacy 12.8 Construction of Nonparametric Bayesian Models 12.9 Posterior Computability 12.10 Algorithmic Sufficiency 12.11 Applications 12.12 Concluding Remarks

22 References

23 Index

24 Wiley End User License Agreement

1 Chapter 11Table 11.1 Reinforcement learning and variational inference viewed as expect...

1 Chapter 1 Figure 1.1 The encoder of an asymmetric autoencoder plays the role of a nonl...

2 Chapter 6Figure 6.1 Typical posterior estimate trajectories for: (a) sampling importa...

3 Chapter 7Figure 7.1 The SVSF state estimation concept.Figure 7.2 Effect of the smoothing subspace on chattering: (a) and (b) ....Figure 7.3 Combining the SVSF with Bayesian filters.

1 Cover

2 Table of Contents

3 Title Page Nonlinear Filters Theory and Applications Peyman Setoodeh McMaster University Ontario, Canada Saeid Habibi McMaster University Ontario, Canada Simon Haykin McMaster University Ontario, Canada

4 Copyright

5 Dedication

6 List of Figures

7 List of Table

8 Preface

9 Acknowledgments

10 Acronyms

11 Begin Reading

12 References

13 Index

14 Wiley End User License Agreement

1 iii

2 iv

3 v

4 xiii

5 xv

6 xvii

7 xviii

8 xix

9 xxi

10 xxii

11 1

12 2

13 3

14 4

15 5

16 7

17 8

18 9

19 10

20 11

21 12

22 13

23 14

24 15

25 16

26 17

27 18

28 19

29 20

30 21

31 22

32 23

33 24

34 25

35 26

36 27

37 29

38 30

39 31

40 32

41 33

42 34

43 35

44 36

45 37

46 38

47 39

48 41

49 42

50 43

51 44

52 45

53 46

54 47

55 49

56 50

57 51

58 52

59 53

60 54

61 55

62 56

63 57

64 58

65 59

66 60

67 61

68 62

69 63

70 64

71 65

72 66

73 67

74 68

75 69

76 70

77 71

78 72

79 73

80 74

81 75

82 76

83 77

84 78

85 79

86 80

87 81

88 82

89 83

90 85

91 86

92 87

93 88

94 89

95 90

96 91

97 92

98 93

99 94

100 95

101 96

102 97

103 98

104 99

105 100

106 101

107 102

108 103

109 104

110 105

111 106

112 107

113 108

114 109

115 110

116 111

117 112

118 113

119 113

120 114

121 115

122 116

123 117

124 118

125 119

126 120

127 121

128 122

129 123

130 124

131 125

132 126

133 127

134 128

135 129

136 130

137 131

138 132

139 133

140 134

141 135

142 136

143 137

144 138

145 139

146 140

147 141

148 142

149 143

150 144

151 145

152 146

153 147

154 148

155 149

156 150

157 151

158 152

159 153

160 154

161 155

162 156

163 157

164 158