Patrick Siarry - Optimization and Machine Learning

5 Introduction

6 Begin Reading

7 List of Authors

8 Index

9 End User License Agreement

1 v

2 iii

3 iv

4 xi

5 xii

6 xiii

7 xiv

8 1

9 3

10 4

11 5

12 6

13 7

14 8

15 9

16 10

17 11

18 12

19 13

20 14

21 15

22 16

23 17

24 18

25 19

26 20

27 21

28 22

29 23

30 24

31 25

32 26

33 27

34 28

35 29

36 30

37 31

38 32

39 33

40 34

41 35

42 36

43 37

44 38

45 39

46 40

47 41

48 42

49 43

50 44

51 45

52 46

53 47

54 48

55 49

56 50

57 51

58 52

59 53

60 55

61 56

62 57

63 58

64 59

65 60

66 61

67 62

68 63

69 64

70 65

71 66

72 67

73 68

74 69

75 70

76 71

77 72

78 73

79 74

80 75

81 76

82 77

83 78

84 79

85 80

86 81

87 82

88 83

89 84

90 85

91 86

92 87

93 88

94 89

95 90

96 91

97 92

98 93

99 94

100 95

101 96

102 97

103 98

104 99

105 100

106 101

107 102

108 103

109 104

110 105

111 106

112 107

113 108

114 109

115 110

116 111

117 112

118 113

119 114

120 115

121 116

122 117

123 118

124 119

125 121

126 122

127 123

128 124

129 125

130 126

131 127

132 128

133 129

134 130

135 131

136 132

137 133

138 134

139 135

140 136

141 137

142 138

143 139

144 140

145 141

146 142

147 143

148 144

149 145

150 146

151 147

152 148

153 149

154 150

155 151

156 152

157 153

158 154

159 155

160 156

161 157

162 158

163 159

164 160

165 161

166 162

167 163

168 164

169 165

170 166

171 167

172 169

173 170

174 171

175 172

176 173

177 174

178 175

179 176

180 177

181 178

182 179

183 180

184 181

185 182

186 183

187 184

188 185

189 186

190 187

191 188

192 189

193 190

194 191

195 192

196 193

197 194

198 195

199 196

200 197

201 198

202 199

203 201

204 202

205 203

206 204

207 205

208 206

209 207

210 208

211 209

212 210

213 211

214 212

215 213

216 214

217 215

218 216

219 217

220 218

221 219

222 220

223 221

224 222

225 223

226 224

227 225

228 226

229 227

230 228

231 229

232 230

233 231

234 232

235 233

236 234

237 235

238 236

239 237

SCIENCES

Computer Science ,

Field Directors – Valérie Berthé and Jean-Charles Pomerol

Operational Research and Decision , Subject Head – Patrick Siarry

Coordinated by

Rachid Chelouah

Patrick Siarry

First published 2022 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the under mentioned address:

ISTE Ltd

27-37 St George’s Road

London SW19 4EU

UK

www.iste.co.uk

John Wiley & Sons, Inc

111 River Street

Hoboken, NJ 07030

USA

www.wiley.com

© ISTE Ltd 2022

The rights of Rachid Chelouah and Patrick Siarry to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Control Number: 2021949293

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library

ISBN 978-1-78945-071-2

ERC code:

PE1 Mathematics

PE1_19 Control theory and optimization

PE6 Computer Science and Informatics

PE6_11 Machine learning, statistical data processing and applications using signal processing (e.g. speech, image, video)

Rachid CHELOUAH

CY Cergy Paris University, France

Machine learning is revolutionizing our world. It is difficult to conceive of any other information technology that has developed so rapidly in recent years, in terms of real impact.

The fields of machine learning and optimization are highly interwoven. Optimization problems form the core of machine learning methods and modern optimization algorithms are using machine learning more and more to improve their efficiency.

Machine learning has applications in all areas of science. There are many learning methods, each of which uses a different algorithmic structure to optimize predictions, based on the data received. Hence, the first objective of this book is to shed light on key principles and methods that are common within both fields.

Machine learning and optimization share three components: representation, evaluation and iterative search. Yet while optimization solvers are generally designed to be fast and accurate on implicit models, machine learning methods need to be generic and trained offline on datasets. Machine learning problems present new challenges for optimization researchers, and machine learning practitioners seek simpler, generic optimization algorithms.

Quite recently, modern approaches to machine learning have also been applied to the design of optimization algorithms themselves, taking advantage of their ability to capture valuable information from complex structures in large spaces. Those capacities appear to be useful, especially for the representation and evaluation components. As large, complex structures are ubiquitous in optimization problems, and can be used as huge implicit datasets, the use of machine learning enabled the efficiency and genericity of optimization methods to be improved.

This book presents modern advances in the selection, configuration and engineering of algorithms that rely on machine learning and optimization. It is structured into two parts. Part 1is dedicated to the most common optimization applications. Part 2describes and implements several applications of machine learning.

Part 1comprises four chapters which focus on real-world application of optimization algorithms.

Chapter 1addresses the problem of vehicle routing with loading constraints and combines two combinatorial optimization problems: the capacity vehicle routing problem (CVRP) and the two-/three-dimensional bin packing problem (2/3D-BPP). The authors have studied real transport problems such as the transport of furniture or industrial machinery.