Dan Sullivan - Official Google Cloud Certified Professional Data Engineer Study Guide

4 Chapter 4Figure 4.1 Summary of compute option featuresFigure 4.2 Edge computing brings some computation outside the cloud and closer to where th...Figure 4.3 A hub-and-spoke message broker patternFigure 4.4 Simple asynchronous message processing

5 Chapter 5Figure 5.1 A basic VM instance provisioning form in the cloud consoleFigure 5.2 Form for creating an instance templateFigure 5.3 Form for creating a managed instance groupFigure 5.4 Cloud console user interface for creating a Kubernetes clusterFigure 5.5 Form to create a Bigtable instanceFigure 5.6 A cloud console form for creating a Cloud Dataproc clusterFigure 5.7 A example log listing in Stackdriver Logging

6 Chapter 6Figure 6.1 Create Role form in the cloud consoleFigure 6.2 Selecting permissions from predefined rolesFigure 6.3 An example of a redacted image generated by the Data Loss Prevention API

7 Chapter 7Figure 7.1 Cloud Bigtable uses a cluster of VMs and the Colossus filesystem for storing, a...Figure 7.2 An example heatmap generated by Cloud Bigtable Key Visualizer Figure 7.3 Specifying multiple clusters when creating a Cloud Bigtable instanceFigure 7.4 Cloud Spanner distributes splits across servers to avoid hotspots.Figure 7.5 An example query execution plan

8 Chapter 8Figure 8.1 Users can search and browse data assets from the Data Catalog overview page.Figure 8.2 Example Data Catalog tag template Figure 8.3 Cloud Dataprep shows statistics about the distribution of data in attributes. Figure 8.4 An example report showing Google Analytics data Figure 8.5 An example Cloud Datalab notebook

9 Chapter 9Figure 9.1 ML pipelines are usually executed in cycles.Figure 9.2 An example of a dataset that can be classified with a linear modelFigure 9.3 An example of a dataset that cannot be classified with a linear modelFigure 9.4 Neural networks have hyperparameters to specify the number of layers and the nu...Figure 9.5 Random forest models have hyperparameters specifying the maximum number of tree...Figure 9.6 The dashed line is a linear model that does not overfit, and the solid line fit...Figure 9.7 A confusion matrix for a classifier making 100 predictions (n = 100)

10 Chapter 10Figure 10.1 CDNs distribute content to servers around the globe.Figure 10.2 Edge computing can be used to process data locally and then send summary data t...Figure 10.3 Reference architecture for Cloud IoT

11 Chapter 11Figure 11.1 An example of how SVMs can be used to predict discrete valuesFigure 11.2 An example decision tree to predict the type of an animalFigure 11.3 An example logistic regression function using a sigmoid curveFigure 11.4 A simple linear regression exampleFigure 11.5 An example deep learning networkFigure 11.6 Examples of a (a) sigmoid function, (b) hyperbolic tangent function, and (c) re...Figure 11.7 The ROC is the curved line, and the AUC is the area under that curve line.

1 Cover

2 Table of Contents

3 Introduction

1 iii

2 iv

3 v

4 vii

5 ix

6 xi

7 xxiii

8 xxiv

9 xxv

10 xxvi

11 xxvii

12 xxviii

13 xxix

14 xxx

15 xxxi

16 xxxii

17 xxxiii

18 xxxiv

19 xxxv

20 xxxvi

21 xxxvii

22 xxxviii

23 1

24 2

25 3

26 4

27 5

28 6

29 7

30 8

31 9

32 10

33 11

34 12

35 13

36 14

37 15

38 16

39 17

40 18

41 19

42 20

43 21

44 22

45 23

46 24

47 25

48 26

49 27

50 29

51 30

52 31

53 32

54 33

55 34

56 35

57 36

58 37

59 38

60 39

61 40

62 41

63 42

64 43

65 44

66 45

67 46

68 47

69 48

70 49

71 50

72 51

73 52

74 53

75 54

76 55

77 56

78 57

79 58

80 59

81 61

82 62

83 63

84 64

85 65

86 66

87 67

88 68

89 69

90 70

91 71

92 72

93 73

94 74

95 75

96 76

97 77

98 78

99 79

100 80

101 81

102 82

103 83

104 84

105 85

106 86

107 87

108 88

109 89

110 90

111 91

112 92

113 93

114 94

115 95

116 96

117 97

118 98

119 99

120 100

121 101

122 102

123 103

124 104

125 105

126 106

127 107

128 108

129 109

130 110

131 111

132 112

133 113

134 114

135 115

136 116

137 117

138 118

139 119

140 120

141 121

142 122

143 123

144 124

145 125

146 126

147 127

148 128

149 129

150 130

151 131

152 132

153 133

154 134

155 135

156 136

157 137

158 139

159 140

160 141

161 142

162 143

163 144

164 145

165 146

166 147

167 148

168 149

169 150

170 151

171 152

172 153

173 154

174 155

175 156

176 157

177 158

178 159

179 160

180 161

181 162

182 163

183 165

184 166

185 167

186 168

187 169

188 170

189 171

190 172

191 173

192 174

193 175

194 176

195 177

196 178

197 179

198 180

199 181

200 182

201 183

202 184

203 185

204 186

205 187

206 188

207 189

208 190

209 191

210 192

211 193

212 194

213 195

214 196

215 197

216 198

217 199

218 200

219 201

220 202

221 203

222 204

223 205

224 206

225 207

226 208

227 209

228 210

229 211

230 212

231 213

232 214

233 215

234 216

235 217

236 218

237 219

238 220

239 221

240 222

241 223

242 224

243 225

244 226

245 227

246 228

247 229

248 230

249 231

250 232

251 233

252 234

253 235

254 236

255 237

256 238

257 239

258 240

259 241

260 242

261 243

262 244

263 245

264 246

265 247

266 248

267 249

268 250

269 251

270 252

271 253

272 254

273 255

274 256

275 257

276 258

277 259

278 260

279 261

280 262

281 263

282 264

283 265

284 266

285 267

286 268

287 269

288 270

289 271

290 272

291 273

292 274

293 275

294 276

295 277

296 278

297 279

298 280

299 281

300 282

301 283

302 284

303 285

304 286

305 287

306 288

307 289

308 290

309 291

310 292

311 293

312 294

313 295

314 296

315 297

316 298

317 299

318 300

319 301

320 302

321 303

322 304

323 305

324 307

325 308

326 309

327 310

328 311

329 312

330 313

331 314

332 315

The Google Cloud Certified Professional Data Engineer exam tests your ability to design, deploy, monitor, and adapt services and infrastructure for data-driven decision-making. The four primary areas of focus in this exam are as follows:

Designing data processing systems

Building and operationalizing data processing systems

Operationalizing machine learning models

Ensuring solution quality

Designing data processing systems involves selecting storage technologies, including relational, analytical, document, and wide-column databases, such as Cloud SQL, BigQuery, Cloud Firestore, and Cloud Bigtable, respectively. You will also be tested on designing pipelines using services such as Cloud Dataflow, Cloud Dataproc, Cloud Pub/Sub, and Cloud Composer. The exam will test your ability to design distributed systems that may include hybrid clouds, message brokers, middleware, and serverless functions. Expect to see questions on migrating data warehouses from on-premises infrastructure to the cloud.