Alan R. Simon - Data Lakes For Dummies

10 Chapter 10FIGURE 10-1: Your hospital’s legacy systems environment.FIGURE 10-2: Selecting data mart dimensional models to retain for your new data...FIGURE 10-3: Replacing best-of-breed applications with an integrated EHR packag...FIGURE 10-4: Pairing your new EHR system with a data lake.FIGURE 10-5: Setting up curated data packages in your data lake gold zone.FIGURE 10-6: Delaying platform decisions until you gain a broader view of your ...FIGURE 10-7: Your EHR system using both streaming and batch feeds into your dat...FIGURE 10-8: Making key ingestion and bronze zone data set decisions.FIGURE 10-9: Streaming persistent data into the gold zone.FIGURE 10-10: Making different architectural decisions for various data streams...FIGURE 10-11: Putting your silver zone to work.FIGURE 10-12: Adding data pipelines to your data lake buildout.FIGURE 10-13: Bringing your data lake sandbox into the picture.

11 Chapter 11FIGURE 11-1: Public versus private clouds: a visual analogy.FIGURE 11-2: Allocation of responsibilities for SaaS, PaaS, and IaaS.

12 Chapter 12FIGURE 12-1: The fundamental structure of Amazon S3.FIGURE 12-2: Mimicking folders in Amazon S3 through filenames.FIGURE 12-3: Building your entire AWS data lake using only S3 for data storage.FIGURE 12-4: Using Glue Crawler and Glue Data Catalog to maintain up-to-date da...FIGURE 12-5: Using a Lake Formation blueprint for data lake ingestion.FIGURE 12-6: Using Amazon Kinesis Data Streams for hospital patient vital signs...FIGURE 12-7: Athena using the Glue Data Catalog to access S3 data with SQL.FIGURE 12-8: Using Amazon Redshift in your data lake’s gold zone.FIGURE 12-9: An end-to-end hospital data lake built on AWS services.

13 Chapter 13FIGURE 13-1: Organization of the Azure cloud.FIGURE 13-2: An Azure data lake framework.FIGURE 13-3: ADLS Gen2, the best of both worlds.FIGURE 13-4: ADLS containers, folders, and files.FIGURE 13-5: Ingesting, copying, and sinking data along an ADF pipeline.FIGURE 13-6: Using Azure Event Hubs for a publish-and-subscribe model.FIGURE 13-7: Bidirectional messaging and streaming with Azure IoT Hub.FIGURE 13-8: Using Azure SQL Database in your Azure data lake.FIGURE 13-9: Azure data lake architecture for IoT analytics.FIGURE 13-10: Azure data lake architecture for industrial IoT predictive mainte...FIGURE 13-11: Azure data lake architecture for defect analysis and prevention.FIGURE 13-12: Azure data lake architecture for rideshare company forecasting.

14 Chapter 14FIGURE 14-1: Your data lake four-element scorecard.FIGURE 14-2: Dividing each data lake evaluation criteria into scoreable element...FIGURE 14-3: Focus only on your raw data.FIGURE 14-4: Identifying your raw data hot spots.FIGURE 14-5: Diving deep into your data lake’s quality and governance.FIGURE 14-6: The ominous results.FIGURE 14-7: Grading your data velocity and latency.FIGURE 14-8: Good news on the data velocity and latency front.FIGURE 14-9: Grading your component architecture.FIGURE 14-10: Bringing together all of your data lake evaluation scores.

15 Chapter 15FIGURE 15-1: The current hospital operational applications.FIGURE 15-2: Peer analytical solutions, one for administrative data and one for...FIGURE 15-3: A downstream data warehouse taking feeds from both Hadoop and AWS.FIGURE 15-4: The current state survey results.FIGURE 15-5: Cataloging and assigning data lake issues.FIGURE 15-6: A two-step process to migrate the hospital’s entire data lake onto...FIGURE 15-7: Introducing streaming to benefit both the medical operations appli...FIGURE 15-8: Adding shells for the silver and gold zones.FIGURE 15-9: Adding a data warehouse component into the overall data lake archi...FIGURE 15-10: Placing master data management in your silver zone.FIGURE 15-11: Addressing the data warehouse–versus–data lake controversy withou...FIGURE 15-12: The data lake remediation timeline.FIGURE 15-13: The inevitable trio of technology, human and organizational facto...

16 Chapter 16FIGURE 16-1: The starting point for the operating room efficiency study.FIGURE 16-2: The first data pipeline to feed existing raw data into a curated g...FIGURE 16-3: Batch ETL of patient bedside data in the current hospital data lak...FIGURE 16-4: Streaming data and streaming analytics for the real-time patient d...FIGURE 16-5: Emergency room data fed through the bronze zone into the silver zo...FIGURE 16-6: Building the first emergency room and inpatient cross-reference wi...FIGURE 16-7: Replacing a batch data feed with split-streaming.FIGURE 16-8: The starting point for analyzing message content versus patient ou...FIGURE 16-9: Building a batch interface between the app and the data lake for m...FIGURE 16-10: Enriching semi-structured data and then repositioning the data in...FIGURE 16-11: Completing the curated data package and the associated analytics.

17 Chapter 17FIGURE 17-1: Dividing your current-state assessment into data and analytics.FIGURE 17-2: Harvey balls for scoring.FIGURE 17-3: Parallel paths of your analytics assessment.FIGURE 17-4: A sample analytics scorecard.FIGURE 17-5: Your data architecture and governance parallel paths.FIGURE 17-6: A sample data architecture and governance scorecard.FIGURE 17-7: Analyzing every scrap of data about an insurance customer: today v...FIGURE 17-8: Your data lake and data warehouse as peers.FIGURE 17-9: Your data warehouse feeding certain data into your data lake.FIGURE 17-10: Progressively turning your data lake vision into a solid blueprin...FIGURE 17-11: A multiphase, multiyear, high-level data lake road map.

18 Chapter 18FIGURE 18-1: Your data lake doing double-duty for transactional and analytical ...FIGURE 18-2: Equipping your data lake with an AI-enabled insights and analytics...

1 Cover

2 Title Page

3 Copyright

4 Table of Contents

5 Begin Reading

6 Index

7 About the Author

1 iii

2 iv

3 1

4 2

5 3

6 4

7 5

8 6

9 7

10 8

11 9

12 10

13 11

14 12

15 13

16 14

17 15

18 16

19 17

20 18

21 19

22 20

23 21

24 22

25 23

26 24

27 25

28 26

29 27

30 28

31 29

32 30

33 31

34 32

35 33

36 34

37 35

38 36

39 37

40 38

41 39

42 40

43 41

44 42

45 43

46 44

47 45

48 46

49 47

50 49

51 50

52 51

53 52

54 53

55 54

56 55

57 56

58 57

59 58

60 59

61 60

62 61

63 62

64 63

65 64

66 65

67 66

68 67

69 68

70 69

71 70

72 71

73 72

74 73

75 74

76 75

77 76

78 77

79 78

80 79

81 80

82 81

83 82

84 83

85 84

86 85

87 86

88 87

89 88

90 89

91 90

92 91

93 92

94 93

95 94

96 95

97 96

98 97

99 98

100 99

101 100

102 101

103 102

104 103

105 104

106 105

107 106

108 107

109 108

110 109

111 110

112 111

113 112

114 113

115 114

116 115

117 116

118 117

119 118

120 119

121 121

122 122

123 123

124 124

125 125

126 126

127 127

128 128

129 129

130 130

131 131

132 132

133 133

134 134

135 135

136 136

137 137

138 138

139 139

140 140

141 141

142 142

143 143

144 144

145 145

146 146

147 147

148 148

149 149

150 151

151 152

152 153

153 154

154 155

155 156

156 157

157 158