Data Science: The Big #SQLServerUserGroupDubai

Size: px

Start display at page:

Download "Data Science: The Big #SQLServerUserGroupDubai"

Iris Goodman
6 years ago
Views:

1 Data Science: The Big #SQLServerUserGroupDubai

6 Job Postings for Data Scientists

7 Top-paying Tech Skills Skill 2016 Change Skill 2016 Change Source: Dice Salary Survey 2017

12 What is data science? Why is it important? Where is this all going?

13 What is data science?

14 Computer Science Data Science Math and Statistics Domain Knowledge

15 What Is a Data Scientist? Performs data science More than a scientist More than an analyst More than a developer

16 What skills are necessary?

17 Data Science Skills Programming Working with data Descriptive statistics Data visualization

18 Data Science Skills Programming Working with data Descriptive statistics Data visualization Statistical modeling Handling Big Data Machine learning Deploying to production

19 What tools are used?

20 SQL Excel Python R MySQL Python tools ggplot SQL Server Tableau JavaScript Matplotlib Java PostgreSQL Oracle D3 Homegrown Hive Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala SQLite C Pig RedShift Weka Hbase (EMR) Perl SPSS Teradata Share of Respondents 70% 60% Data Science Tools 50% 40% 30% 20% 10% 0% Tool: language, platform, analytics Source: O Reilly 2015 Data Science Salary Survey

SQL Excel Python R MySQL Python tools ggplot SQL Server Tableau JavaScript Matplotlib Java PostgreSQL Oracle D3 Homegrown Hive Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala

21 SQL Excel Python R MySQL Python tools ggplot SQL Server Tableau JavaScript Matplotlib Java PostgreSQL Oracle D3 Homegrown Hive Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala SQLite C Pig RedShift Weka Hbase (EMR) Perl SPSS Teradata Share of Respondents 70% 60% Data Science Tools 50% 40% 30% 20% 10% 0% Tool: language, platform, analytics Source: O Reilly 2015 Data Science Salary Survey

22 SQL Excel Python R MySQL Python tools ggplot SQL Server Tableau JavaScript Matplotlib Java PostgreSQL Oracle D3 Homegrown Hive Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala SQLite C Pig RedShift Weka Hbase (EMR) Perl SPSS Teradata Share of Respondents 70% 60% Data Science Tools 50% 40% 30% 20% 10% 0% Tool: language, platform, analytics Source: O Reilly 2015 Data Science Salary Survey

Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala

Respondents 70% 60% Data Science Tools 50% 40% 30% 20% 10% 0% Tool:

23 SQL Excel Python R MySQL Python tools ggplot SQL Server Tableau JavaScript Matplotlib Java PostgreSQL Oracle D3 Homegrown Hive Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala SQLite C Pig RedShift Weka Hbase (EMR) Perl SPSS Teradata Share of Respondents 70% 60% Data Science Tools 50% 40% 30% 20% 10% 0% Tool: language, platform, analytics Source: O Reilly 2015 Data Science Salary Survey

24 SQL Excel Python R MySQL Python tools ggplot SQL Server Tableau JavaScript Matplotlib Java PostgreSQL Oracle D3 Homegrown Hive Spark Cloudera Visual Basic MongoDB Hadoop SAS C++ PowerPivot Scala SQLite C Pig RedShift Weka Hbase (EMR) Perl SPSS Teradata Share of Respondents 70% 60% Data Science Tools 50% 40% 30% 20% 10% 0% Tool: language, platform, analytics Source: O Reilly 2015 Data Science Salary Survey

25 How is data science performed?

26 The Data Science Process Data

27 The Data Science Process Find a question Data

28 The Data Science Process Find a question Collect the data Data

29 The Data Science Process Find a question Collect the data Data Prepare the data

30 The Data Science Process Find a question Collect the data Data Prepare the data Create a model

31 The Data Science Process Find a question Collect the data Evaluate the model Data Prepare the data Create a model

32 The Data Science Process Find a question Deploy the model Collect the data Evaluate the model Data Prepare the data Create a model

33 The Data Science Process Find a question Deploy the model Collect the data Evaluate the model Data Prepare the data Create a model

34 The Data Science Process Find a question Iterative process Deploy the model Explore the data Evaluate the model Data Prepare the data Create a model

35 The Data Science Process Find a question Iterative process Non-sequential Deploy the model Explore the data Evaluate the model Data Prepare the data Create a model

36 The Data Science Process Find a question Iterative process Deploy the model Explore the data Non-sequential Early termination Evaluate the model Data Prepare the data Create a model

37 Why is data science important?

38 Data Analytics

39 Internet of Things Data Analytics

40 Internet of Things Data Analytics Big Data

41 Internet of Things Data Analytics Machine Learning Big Data

42 Internet of Things Data Analytics Machine Learning Big Data

43 Trends Past Present Future

44 Driven by economics Possible by technology

45 Cost Cost Value

46 Internet of Things Data Analytics Machine Learning Big Data

47 Internet of Things Data Analytics Machine Learning Big Data

48 Data Analysis (The Past)

53 Collecting, analyzing, and communicating data was difficult, expensive, and slow.

54 Data Analytics (The Present)

62 Collecting, analyzing, and communicating data is easy, inexpensive, and fast.

63 Data-Driven Decision Making (The Future)

68 Retail Sales Total Products Total Sales Sales by Product Type New Products This Year Annual Sales Comparison by Month Sales per Square Foot by Total Sales Variance and District

69 Internet Sales Show me sales by gender and marital status. Displaying sum of sales by gender and marital status Marital Status: Married Single Show me sales by gender and marital status. Male Female $0k $5k $10k $15k

73 4% higher productivity 6% higher profits Source:

75 Empowering people to make better decisions is not the end goal it s just the beginning

76 Internet of Things Data Analytics Machine Learning Big Data

77 The Internet (The Past)

79 Cost Speed Bandwidth

82 The internet was expensive, slow, and not generating much data.

83 Internet of Things (The Present)

85 Cost Speed Bandwidth

90 Billions of Devices Growth of the IoT Devices Year Source: NCTA, 2014

91 50 billion IoT devices by 2020

93 The internet of things is cheap, fast, and generating tons of data.

94 Internet of Everything (The Future)

95 Cost Speed Bandwidth

97 An internet connection will likely be as common to devices as electricity.

99 We re building a peripheral nervous system for our planet... but it needs a brain.

100 FUN GAME 1 Is It IoT?

101 Is it IoT?

102 YES!

103 Is it IoT?

104 YES!

105 Is it IoT?

106 YES!

107 Is it IoT?

108 NO : (

109 Internet of Things Data Analytics Machine Learning Big Data

110 Data (The Past)

111

112

113

114

115

116 Data sets were small, slow, and had little diversity.

117 Big Data (The Present)

118 Data in Zettabytes (ZB) Global Data Growth Year Source: UNECE Statistics Wikis

119 Doubling every two years

120 Volume Velocity Big Data Variety

121 Volume Velocity Big Data Variety

122 Volume

123

124

125 Velocity

126

127

128 Variety

129 INTEGRATION

130 Gender: Female Age: 31 Emotion: Happy Gender: Male Age: 5 Emotion: Happy Apple

131 Today s data sets are bigger, faster, and more diverse.

132 Just Data Again (The Future)

133 Cost

134

135

136

137 We re creating new tools to automate feature extraction... but it requires machine learning.

138 FUN GAME 2 Am I Smarter Than Big Data?

139 What Do These Have In Common?

140

141 What Do These Three Things Predict?

142

143

144

145 Internet of Things Data Analytics Machine Learning Big Data

146 Artificial Intelligence (The Past)

147 Source: Evan Amos

148

149

150

151 AI Winter has ended and things are warming up again.

152 Machine Learning (The Present)

153 Artificial Intelligence Machine Learning Statistics

154 f x

155 f x Data Function Prediction

156 f x Data Function Prediction Cat Dog

157 f x Data Function Prediction Cat Dog Is cat?

158 f x Data Function Prediction Cat Dog Is cat? Yes

159

160

161

162

163

164 The next generation of ML will be able to complete even more complex tasks.

165 Deep Learning (The Future)

166 Deep Learning Human Cat Dog Car

167 Deep Neural Network input hidden 1 hidden 2 hidden 3 output

168 Deep Neural Network input hidden 1 hidden 2 hidden 3 output

169 Deep Neural Network input hidden 1 hidden 2 hidden 3 output

170 Deep Neural Network input hidden 1 hidden 2 hidden 3 output

171 Deep Neural Network John Jane Miko Lee input hidden 1 hidden 2 hidden 3 output

172

173

174 f x

175

176

177 AI Winter 2.0?

178 AI AI Winter 2.0? 2.0? or Human Winter 1.0?

179 FUN GAME 3 Dog or Mop?

180 MOP!

181 DOG!

182

183 DOG!

184 MOP!

185 DOG! MOP!

186

187

188 Closing the Loop (The Future of Data Science)

189 Data Analytics

190 Internet of Things Data Analytics

191 Internet of Things Data Analytics Big Data

192 Internet of Things Data Analytics Machine Learning Big Data

193 Internet of Things Data Analytics Machine Learning Big Data

194 Internet of Things Data Analytics Machine Learning Big Data

195 Internet of Things Data Analytics Machine Learning Big Data

196 Fully Autonomous Systems

197 Smart systems Cloud robotics Cyber-physical systems

198

199

200

201

202 Embedded intelligence will be woven into the fabric of our society.

203

204 The Big Data Universe Amount of data as of 2016 in petabytes Human brain 2.5 PB Ebay 90 PB Google 15,000 PB (estimated) Spotify 10 PB Facebook 300 PB Source: The Royal Society, 2016

205

206

207 Complexity High Automation Framework Routine & Complex Non-routine & Complex Routine & Simple Non-routine & Simple Low High Low Repetitiveness Source: Abhas Gupta -The Automation Framework

208 Complexity High Automation Technology Deep Learning High-level Programming Conventional Machine Learning Low High Low Repetitiveness Source: Abhas Gupta -The Automation Framework

209 Complexity High Retail Salesperson Fold Clothes Greet Customers Convert Customers Sizing Inventory Count Inventory Pull Cash Register Low High Low Repetitiveness Source: Abhas Gupta -The Automation Framework

210 Complexity High Medical Doctor Treatment Plan Diagnose Disease Rare Disease Build Trust Routine Check-up Input EMR Write Prescription Low High Low Repetitiveness Source: Abhas Gupta -The Automation Framework

211 Source: CGP Grey

212 Industrial Robots (per 1000 US workers) Rise of the Robots Year Source: International Federation of Robotics

213 Data science will amplify this trend

214

215 Which side of this new economy will your job be on? The side that s leading or the side being eliminated.

216

217

218

219

220 What will you choose?

221 Welcome Robot Overlords!!!

222

223 Where to We Go Next?

224

225 Where to Go Next Pluralsight: Coursera: Data Camp:

226 Recommended Courses Data Science: The Big Picture Data Science with R Exploratory Data Analysis with R Data Visualization with R (3-part)

227

228 Feedback Very important to me! What did you like? What could I improve?

229 Conclusion

230 Internet of Things Data Analytics Machine Learning Big Data

231 Are you prepared? Is your organization? Is our world prepared?

232

233 Thank You! Matthew Renze Data Science Consultant Renze Consulting Website:

DevSci: Better Software Through Data #KCDC2018

DevSci: Better Software Through Data Science @MatthewRenze #KCDC2018 What is data science? Why is it important? How do I get started? Job Postings for Data Scientists Top-paying Tech Skills Skill