Artificial Intelligence & Machine Learning. Elena Ehrlich, PhD

Artificial Intelligence & Machine Learning Elena Ehrlich, PhD eeehrlic@amazon.com

What is AI? https://www.geospatialworld.net/blogs/difference-between-ai%ef%bb%bf-machine-learning-and-deep-learning/

Agenda Image & Video Recognition Rekognition Deep-Learning Enabled Video Cameras DeepLens Natural Language Understanding Comprehend Voice & Convseration Bots Polly, Lex, & Alexa Fully-Managed Machine Learning Sagemaker

Image Analysis AWS Rekognition

Rekognition: Object & Scene Detection Bay Beach Coast Outdoors Sea Water Palm_tree Plant Tree Summer Landscape Nature Hotel Category 99.18% 99.18% 99.18% 99.18% 99.18% 99.18% 99.21% 99.21% 99.21% 58.3% 51.84% 51.84% 51.24% Confidence

Rekognition: Facial Analysis { "contentstring": } { } "Attributes": [ ], "ALL" "Image": { } "Bytes": "..." DetectFaces "FaceDetails": [{ "BoundingBox": { "Height": 0.22111110389232635, "Left": 0.29600000381469727, "Top": 0.08888889104127884, "Width": 0.4000000059604645 }, "Confidence": 99.9970474243164, "Emotions": [{ "Confidence": 98.48326110839844, "Type": "HAPPY" }, { "Confidence": 15.214723587036133, "Type": "CALM" }, { "Confidence": 1.2157082557678223, "Type": "CONFUSED" }], "AgeRange": { "High": 47, "Low": 30 }, "Beard": { "Confidence": 95.77610778808594, "Value": false }, "Eyeglasses": { "Confidence": 99.68527221679688, "Value": true }, "EyesOpen": { "Confidence": 99.99991607666016, "Value": true }, "Gender": { "Confidence": 99.92896270751953, "Value": Female" }, "MouthOpen": { "Confidence": 99.90928649902344, "Value": true }, "Mustache": { smart cropping & ad overlays sentiment capture demographic analysis face editing & pixelation

Rekognition: Compare Faces Face Comparision

Rekognition: Image Moderation Confidence score Hierarchical taxonomy Suggestive 82.7% Female Swimwear or Underwear 82.7% Nudity and Sexuality 50.1% Covered Nudity 50.1% "ModerationLabels": [ { "Confidence": 82.7555923461914, "Name": "Suggestive", "ParentName": "" }, { "Confidence": 82.7555923461914, "Name": "Female Swimwear or Underwear", "ParentName": "Suggestive" }, { "Confidence": 50.11056137084961, "Name": "Covered Nudity", "ParentName": "Nudity and Sexuality" }, { "Confidence": 50.11056137084961, "Name": "Nudity and Sexuality", "ParentName": "" }, ]

Interesting Demos Time permitting https://console.aws.amazon.com/rekognition/home?region=us-east-1#/label-detection https://d3qtbfbtl5c95j.cloudfront.net/main.html http://iad-front.deepvideoanalysis.cloud/results.html#!?identifier=introducingamazongo.mp4 https://aws.amazon.com/rekognition/customers/

Deep-Learning Enabled Video Cameras AWS DeepLens

DeepLens: Deep-Learning Enabled Video Camera A DL video camera uses deep convolutional neural networks (CNNs) to analyze visual imagery. The device itself is a development environment to build computer vision applications. April 2018 AWS DeepLens communicates with the following ML endpoints: Amazon SageMaker, for model training and validation AWS Lambda, event-driven triggers run inference against CNN models AWS Greengrass, for deploying updates and functions to your device and other IoT devices

Natural Language Understanding AWS Comprehend

Comprehend: Keyword, Sentiment, & Topic Modeling https://www.ip-watch.org/2018/01/24/itu-4-5-people-ldcs-can-access-mobile-networks-not-using-internet/

Comprehend: Keyword, Sentiment, & Topic Modeling

Life-like Speech AWS Polly

Polly: Life-like Speech Service Plain Text SSML Lexicons Speech Synthesis Markup Language <speak> - Start Tag <break> - Pause in Speech <lang> - Specifies the language <mark> - Tag Name for specific word <p> - Indicates Paragraph <phoneme>- phonetic pronunciation <prosody> - Controls the volume <s> - Indicates a sentence <say-as>- Interpretation <sub> - Alias words <w> - Customize pronunciation <amazon:effect name="whispered"> Plain Text SSML <lexeme> <grapheme>espresso</grapheme > <alias>ess-press-oh</alias> </lexeme> Lexicons

Conversational Engines AWS Lex

Lex: The Advent Of Conversational Interactions 1st Gen: Machine-oriented interactions Availability 2nd Gen: Control-oriented & translated 3rd Gen: Intent-oriented Testing Speech Recognition Business Logic Language Understanding Security Mobile Disparate Systems Messaging platforms Scale Authentication

Lex Converstaional Engines Operational Bots Chatbots for IT automation Informational Bots Chatbots for everyday consumer requests Application Bots Build powerful interfaces to mobile applications Enterprise Productivity Bots Streamline enterprise work activities and improve efficiencies Reset my Password TCO analysis Productivity. News updates Weather information Game scores. Book tickets Order food Manage bank accounts. Check sales numbers Marketing performance Inventory status. Internet of Things (IoT) Bots Enable conversational interfaces for device interactions Wearables Appliances Auto.

Sagemaker: Fully-Managed Machine Learning Most Common Algorithms Provided Linear Learner Factorization Machines XGBoost Algorithm Image Classification Algorithm Amazon SageMaker Sequence2Sequence K-Means Algorithm Principal Component Analysis (PCA) Latent Dirichlet Allocation (LDA) Neural Topic Model (NTM) DeepAR Forecasting BlazingText import sagemaker 10x Performance Single-Click Training OpenSource tools TensorFlow Apache MXNet A/B Testing Built-in Train Models at Petabyte Scale Deploy in Production Auto-Scaling Cluster of AWS EC2 Instances

AI/ML Adoption Benefits 1 MAKING THE BEST USE OF A DATA SCIENTISTS TIME While the power of ML is unrivaled, data scientists spend around 80% of their time on preparing and managing data for analysis hence only 20% of their time is used to derive insights 2 CONVERTING THE POWER OF MACHINE LEARNING INTO BUSINESS VALUE While 60% of companies agree that big data will help improve their decision making and competitiveness only 28% indicate that they are currently generating strategic value from their data 3 EMBEDDING MACHINE LEARNING INTO THE FABRIC OF YOUR BUSINESS The value of data science relies upon operationalizing models within business applications and processes, yet 50% of the predictive models [built] don t get implemented

Thank you Elena Ehrlich, PhD eeehrlic@amazon.com

Appendix

AWS AI/ML: The Stack Services Speech: Polly & Transcribe Vision: Rekognition Image & Rekognition Video Language: Lex, Translate & Comprehend Platforms SageMaker AWS DeepLens Amazon ML Spark & EMR Mechanical Turk Frameworks Apache MXNet Tensor- Flow Caffe2 & Caffe AWS Deep Learning AMI Cognitive Toolkit PyTorch Gluon Keras

AWS AI/ML: Notable Successes Services Platforms Frameworks

AWS AI/ML: Solutions for Every Skill Level Services Designed for Developers & Data Scientists Solution-oriented Prebuilt Models Available via APIs Image Analysis, NLU, NLP, Translation, Text-to-Speech & Speech-to-Text Platforms Designed for Data Scientists to Address Common Needs Fully Managed Platform for Model Building Reduces the Heavy Lifting in Model Building & Deployment Frameworks Designed for Data Scientists to Address Advanced / Emerging Needs Provides Maximum Flexibility to develop on the leading AI Frameworks Enables Expert AI Systems to be Developed & Deployed

Rekognition: Search & Understand Visual Content Real-time & batch image analysis Object & Scene Detection Facial Detection Facial Analysis Face Search Image Moderation Celebrity Recognition

Rekognition: Image Moderation Upload picture Object Creation Lambda Users S3 Bucket Rekognition No inappropriate content detected Inappropriate content detected Rejected Picture posted to end users Approved Manual Review User Notification

Rekognition: Video - Case Study Architecture

Polly: Life-like Speech Service Converts text to life-like speech Fully managed 47 voices 27 languages Low latency, real time

Lex: Build Natural, Conversational Interactions In Voice & Text Voice & Text Chatbots Powers Alexa Voice interactions on mobile, web & devices Text interaction with Slack & Messenger (with more coming) Enterprise Connectors Salesforce Microsoft Dynamics Marketo Zendesk Quickbooks Hubspot

Lex: Build Natural, Conversational Interactions In Voice & Text BOT Intent Slot & Slot type An Amazon Lex bot is powered by Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) capabilities An intent represents an action that the user wants to perform Intent name A descriptive name for the intent. Sample utterances How a user might convey the intent. How to fulfill the intent How you want to fulfill the intent after the user provides the necessary information Slot - An intent can require zero or more slots or parameters Slot type Each slot has a type. You can create your custom slot types or use built-in slot types BOT Intent Slot & Slot type

Response Cards Simplify interactions for your users Increase bot's accuracy Can be used with Facebook Messenger, Slack, and Twilio as well as your own client applications.

DeepLens Architecture

IoT Anomaly Detection AWS Kinesis Analytics

Kinesis Analytics: real-time insights from streaming data

AI/ML Assessment AI Inquisitors AI Adopters AI Experts Interested in AI but have limited expertise and/or resources Limited expertise and/or use of AI for one-off projects Advanced expertise and/or use of embedded AI in apps

Assessing POC Targets: Criteria Business Value Ability to Execute Data Availability

AI/ML Assessment Prep Question Sample Answer What Business or Operational benefits are you trying to drive? Improve content personalization How will you consume the outputs and put them into action? What types of data is available today? Where does the data reside? What types of analytics and/or machine learning are being employed today? What staff and/or consultants currently support these activities? Content will be distributed at a targeted level Content and subscription data Business Intelligence Predictive Analytics Data Engineers Data Scientists What software currently supports these activities? R / Python What is your ideal scenario in tackling these business objectives? One-to-one content for individuals What challenges have you experienced when deploying AI? Prioritization of Targets Operationalization