All the Machine Learning Announcements at AWS re:Invent 2020

Feeling a little overwhelmed by all the Machine Learning announcements so far? Here’s your quick catch-up…

AWS re:Invent Machine Learning Keynote

The Themes

1. Anomaly detection is a big theme — lots of new services (Amazon Lookout for Metrics, Vision and Equipment, DevOps Guru, AWS IoT Device Defender ML Detector, Amazon Monitron, AWS Cost Anomaly Detection…) and integrations to make this easier across the board in everything from your own data to industrial equipment.
2. AWS ML moves further up the stack — previously the top of the stack was the managed AI Services, we’re seeing a level above that now with products that package multiple services (Monitron, HealthLake) to make it easier for anyone to use them.
3. Responsible AI edges in there with a new bias detection functionality in SageMaker (Clarify). Let’s hope this might start to extend the conversation further into ethics and things like carbon-neutral data centres! (edit: at the AWS re:Invent Infrastructure Keynote covered a lot of information around efforts in renewable energy and sustainability, which was great to hear).

The Actual Announcements

There has been a lot announced — in the run-up to re:Invent, at Andy Jassy’s keynote, at the first machine learning focussed keynote with Swami Sivasubramanian, and even just quietly dropped out there. Here they are all in the one place to save you searching!

Instances

• AWS Trainium — high performance machine learning training chip, custom designed by AWS to offer cost effective and high performance for Machine Learning Training. This uses the same AWS Neuron SDK that the AWS Inferentia instances announced last year use.
• Amazon EC2 instances powered by Habana Gaudi — specifically designed for training deep learning models, these can offer up to 40% better price performance for deep learning models.
• Improved Tagging support for Amazon Machine Images and EBS Snapshots — new support for tag-on-create and tag-based access control.
• Amazon EC2 P4d instance support in Amazon SageMaker.

Amazon SageMaker

• SageMaker Clarify — tooling to provide greater visibility into training data and models to identify and limit bias and explain predictions.
• SageMaker Data Wrangler — a single visual interface for data preparation workflow, including data selection, cleansing, exploration, and visualisation with over 300 built-in data transformations so you can quickly normalise, transform, and combine features without having to write any code
• SageMaker Feature Store — fully managed, purpose-built repository to store, update, retrieve, and share machine learning features.
• SageMaker Pipelines — continuous integration and continuous delivery (CI/CD) service for machine learning, allowing you to create, automate, and manage end-to-end ML workflows at scale.
• SageMaker Distributed Training — easily add data parallelism or model parallelism to your PyTorch and TensorFlow training scripts for faster training.
• SageMaker Edge Manager — model management and monitoring across fleets of edge devices such as smart cameras, robots, personal computers, and mobile devices.
• Deep Profiling added to SageMaker Debugger — new capability to visually profile and monitor hardware and system resource utilization in during model training.
• Additional SageMaker Model Monitoring capabilities — new capabilities to detect drift in model quality, model bias, and feature importance in running production models.
• SageMaker JumpStart — quick-start set of one-click deployable solutions for over 150 pre-trained models.
• Secure SageMaker Studio Access available from Amazon Virtual Private Cloud (VPC) through AWS PrivateLink, allowing access without going through the public internet.
• Amazon SageMaker Autopilot adds Deep Learning Models allowing for the automatic creation of accurate models for problems with multi-dimensional, multi-class datasets.

Data, Search and Analytics

• Amazon Redshift ML — create, train, and apply machine learning models using familiar SQL commands in Amazon Redshift data warehouses using SageMaker Autopilot.
• Amazon Neptune ML — run machine learning directly using Neptune APIs and queries using Graph Neural Networks (GNNs) purpose-built for graph data.
• Amazon Quicksight Q — automatically understands the relationships across your data using ML, allowing for natural language business questions about your data to be answered instantly.
• New Connector Library for Amazon Kendra — new native and partner connectors, including Google Drive, can now be indexed and searched through Kendra.
• Incremental Learning for Amazon Kendra — automatic incremental learning to continuously tune and optimize search results based on end-user search patterns and feedback.
• Custom synonyms in Amazon Kendra – expand the search vocabulary with your own specific business terminology
• Amazon HealthLake (Preview) — secure HIPAA-eligible data lake designed for healthcare providers, health insurance companies, and pharmaceutical companies. It automatically uses specialised machine learning models, like natural language processing, to automatically extract meaningful medical information from the data and provides powerful query and search capabilities to remove the heavy lifting of organising, indexing, and structuring patient information to provide a complete view of the health of individual patients and entire patient populations in a secure, compliant, and auditable manner.
• Amazon Lookout for Metrics — automatically detect, prioritise and diagnose anomalies in time series data from various Amazon data stores and third-party SaaS applications. This pairs with Amazon AppFlow to get data in a few clicks from those data sources.
• Accuracy measurements for individual items in Amazon Forecast — view accuracy for individual items and export forecasts generated during training by being able to access the forecasted values from Forecast’s internal testing of splitting the data into training and backtest data groups to compare forecasts versus observed data and item-level accuracy metrics.
• Weather Index added to Amazon Forecast — automatically including the latest local weather information to your demand forecasts with one click and at no extra cost.
• Amazon EMR Studio (Preview)— integrated development environment (IDE) running on EMR clusters for data scientists and data engineers, providing fully managed Jupyter Notebooks, and tools like Spark UI and YARN Timeline Service and the ability to execute parameterized notebooks as part of scheduled workflows using orchestration services like Apache Airflow or Amazon Managed Workflows for Apache Airflow.

Development/Application Performance

• Amazon DevOps Guru (Preview)— automatic anomaly detection in your application’s operational performance and availability, with recommendations and a visualisation dashboard.
• Memory Profiling in Amazon CodeGuru — new capabilities added to profile your application’s memory, giving you a consolidated view of the heap, both as a summary and a time series.
• Python support in Amazon CodeGuru — now Python code as well as Java can be Reviewed and Profiled.
• Security Detectors in Amazon CodeGuru — help identify security risks from the top ten Open Web Application Security Project (OWASP) categories, security best practices for AWS APIs, and common Java crypto libraries.
• Code Quality Detector in Amazon CodeGuru — new metrics to help manage technical debt and codebase maintainability — method source lines of code, cyclomatic complexity, method fan out, class fan out and class cohesion.
• Tag Support in Amazon CodeGuru

Industrial/Manufacturing/Commercial

• Amazon Lookout for Vision — service to find visual defects in industrial products, using computer vision to identify missing components in products, damage to vehicles or structures, irregularities in production lines, and even miniscule defects in silicon wafers — or any other physical item where quality is important.
• Amazon Lookout for Equipment (Preview) — service which provides customers with existing sensors on their industrial equipment, a way to send their sensor data to AWS to build machine learning models for them and return predictions to detect abnormal equipment behavior.

Devices

• Amazon Monitron — end-to-end system that uses ML to detect abnormal behaviour in industrial machinery. It bundles sensors, a gateway device to transfer sensor data to the cloud, the Monitor service itself running on that data, and a companion mobile app to set up the devices and receive reports on operating behaviour and alerts to potential failures in your machinery. It requires no specialist knowledge to set up or use.
• AWS Panorama Appliance — a machine learning appliance that allows you to to add CV to your existing IP cameras, and automate tasks that traditionally required human inspection and monitoring.
• AWS Panorama SDK — allows device manufacturers to add AWS Panorama to their own edge devices and smart cameras.
• New divisions, rewards, and community leagues for AWS DeepRacer League
• New ML Detector feature for AWS IoT Device Defender (Preview) — monitor your IoT devices by identifying and alarming on anomalous datapoints (e.g., authorization failure counts, message sent counts) using ML models that are automatically retrained on your device data daily.
• AWS DeepComposer adds a new learning capsule that dives deep into Transformer models.

Amazon Connect

• Amazon Connect Wisdom (Preview) — natural language search for agents across connected repositories, and real-time speech analytics to detect customer issues during calls and provide agents recommendations and answers.
• Amazon Connect Customer Profiles — automatically brings together customer information from multiple applications into a unified customer profile
• Real-Time Contact Lens For Amazon Connect — extending the current Contact Lens functionality by adding the ability to get alerted to issues during live customer calls and deliver proactive assistance to agents while calls are in progress.
• Amazon Connect Tasks — allows agents to create and complete tasks in the same user interface they take calls and chats.
• Amazon Connect Voice ID (Preview) — real-time caller authentication using ML-powered voice analysis.
• Added inbound telephony for Argentina, Chile, Mexico, and Peru.
• Reduced telephony rates in Argentina, Chile, Mexico, and Peru from the US East (N. Virginia) and US West (Oregon) regions.
• Apple Business Chat support (Preview) — provide convenient and familiar customer service directly through the Apple Messages app
• Support for Amazon Lex chatbots with Latin American Spanish and German

Conversation and Text

• Amazon Lex additional language support — German and American Spanish added.
• Amazon Transcribe streaming transcription support extended — Brazilian Portuguese, Japanese and Korean in the South America (Sao Paulo), Asia Pacific (Tokyo) and Asia Pacific (Seoul) regions now supported.
• Amazon Transcribe adds native support for Ogg opus and FLAC encoded audio
• Amazon Transcribe Medical extended specialities — new streaming transcription support for additional medical specialties: cardiology, oncology, neurology, radiology, and urology
• Amazon Translate additional language support — Armenian, Catalan, Gujarati, Haitian, Icelandic, Kannada, Kazakh, Lithuanian, Malayalam, Macedonian, Maltese, Mongolian, Sinhala, Telugu, Uzbek, and Welsh added.

Other

• AWS Cost Anomaly Detection becomes Generally Available — a free service that using ML to monitor your spending patterns within your AWS Account and detect anomalous spend and provide root cause analysis.

Note: This list covers everything from Nov 24th to Dec 21st – there are of course lots of new features released continuously through the year.

To see some reactions to the Machine Learning keynote you can watch Mike Chamber get together AWS Heroes immediate reactions in this video, read Luca Bianchi’s summary or Zamira Jaupaj’s live blog post.

I think the overall message is still the same as we have seen for the past few years — AWS continues on its goal of making machine learning as expansive and impactful as it can be by not just creating low-level tooling, but creating a wide array of tools and packages that make it easier for increasingly more and more people to access and use machine learning in their own environments.

If you are interested in machine learning and AWS, do connect with me on twitter and let’s have a chat!