Where supervised learning involves matching human-created tags (as in image recognition, for example) and unsupervised learning basically finds patterns in complex data, reinforcement learning models work by trying various strategies to optimise a specified reward function.
Apart from the car itself (which is available for pre-order at US$249, with a regular price of US$399), the DeepRacer system includes a 3D physics simulator in the cloud. Users create their own reward function and train the model in the simulator — using Amazon SageMaker — before deploying it to the car.
SageMaker Neo, available immediately, compiles deep learning models for specific hardware platforms (including those from Nvidia, Intel and Arm), with automatic optimisation providing up to double the performance with no loss of accuracy.
The inaugural event, at AWS re:Invent 2018, gave entrants just 22 hours to develop and test their reinforcement learning models. The winner was Rick Fish, co-founder of UK-based Jigsaw XYZ, who recorded a lap time of 51.50 sec at the specially-constructed track in the MGM Grand Garden Arena.
In related news, lower-level ML services announced by AWS at re:Invent 2018 include new EC2 instances (available next week), a version of TensorFlow optimised to run on AWS (improving GPU utilisation from 65% to 90%, and halving training times compared with the previous market-leading platform), and Amazon Elastic Inference (elastic GPU provisioning for EC2 instances, with pay-for-use pricing delivering a "huge saving").
Jassy also announced that AWS Inferentia, a custom designed, high-performance machine learning inferencing chip. to be available in 2019, Inferentia supports multiple data types and multiple frameworks (including TensorFlow and Pytorch), and delivers hundreds of TFLOPS per chip.
"This is a big gamechanger," said Jassy.
Disclosure: The writer attended AWS re:Invent as a guest of the company.