Short CourseBeginner1 Hour 1 Minute

Introduction to On-Device AI

Instructor: Krishna Sridhar

Qualcomm
  • Beginner
  • 1 Hour 1 Minute
  • 7 Video Lessons
  • 5 Code Examples
  • Instructor: Krishna Sridhar
    • Qualcomm
    Qualcomm

What you'll learn

  • Learn to deploy AI models on edge devices like smartphones, using their local compute power for faster and more secure inference.

  • Explore model conversion by, converting your PyTorch/TensorFlow models for device compatibility, and quantize them to achieve performance gains while reducing model size.

  • Learn about device integration, including runtime dependencies, and how GPU, NPU, and CPU compute unit utilization affect performance.

About this course

As AI moves beyond the cloud, on-device inference is rapidly expanding to smartphones, IoT devices, robots, AR/VR headsets, and more. Billions of mobile and other edge devices are ready to run optimized AI models. 

This course equips you with key skills to deploy AI on device:

  • Explore how deploying models on device reduces latency, enhances efficiency, and preserves privacy.
  • Go through key concepts of on-device deployment such as neural network graph capture, on-device compilation, and hardware acceleration.
  • Convert pretrained models from PyTorch and TensorFlow for on-device compatibility.
  • Deploy a real-time image segmentation model on device with just a few lines of code.
  • Test your model performance and validate numerical accuracy when deploying to on-device environments
  • Quantize and make your model up to 4x faster and 4x smaller for higher on-device performance.
  • See a demonstration of the steps for integrating the model into a functioning Android app.

Learn from Krishna Sridhar, Senior Director of Engineering at Qualcomm, who has played a pivotal role in deploying over 1,000 models on devices and, with his team, has created the infrastructure used by over 100,000 applications.

By learning these techniques, you’ll be positioned to develop and deploy AI to billions of devices and optimize your complex models to run efficiently on the edge.

Who should join?

This course is designed for beginner AI developers, ML engineers, data scientists, and mobile developers looking to deploy optimized models on edge devices. Familiarity with Python, as well as PyTorch or TensorFlow is recommended.

Course Outline

7 Lessons・5 Code Examples
  • Introduction

    Video4 mins

  • Why on-device

    Video5 mins

  • Deploying Segmentation Models On-Device

    Video with code examples15 mins

  • Preparing for on-device deployment

    Video with code examples14 mins

  • Quantizing Models

    Video with code examples13 mins

  • Device Integration

    Video6 mins

  • Conclusion

    Video1 min

  • Appendix - Building the App

    Code examples1 min

  • Appendix - Tips and Help

    Code examples1 min

Instructor

Krishna Sridhar

Krishna Sridhar

Senior Director of Engineering at Qualcomm

Course access is free for a limited time during the DeepLearning.AI learning platform beta!

Want to learn more about Generative AI?

Keep learning with updates on curated AI news, courses, and events, as well as Andrew’s thoughts from DeepLearning.AI!