AI at the Edge: Making Always-on Machine Vision a Reality

SEMI's Serena Brischetto chats with Qualcomm Technologies Senior Director of Engineering Evgeni Gousev about AI at the edge imperatives and devices ahead of the European MEMS & Sensors and Imaging & Sensors Summits. 

Q: Why is it critical to do on-device AI? What role will the cloud play? And 5G (connectivity)?

Gousev: On-device AI is critical for reliability, latency, privacy, and efficient use of network bandwidth. The cloud and 5G will complement the on-device processing. On-device processing makes devices more power efficient and extends battery life.  For these reasons, more and more data processing and analytics will be happening on-device, at the edge. Training will still mostly run in the cloud, but even some of it will move to the device. As 5G connects everything everywhere with the scale of massive IoT, on-device AI will help make sense of all that data. 

Q: What are some example of on-device AI applications?

Gousev: AI is powering a lot of visual and audio intelligence and enables new interesting and valuable use cases. Some examples include:

• Security and home camera: Smart detection for when important activities are happening and not requiring 24/7 video streaming (for example, detect a person rather than a smart vacuum cleaner robot).
• Virtual assistant (smart speaker, phone, etc.): Personalization for natural and intuitive conversations and visual interfaces.
• Phones: Naturally, the smartphone is the pervasive platform for AI. Your phone will detect your context, such as if you are in the car. You can also apply machine learning to smartphones for a better user experience, such as improved power management for better battery life, enhanced photography, and on-device malware detection. And many other examples.
• Smart transportation: On-device AI is beneficial, for example, for sending less data to the cloud in order to know how many seats are available on a bus.
• Industrial IoT: Automating the factory of the future will require lots of AI, from visual inspection for defects and intricate robotic control for assembly.
• Drones/robots: Self navigation in unknown environment as well as coordination with other drones/robots.
• Auto: Machine learning for passenger safety, scene understanding, sensor fusion, path planning, etc. The huge, real benefit of autonomous driving is saving lives and time.

Gousev: Yes, many applications would like to have always-on computer vision, such as smartphones, toys, and security cameras. Ultra-low power AI capabilities are critical for the majority battery-operated devices. An energy-efficient solution consists of a very low-power computer vision processor for basic processing that would trigger turning on a processor with more compute when appropriate. This overall saves power and gives the feel of always-on intelligence – it provides the best of both worlds.

Q: What role will Qualcomm play in AI and making it ubiquitous across devices? Do you have some specific examples?

Gousev: Qualcomm continues to innovate and revolutionize the connected world. Qualcomm technologies are in the “hearts” of the modern mobile platform, and we supply chips and systems across industries and devices. Industrial and consumer IoT shows significant growth too, especially in the battery-operated domain.

The mobile industry unique in its ability to provide tech innovations to other industries, and it will be no different with spreading AI technology due to mobile’s:

• Unmatched scale: The smartphone has been the largest platform for computer vision technologies, accelerating development. Gartner predicts greater than 8.6 billion cumulative smartphone shipments between 2018 and 2022.
• Rapid replacement cycles: Consumers replace their phones every two to three years.
• Integrated and optimized technologies: Mobile is very challenging design environment. It requires small size, high performance, and long battery life. With scale and rapid replacements rates, new technologies quickly proliferate and create a large user base.

Q: How are you going to make these complex AI inferences run on the device when they have traditionally run on the cloud?

Our heritage in low-power processing (heterogeneous computing, system solutions, hardware innovations, power-efficient algorithms and software, latest process node, etc.) and connectivity (3G, 4G, 5G, Wi-Fi, Bluetooth, etc.) is essential for AI.

We are focused on three fronts to advance AI to the next, always-on levels:

• Specialized hardware: Low-power hardware, enhanced architectures and dedicated acceleration, to handle machine learning workloads.
• Algorithmic advancements: Research related to power efficiency, training with semi-supervised approaches, data sparsity, distributed learning, privacy protecting approaches, etc.
• Network optimizations: Research related to compression, inter-layer optimizations, and other techniques to take better advantage of memory and space/time complexity.

Serena Brischetto is a marketing and communications manager at SEMI Europe.