Meta Powers Billions of Users with On-Device AI via ExecuTorch

Framework Enhances Performance, Privacy, and Latency Across Instagram, WhatsApp, and Messenger

Meta is significantly boosting the capabilities of its applications by deploying its open-source inference framework, ExecuTorch, directly onto user devices. This strategic move is enhancing user experiences on platforms like Instagram, WhatsApp, and Messenger, impacting billions globally.

ExecuTorch Drives On-Device Innovation

ExecuTorch, a collaborative effort involving tech giants such as Arm, Apple, and Qualcomm, is Meta’s solution for running machine learning (ML) models on edge devices. Implementing on-device ML is crucial for Meta’s family of apps, as it slashes latency, protects user privacy by keeping data local, and ensures functionality even offline.

Over the past year, Meta has integrated ExecuTorch across its popular applications, reporting substantial improvements in model performance, enhanced privacy safeguards, and reduced latency compared to its previous mobile ML stack. The framework leverages PyTorch 2.x technologies to transform models into efficient, compact formats suitable for on-device deployment.

Instagram Enhances Creativity with ExecuTorch

Instagram’s new “Cutouts” feature, which allows users to create personalized stickers from photos and videos, now runs on ExecuTorch. The integration of SqueezeSAM, a streamlined version of Meta’s Segment Anything Model, has led to marked performance gains on both Android and iOS devices. This speed increase has directly translated into higher daily active user engagement for the Cutouts feature.

WhatsApp Optimizes Calls with ExecuTorch

To ensure seamless performance regardless of network conditions, WhatsApp has developed bandwidth estimation models utilizing ExecuTorch. These models optimize video streaming and call quality by intelligently adapting to available network bandwidth. ExecuTorch integration has significantly reduced model load times and average inference times, while also decreasing app not responsive (ANR) events.

Furthermore, the migration to ExecuTorch has bolstered security through the implementation of fuzzing tests. Encouraged by these results, WhatsApp is actively migrating other key models, including those for on-device noise cancellation and video enhancement, to the ExecuTorch framework.

Messenger Secures E2EE with On-Device ML

ExecuTorch plays a vital role in enabling end-to-end encryption (E2EE) on Messenger by migrating server-side models to run locally on user devices, ensuring message privacy. A key application is the on-device language identification (LID) model, which detects text language for features like translation and content recommendations.

The move to ExecuTorch has made on-device LID considerably faster, conserving server and network resources. Meta has also deployed models for optimizing video calling quality and image cutouts on Messenger using ExecuTorch. These transitions have improved infrastructure efficiency and enabled global scalability for these features.

Facebook Surfaces Music Suggestions with ExecuTorch

Facebook’s SceneX AI model, responsible for tasks like image recognition and AI-generated backgrounds, has been adapted to run on ExecuTorch. This integration now allows SceneX to suggest background music for Facebook Stories based on image content. Early results indicate performance improvements across a range of devices compared to the previous system.

The company is also testing other ExecuTorch-powered models for Facebook, including those aimed at enhancing image quality and reducing background noise during calls. According to Statista, the number of active mobile internet users worldwide reached 5.35 billion in 2023 (Statista, 2023).

Join the ExecuTorch Development Community

Meta encourages industry-wide adoption and contribution to ExecuTorch, sharing its successful implementation in addressing large-scale on-device ML challenges. Developers are invited to contribute and provide feedback on the project’s GitHub page or join the community on the ExecuTorch Discord server.

Meta aims to foster further innovation in on-device ML and collaboratively shape the future of edge AI with the broader developer community.