Google detailed new Android capabilities today designed to integrate its Gemini AI assistant more deeply into the operating system and third-party applications, unveiling two core approaches: AppFunctions and UI automation. The announcements build on the initial rollout of Gemini as a standalone app and signal a broader strategy to embed AI functionality across the Android ecosystem.
AppFunctions, initially previewed last year, is an Android 16 platform feature coupled with a Jetpack library. It allows app developers to expose specific functions within their applications that Gemini – or other agent apps – can access and execute directly on the device. Google positions this as a locally-executed equivalent to the Model Context Protocol (MCP) commonly used for agent and server-side tools. The company provided examples including task management, media control, cross-app workflows and calendar scheduling. For instance, a user could request, “Remind me to pick up my package at work today at 5 PM,” and Gemini would utilize AppFunctions to invoke a task management app, automatically creating a reminder with the specified details.
Samsung is among the first manufacturers to implement AppFunctions, with integration planned for the Galaxy S26 series and devices running One UI 8.5, and higher. A demonstration showcased Gemini responding to the prompt, “Show me pictures of my cat from Samsung Gallery,” by directly accessing the Gallery app and displaying the requested images within the Gemini interface, without requiring the user to switch applications. The functionality extends to multimodal interactions, accepting both voice and text input, and allows users to further utilize the retrieved images, such as sharing them via text message.
Google confirmed that Gemini is already leveraging AppFunctions to integrate with its own Google apps – Calendar, Notes, and Tasks – as well as default applications provided by original equipment manufacturers (OEMs).
Alongside AppFunctions, Google is developing a UI automation framework intended to enable agentic interactions with apps even without dedicated AppFunctions integrations. This approach allows AI assistants to intelligently execute generic tasks within installed applications, requiring no code changes from developers. The company described this as a low-effort method for extending agentic reach.
Android 17 will further expand these capabilities, according to Google. The company is currently collaborating with a select group of app developers to refine the user experience, with plans to share more details later this year regarding access to AppFunctions and UI automation for broader developer adoption. Google emphasized a focus on high-quality user experiences as the ecosystem evolves.
