Google has announced a major expansion of its Project Mariner AI system, bringing its computer control capabilities to developers through the Gemini API and Vertex AI platforms.
First unveiled in late 2024, Project Mariner represents Google's ambitious effort to transform how users interact with digital interfaces through AI agents. The system can understand and reason across information displayed on a computer screen, including text, images, code, and web forms, then autonomously navigate websites and complete complex tasks.
The latest version of Project Mariner has been significantly enhanced to run on virtual machines in the cloud, similar to agents from OpenAI and Amazon. This cloud-based approach allows users to work on other projects while Project Mariner completes tasks in the background, handling up to ten different operations simultaneously—a substantial improvement over its predecessor that ran in the browser.
Several companies are already exploring Project Mariner's potential, including automation specialists Automation Anywhere and UiPath, along with Browserbase, Autotab, The Interaction Company, and Cartwheel. These early adopters are leveraging the technology's ability to automate complex web-based workflows that previously required extensive human intervention.
Google has also implemented advanced security measures to protect against threats like indirect prompt injections, where malicious instructions might be embedded in data retrieved by AI models. According to Google, these security enhancements have significantly increased Gemini's protection rate during tool use, making Gemini 2.5 the company's most secure model family to date.
Broader developer access to Project Mariner's capabilities is scheduled for this summer, potentially revolutionizing how developers build AI applications that can control and interact with computer interfaces. The technology is also being integrated into Google Search's AI Mode, where it will initially handle tasks like purchasing event tickets, making restaurant reservations, and scheduling local appointments.