1. Google brings Gemini computer use to the Mac
Google on Thursday, November 19, added Gemini computer use to its Mac app, letting the AI assistant open applications, work through menus, complete forms and move information between local files and websites on a user's behalf. The agent runs in a separate background window, and Google's own documentation warns about prompt injection, broad app access and backups that cannot always be made.
The release marks a shift in what the assistant is for. Instead of answering a question and leaving the user to carry out the steps, Gemini now performs the clicks and keystrokes itself. Google pitches the feature as a way to handle multi-step tasks that span desktop programs, files stored on the computer and the web, with the user simply describing the goal. In the company's words, users can "just describe what you need" and let Gemini handle the clicking, typing, menus and forms.
That convenience comes with a trade-off that the company itself spells out. Handing an AI agent control of desktop software means handing it access to whatever those programs can reach, and Google's support materials are candid about what can go wrong.
2. What is Gemini computer use?
Gemini computer use is a mode in which the assistant operates a Mac's software directly, the way a person would, rather than describing how to do something. It works through the same visible controls a user relies on: buttons, menus, text fields and file dialogs.
Google calls the workspace an "isolated background window." The agent does its work on a virtual screen that is separate from the main display, so the user keeps control of the cursor and can carry on with other tasks. A live preview inside the Gemini chat shows what the agent is doing, and the user can interrupt if a task heads in the wrong direction.
The separate window addresses a practical problem with desktop agents. An assistant that takes over the visible mouse pointer makes the computer hard to use while a long task is running, because any movement by the person can collide with the agent's actions. By moving the work off the main screen, Google lets the user keep working while Gemini handles a parallel workflow.
3. What Google says the agent can do
Google's examples give a sense of the intended use. One involves pulling shipping details out of PDF documents and entering them into online forms. Another covers editing a batch of photos. A third has Gemini build a presentation using figures from a data dashboard.
The company also suggests consumer tasks, such as researching prices and preparing resale listings from photos already stored on the Mac. Each of these jobs involves moving between at least two programs, often a local file and a web page, which is precisely the kind of repetitive, multi-step work that is tedious to do by hand.
The common thread is that the assistant becomes an operator rather than an adviser. A chatbot that tells a user which menu to open still leaves the work to the person. A computer-use agent opens the menu, makes the selection and moves on to the next step.
4. Who can use it and how to turn it on
The rollout is limited at launch. The feature is available in English in the United States to adult users with personal paid Google accounts. It requires a Mac with Apple Silicon, the company's in-house processors, running macOS Sequoia 15.0 or later. Work and school accounts are not supported at launch.
To start, users switch on computer use in the Mac app, open Task mode and address the agent with the prompt "@Computer use." Before Gemini can control desktop applications, Google's help center says the user must also grant two macOS system permissions: Accessibility, and Screen & System Audio Recording. Those settings are what allow any third-party software to read the screen and send input to other programs, which is why the operating system gates them behind explicit approval.
Google's support documentation adds another layer of permission inside Gemini itself. The first time the agent wants to control a particular app, it asks. The user can refuse, allow access once, or allow it on an ongoing basis.
5. The controls Google has built in
Users can set lists of allowed and blocked applications, watch the virtual screen as the agent works, and stop a task at any point. Actions Google classifies as sensitive require the user's confirmation before they go ahead.
There is an important exception. Once an app is set to "always allow," Gemini can return to it later without a fresh app-level prompt. The convenience of not being asked repeatedly also means fewer moments at which a user might notice that the agent is doing something unexpected.
These safeguards reduce the chance of an unwanted action, but they rely on the user's attention. A live preview only helps if someone is watching it, and a confirmation request only protects the user if they read it carefully before approving.
6. What happens when the agent makes a mistake?
If Gemini makes an error, the separate window does not shield a user's files or accounts. Google's documentation states that the agent can reach active sessions in desktop applications, can encounter authentication codes displayed on screen, and can take actions that some apps do not allow to be undone.
Active sessions are the logged-in states that let a person use an email client, a banking app or a messaging program without signing in each time. An agent operating inside those programs inherits the same access. Authentication codes, such as one-time passcodes used for two-step sign-in, are designed to prove that the account holder is present; an agent that can read them on screen sits inside that security boundary.
Irreversible actions are the most serious category. Sending a message, submitting a form, deleting a file or confirming a purchase may not have an undo button, so an error in one of those steps cannot always be fixed after the fact.
7. Why prompt injection is the main concern
Prompt injection is an attack in which instructions are hidden inside a document, web page or other content in order to steer an AI system into doing something its user never asked for. A human reader might skim past a line of hidden text; an AI model processing the same page may treat it as a command.
The risk grows with the agent's reach. Because Gemini can move between several applications, it could encounter hostile instructions in one program, such as a web page or a downloaded file, and then act on them in another, such as an email client or a file manager. That ability to carry information across apps is exactly what makes the feature useful, and it is also what gives an attacker more room to work with.
Google's decision to address the issue in its own documentation reflects how seriously the problem is treated across the industry. No AI developer has yet shown a complete defense against prompt injection, which is why permission prompts, confirmations and app restrictions serve as the main lines of protection.
8. Backups help, but not always
Google offers optional on-demand file backups as a safety net, but it cautions that they cannot always be created. Where a backup does work, the temporary copies remain available for 24 hours or until the user's next prompt.
Some applications may prevent their content from being backed up at all. In those cases, a mistake made by the agent inside that app has no automatic recovery path, which brings the question of irreversible actions back into focus.
The short retention window also matters. A user who notices a problem a day later, or after issuing another instruction, may find that the backup is already gone.
9. How it fits the wider race for desktop AI agents
The Mac feature gives Google's agent strategy a local dimension. The company had already introduced Gemini Spark, a task agent that runs in the cloud. Computer use on the Mac brings that kind of automation onto the user's own machine, where it can work with local files and installed programs.
Google is not alone. OpenAI's Codex, Anthropic's Claude and Kimi Work are among the tools that already offer some form of desktop control, and the category has become one of the main areas of competition among AI developers.
Apple, for its part, moved in the direction of tighter control just before Google's launch. Three days earlier, the macOS 27.3 beta introduced stricter access settings that let users limit powerful permissions to specific apps or folders and require fresh approval for each access request. The timing underlines the tension at the center of desktop agents: they are most useful when they can reach everything, and safest when they can reach very little.
10. The trade-off for Mac users
For people weighing whether to switch the feature on, the decision comes down to how much privacy and control they are willing to give up for convenience. Granting screen recording and accessibility permissions to an AI agent, and then allowing it into logged-in applications, exposes a great deal of personal information to an automated system.
Google has added permissions, confirmations, blocklists and backups to limit the damage. Its own documentation, however, makes clear that those measures are not complete, and that the risks of hidden instructions, broad access and unrecoverable actions remain part of the package. Anyone enabling it should review which apps the agent may reach and keep an eye on its work.
