GitHub Copilot can now operate desktop apps with explicit permissions

What the preview adds

Computer use gives Copilot tools for reading an application's accessibility tree and, when needed, screenshots. The agent can click controls, enter and edit text, press keys, scroll, drag, and navigate between applications. GitHub positions it for legacy and GUI-only software that lacks an API, command-line interface, or MCP integration.

The feature is available in GitHub Copilot CLI and the GitHub Copilot app on macOS and Windows. GitHub's launch examples include summarizing browser notifications, updating a presentation, and moving information through a multi-application workflow. The release is a public preview, so behavior and controls can change.

When computer use is the right tool

Use computer control when the job genuinely depends on a visual interface. A desktop-only line-of-business application, a presentation editor, or an old GUI with no integration surface can be a reasonable candidate. Describe the intended outcome, the applications involved, and constraints such as fields that must remain unchanged.

Prefer an API, MCP server, terminal command, filesystem tool, or dedicated browser tool when one can perform the same job. GitHub's documentation says these options usually return more structured information and produce more predictable results. Visual control adds ambiguity from window position, timing, focus, and application updates.

How access and stop controls work

Computer use is disabled by default. In Copilot CLI, /computer on enables it, /computer show reports its state, and /computer off disables it. In the Copilot app, the Computer Use setting provides the same switch. Enterprise administrators can disable the feature through managed settings.

Tool-permission settings decide whether Copilot asks before controlling an application. An approval can cover the current session or be saved locally as Always allow. Saved approvals apply to the CLI and app on the same computer. Permission rules that deny a tool take priority. On macOS, Accessibility permission enables control and Screen Recording provides visual context.

Run a low-risk trial before using it on real work

• Choose a reversible task in a test document or non-production account, with no payment, publishing, deletion, or customer-data access.
• Close unrelated windows and remove secrets or personal information that could appear in screenshots or the accessibility tree.
• Enable computer use for the session and approve only the application needed for the task. Avoid Always allow during the trial.
• State the desired result, allowed applications, fields that may change, and a clear point where the agent must stop for review.
• Watch the activity log and interrupt unexpected behavior by pressing Esc twice in the CLI, or Stop or Esc in the app.
• Compare the final application state with the request, record any wrong focus or repeated action, then decide whether a direct integration would be safer.

Limits and security checks

GitHub warns that interface changes, window state, timing, and non-standard controls can cause the agent to select the wrong control, enter text in the wrong location, repeat an action, or stop. A successful test on one application version does not prove the same workflow will behave consistently after an update.

On-screen instructions can also be misleading, and application windows may reveal data about you or other people. GitHub advises using computer use only with applications and tasks whose visible content you are comfortable providing as context to Copilot. Review every high-impact result, especially changes involving data, accounts, money, or other people.